A multi-modal classifier system for missense mutation pathogenicity prediction

Abstract

The present invention relates to a computer-implemented multi-module classifier method and system for providing a pathogenicity classification score of a variant of a protein of interest. The classifier comprises a sequence module based on a protein language model (PLM); a structure module based on a graph neural network (GNN); a property module; and a unified head module based on a machine learning model. The invention further relates to methods for preparing, training, and implementing the multi-module classifier system.

Description

[0001] A MULTI-MODAL CLASSIFIER SYSTEM FOR MISSENSE MUTATION PATHOGENICITY PREDICTION

[0002] FIELD OF THE INVENTION

[0003] The present invention is generally directed to protein function prediction. More specifically, the invention relates to an integrated, multi-modal classifier system for predicting the pathogenic effect of mis sense mutations.

[0004] BACKGROUND OF THE INVENTION

[0005] Predicting the pathogenicity of missense mutations is crucial for both understanding underlying genetic processes and enabling more precise clinical interventions. Despite its importance, this task remains challenging due to the diverse molecular mechanisms and functional consequences that can arise from such mutations. The prevalent use of next generation sequencing in recent years made the interpretation of Variants of Uncertain Significance (VUS) a major challenge in genetics and molecular oncology and further underscored the necessity of precise interpretation within the realm of personalized medicine.

[0006] Understanding the functional consequences of genetic mutations is at the core of both fundamental biological research and translational clinical medicine. Among various types of genetic mutations, missense variations, where a single nucleotide change results in a different amino acid in the protein sequence, stand out as particularly significant. Predicting the pathogenicity of these missense variations is crucial for diagnosing genetic diseases and somatic disorders including cancer, informing therapeutic interventions, as well as advancing the understanding of functional genomics and proteomics. However, these variations can have myriad effects on protein structure, function, and interaction, making them challenging to analyze and interpret.

[0007] Several computational tools have been developed to address the above challenges, with methods ranging from simple linear regression models to more complex machine learning algorithms, incorporating energy, functional, conservational and statistical features. While effective to some extent, these predictors often rely on limited types of data and fail to capture the full complexity of biological systems.

[0008] Accordingly, there is a need for an integrative tool which facilitates interpreting and predicting, with a clinically relevant accuracy, the pathogenic potential of missense mutations. SUMMARY OF INVENTION

[0009] The following embodiments and aspects thereof are described and illustrated in conjunction with compositions and methods which are meant to be exemplary and illustrative, not limiting in scope. In various embodiments, one or more of the above-described problems have been reduced or eliminated, while other embodiments are directed to other advantages or improvements.

[0010] In some embodiments, there is provided a computer-implemented method for providing a pathogenicity classification of a variant of a protein of interest by a multi-module classifier system, the method including: a. providing a multi-module classifier system including: i. a sequence module based on a protein language model (PLM); ii. a structure module based on a graph neural network (GNN); iii. a property module; and iv. a unified head module including a machine learning model; b. inputting into the sequence module a wild-type sequence and a variant sequence of the protein of interest, and obtaining a sequence representation vector; c. inputting into the structure module a structure or structure prediction of the wild-type sequence and a structure or structure prediction of the variant sequence, and obtaining a structure representation vector; d. inputting into the property module numerical values associated with physical properties of the protein of interest, and obtaining a property representation vector; and e. processing the sequence representation vector, the structure representation vector, and the property representation vector by the unified head module, and obtaining a pathogenicity classification score.

[0011] In some embodiments, at least one of inputting a wild-type sequence of the protein of interest, inputting a structure or structure prediction of the wild-type sequence and a structure or structure prediction of the variant, and inputting numerical values associated with physical properties of the protein of interest, is performed by at least one additional module of the multimodule classifier system.

[0012] In some embodiments, inputting a wild-type sequence of the protein of interest, inputting a structure or structure prediction of the wild-type sequence and a structure or structure prediction of the variant, and inputting numerical values associated with physical properties of the protein of interest, are performed by additional module of the multi-module classifier system.

[0013] In some embodiments, the PLM is a model selected from Ankh, ProtBert-BFD, ProteinBert, ESMlv, and ESMlb. In some embodiments, the PLM model is Ankh-large.

[0014] In some embodiments, the PLM is pre-trained on a large amount of sequence data.

[0015] In some embodiments, the variant sequence includes a single amino acid change compared to the wild-type sequence. In some embodiments, the variant sequence includes a missense mutations compared to the wild-type sequence.

[0016] In some embodiments, the GNN is selected from Graph Attention Network (GAT) and Equivariant Graph Convolution Network (EGCN).

[0017] In some embodiments, the GNN accepts a PDB structure transformed into a 3D graph representation.

[0018] In some embodiments, the structure prediction of the variant sequence is pre-calculated by a molecular dynamics tool such as FoldX, BioPython, PyMOL, MAESTRO, and DUET.

[0019] In some embodiments, the structure or structure prediction includes features selected from atom distance, direction, orientation, and position, relative to atoms of the neighboring residues.

[0020] In some embodiments, the property module is based on a model selected from a fully- connected neural network (FCN), Random Forest, and a gradient-boosting (XGBoost).

[0021] In some embodiments, the property module is based on a fully-connected neural network (FCN) model.

[0022] In some embodiments, the numerical values are selected from a free energy (AAG) difference between wild-type and mutant, a root mean square deviation (RMSD) of atoms, a BLOSUM matrix substitution values, population distribution of mutations, co-evolutionary information, and / or parameters presented in Table 1.

[0023] In some embodiments, the numerical values include pre-calculated biophysical properties of the protein of interest.

[0024] In some embodiments, the machine learning model of the unified head module is selected from a deep learning network, a support vector machine (SVM), and k-Nearest Neighbors (KNN).

[0025] In some embodiments, the unified head module includes at least one attention layer and / or at least one fully-connected layer.

[0026] In some embodiments, the sequence module output, the structure module output, and the property module output are combined in the unified head module by concatenation of the respective representation vectors.

[0027] In some embodiments, the pathogenicity classification is selected from a classification of the variant as being pathogenic or benign, and a classification of the variant as including a gain of function (GOF) or a loss of function (LOF) mutation.

[0028] In some embodiments, the multi-module classifier system is capable of distinguishing between a gain of function (GOF) and a loss of function (LOF) mutations with an accuracy of at least about 0.7, a precision of at least about 0.8, and / or an area under the curve (AUC) of a receiver operated calculation (ROC) of at least about 0.7.

[0029] In some embodiments, the representation vectors of the sequence module and / or the structure module include features pooled from more than one layer of the corresponding model.

[0030] In some embodiments, the features pooled from more than one layer relate to at least one amino acid which is different between the wild-type sequence and the variant sequence (namely, a variation position).

[0031] In some embodiments, at least one of the sequence module, the structure module, and the property module is capable of providing a classification score.

[0032] In some embodiments, the sequence module, the structure module, and / or the property module, train with labels retrieved from an annotated sequence database.

[0033] In some embodiments, the labels are ”GOF” and ”LOF” and / or ’’pathogenic” and ’’benign”, and / or provide information related to the definition of ”GOF” and ”LOF”, and / or ’’pathogenic” and ’’benign”.

[0034] In some embodiments, the multi-module classifier system further provides in step (e), in addition to the pathogenicity score, an explainability feature at an amino acid residue level, based on Gradient-weighted Class Activation Mapping (GradCAM), the explainability feature including a vector having a dimension which is the wild-type or the variant sequence length, representing, for each amino acid in the sequence, an effect of the amino acid on the pathogenicity classification.

[0035] In some embodiments, the amino acid at the specific position is the same in the wild-type and variant sequences.

[0036] In some embodiments, the method further includes displaying the explainability feature by a rendering computer software.

[0037] In some embodiments, the method further includes calibrating the multi-module classifier system.

[0038] In some embodiments, there is provided a computer-implemented multi-module classifier system for providing a pathogenicity classification of a variant of a protein of interest, the classifier including: a. a sequence module based on a protein language model (PLM) configured to accept a wild-type sequence and a variant sequence of the protein of interest as input and to provide a sequence representation vector; b. a structure module based on a graph neural network (GNN) configured to accept a structure or structure prediction for the wild-type sequence and a structure or structure prediction of the variant sequence as input and to provide a structure representation vector; c. a property module configured to accept numerical values related to physical properties of the protein of interest as input and to provide a property representation vector; and d. a unified head module based on a machine learning model, configured to accept the representation vectors of the sequence module, the structure module and the property module, and to provide a pathogenicity classification score as output.

[0039] In some embodiments, there is provided a computer-implemented method for generating a multi-module classifier system for predicting the pathogenic potential of protein variants, the method including: a. training a sequence module based on a protein language model (PLM) by inputting pairs of variant and wild-type protein sequences, each pair corresponding to a single protein, and obtaining sequence representation vectors; b. training a structure module based on a graph neural network (GNN) by inputting pairs of variant and wild-type protein structures or structure predictions, each pair corresponding to a single protein, and obtaining structure representation vectors; c. training a property module by inputting numerical values associated with physical properties of pairs of variant and wild-type proteins, each pair corresponding to a single protein, and obtaining property representation vectors; and d. processing the sequence representation vectors, the structure representation vectors, and the property module representation vectors by a unified head module based on a machine learning model, and further fine-tuning (training) the three modules together with the unified head to obtain pathogenicity classification scores.

[0040] In some embodiments, the trainings of the sequence module, the structure module, and the property module in steps (a)-(c) are conducted at a number of epochs and initial learning rate that are higher than the number of epochs and initial learning rate used for training the three models together with the unified head in step (d).

[0041] In some embodiments, training of the sequence module includes pooling representations for at least one amino acid at a variation position of a protein sequence from more than one layer, and concatenating it to a sequence representation vector corresponding to the protein.

[0042] In some embodiments, training of the structure module includes pooling representations for at least one amino acid at a variation position of a protein sequence from more than one layer, and concatenating it to a structure representation vector corresponding to the protein.

[0043] In some embodiments, the training both of the separate modules and of the modules together with the unified head further includes inputting labels retrieved from an annotated variant effect database.

[0044] In addition to the exemplary aspects and embodiments described above, further aspects and embodiments will become apparent by reference to the figures and by study of the following detailed descriptions.

[0045] BRIEF DESCRIPTION OF DRAWINGS

[0046] The invention will now be described in relation to certain examples and embodiments with reference to the following illustrative figures.

[0047] Fig. 1 : A systematic overview of the TriVIAl missense pathogenicity ensemble classifier, according to some embodiments of the invention. TriVIAl is built upon three input types: sequences, structures, and tabular physical properties, and forwards them to three independent models: PLM, GNN, and fully-connected neural network (FCN), respectively. These models create a latent representation with a unified dimension that are then concatenated and passed through either a stack of dense layers or a multi-head attention network towards a final classification layer.

[0048] Figs. 2A-2E: box plots of prediction error rate by mutation position, for each of the five model flavors. The X axis represents the mutation position, in 250 bin size. The Y axis shows the distribution of the error rate for each bucket. Fig. 2A. PLM + tabular + GAT; Fig. 2B. GAT; Fig. 2C. PLM + tabular + EGCN; Fig. 2D. EGCN; Fig. 2E. PLM. The sequence-only classifier (Fig. 2E) has a higher error rate when the mutation occurs beyond the model's context window of 1022 residues.

[0049] Figs. 3A-3E: A box plot of prediction error rate by mean AlphaFold's pLDDT. Fig. 3A. PLM + tabular + GAT; Fig. 3B. GAT; Fig. 3C. PLM + tabular + EGCN; Fig. 3D. EGCN; Fig. 3E. The structure-only classifiers have a higher error rate where the overall mean pLDDT is lower (Figs. 3B, 3D), i.e. there is opposite correlation between the mean pLDDT and the error rate of the structure models.

[0050] Fig. 4: Validation Receiver Operating Characteristics (ROC) curves for TriVIAl and other leading classifiers on the validation set (based on 9352 DMS variants). TriVIAl results are drawn with bolder line. TriVIAl and AlphaMissense yield the top performance of 0.88 AUC while other tools reach notably lower results.

[0051] Fig. 5 : Precision-Recall curves (PRC) plot on the validation set, for TriVIAl and the other top-20 classifiers. TriVIAl results are drawn with bolder lines. The scores were taken from dbNSFP. Fig. 6: ROC AUC bar comparison. TriVIAl bars are in blue, other leading classifiers bars in grey. Data same as in Fig. 4.

[0052] Figure 7: PRC AUC bar comparison. TriVIAl bars are in blue, other leading classifiers bars in grey. Data same as in Fig. 5.

[0053] Fig. 8 : Brier score bar comparison of TriVIAl with other leading classifiers. TriVIAl bars are in blue.

[0054] Fig. 9: t-stochastic neighbor embedding (t-SNE) 2D plot of the pathogenicity classifier's hidden representation, colored by LOF (blue) or GOF (orange) label. The hidden representation is taken from the last layer of the final classification, after the fusion blocks.

[0055] Fig. 10: Sequence model explanation of BRAF V600E mutation, focusing on the DGF motif area. Blue lines refer to the residues that contributed to the predicted outcome according to the protein language model, with connection width reflecting the relative contribution. The sequence model uses mainly residues from the DFG motif (594 to 596) and from the ATP binding site, such as 483, 501 and nearby residues.

[0056] Fig. 11: Structure model explanation of BRAF V600E mutation, focusing on the A-loop area. Pink lines refer to the residues that contributed to the predicted outcome according to the structure model, with connection width reflecting the relative contribution. The structure model finds mainly the residues from the activation loop (A-loop): residues 594 to 600, and the Glycine- rich loop (P-loop) - residues 464 to 471 - as contributing the most to the classifier decision.

[0057] Fig. 12: JAK2 V617F Sequence model explanation, focusing on the JH1 and JH2 domains area. Blue lines refer to the residues that contributed to the predicted outcome according to the sequence model, with connection width reflecting the relative contribution. The sequence model focuses on residues around 588-607, 683, 592, in JH1 domain, and residues around 883-885, 922- 925, 935-947 in JH2 pseudokinase domain, which contribute to the stabilization of the JH2-JH1 interaction through salt bridging or van der Waals forces.

[0058] Fig. 13: JAK2 V617F Structure model explanation, focusing on the Phenyl ring area. Pink lines refer to the residues that contributed to the predicted outcome according to the structure model, with connection width reflecting the relative contribution. The structure model finds, besides residues 523 and 570 which are crucial to the auto-inhibition process of the wild-type form, those in the Phenyl ring of residues around 594 and the residues involved in the phosphorylation of JH2 domain and thus the SH2-JH1 interaction.

[0059] Fig. 14: KRAS G12D Sequence model explanation, focusing on the switches area. Blue lines refer to the residues that contributed to the predicted outcome according to the sequence model, with connection width reflecting the relative contribution. The sequence model utilizes residues mainly from the allosteric sites in switch I and II: residues 30-40 and 58-70, respectively; and the membrane association domain - C-terminal structural element, named the hypervariable region (HVR), which plays a crucial role in anchoring RAS to the membrane in residues 180-188. In addition, it utilizes lysine residues such as K101, K104, K128 and K147 which were observed to function as acetylation sites.

[0060] Fig. 15: KRAS G12D Structure model explanation, focusing on the P-loop area. Pink lines refer to the residues that contributed to the predicted outcome according to the structure model, with connection width reflecting the relative contribution. . The structure model focuses on the P- loop, in residues 10-17, and Switch I and II regions, which undergo conformational changes upon GTP or GDP binding. The G12D mutation alters the dynamics of these regions, affecting how KRAS interacts with GTPase-activating proteins (GAPs), Guanine nucleotide exchange factors (GEFs), and downstream effectors.

[0061] Fig. 16: VHL R161Q Sequence model explanation, focusing on the P-domain area. Blue lines refer to the residues that contributed to the predicted outcome according to the sequence model, with connection width reflecting the relative contribution. The sequence model predicts the pathogenicity using residues from the P-domain of pVHL (residues 63-154 and 193-204), which are involved in the binding sites of both elongin-C and HIF.

[0062] Fig. 17: VHL R161Q Structure model explanation, focusing on the on the a-helix domain. Pink lines refer to the residues that contributed to the predicted outcome according to the structure model, with connection width reflecting the relative contribution. The structure model focuses on residues from a-helical domain (residues 155-192) which directly contacts elongin-C.

[0063] DETAILED DESCRIPTION OF THE INVENTION

[0064] In the following description, various aspects of the disclosure will be described. For the purpose of explanation, specific configurations and details are set forth in order to provide a thorough understanding of the different aspects of the disclosure. However, it will also be apparent to one skilled in the art that the disclosure may be practiced without specific details being presented herein. Furthermore, well-known features may be omitted or simplified in order not to obscure the disclosure.

[0065] While nonsense and frameshift mutations frequently result in significantly dysfunctional proteins with altered function and / or stability, the effect of missense mutations is sometimes more nuanced and its biological and / or clinical implications are not self-evident.

[0066] Most known pathogenic mutations occur in protein-coding regions of DNA and change the way proteins are made. Deciphering the protein structure therefore provides great insight into the molecular mechanisms underlying biological functions in human disease. While there have recently been major advances in the artificial intelligence-based prediction of protein structure, the determination of the biological and clinical relevance of specific mutations is not yet up to clinical standards. This challenge is of utmost medical importance when decisions, as critical as suggesting termination of pregnancy or recommending cancer-directed rational drugs, depend on the accuracy of prediction of the effect of the specific mutation. Currently, available tools are aiming to characterize the effect of a mutation on the functionality of the protein according to biochemical criteria, independent of the biological context. A specific change in protein structure can result either in loss of function (LOF) or gain-of-function (GOF) and the ability to identify the directionality of effect needs to be taken into consideration when interpreting the biological outcome of the mutation.

[0067] Recent advancements in deep learning have resulted in improving the accuracy of predictive models. Specifically, protein language transformer models, inspired by breakthroughs in natural language processing, have been applied to predict the pathogenicity of variant effect prediction (VEP) by analyzing amino acid sequences. Concurrently, structure-based graph neural networks have emerged as a powerful tool for understanding protein interactions and conformations. The critical role played by AlphaFold in advancing protein structure prediction has opened new doors for employing graph neural networks over predicted protein structural data in various protein- related tasks.

[0068] The present invention introduces Triple-modalities Variant Interpretation and Analysis (TriVIAl), a machine learning framework integrating sequence-based information, structural information, and tabular information, by incorporating three complementing modalities for improved prediction of missense mutations pathogenicity: protein language model (PLM), graph neural network (GNN) and a tabular model incorporating physical properties from the protein structure, as shown in Fig. 1. The TriVIAl ensemble's predictions compare favorably with the existing tools across various metrics, achieving an AUC-ROC of 0.887 (marginally better than AlphaMissense with a score of 0.886, and superior to metaRNN and ClinPred, with a score of 0.82, Fig. 4), a precision-recall curve (PRC) score of 0.68, and a Brier score of 0.16, and also generalizes effectively, capturing nuanced representations of the biological principles governing protein function.

[0069] The TriVIAl ensemble is also endowed with two major advantages compared to other available tools. The first is the incorporation of biological insights which allow to differentiate between Gain of Function (GOF) and Loss of Function (LOF) variants in proteins. GOF refers to mutations that enhance or introduce new functions that tend to cluster in specific hotspots, while LOF pertains to changes resulting in partial or complete loss of function that are usually dispersed. This distinction is vital for drug discovery, personalized medicine, and understanding disease mechanisms. Importantly, the advantage over other available tools is more noticeable with GOF mutations as their effect on the protein structure is less disruptive and can be misinterpreted by current variant prioritization strategies.

[0070] The second significant advantageous feature of TriVIAl is the explainability of the ensemble, provided in addition to providing classification labels. The explainability framework enables visualizing the input residues that contribute the most to the output prediction, which stands in contrast to other available Al-based pathogenicity predicting algorithms which constitute a black box for the users. This feature enables a deeper understanding of the mechanisms involved in making the mutation pathogenic or benign, and is especially important in analysis of GOF mutations, where the changes in the protein structure and stability are usually subtler and may depend on environment-related features such as binding sites and interaction with other proteins. This makes this feature a major tool in considering the clinical responsibility of the medical decision-makers using Al-based pathogenicity predictors.

[0071] It is known that not all mutations, even those that affect the protein they reside in, are equally important in treating disease. Example from the oncogenes or tumor suppressor genes such as KRAS, protein tyrosine kinase growth factor receptors, and APC, indicate that some mutations usually do not contribute to cancer, while other mutations result in malignant transformation. Furthermore, mutated NOTCH, FMS, and p53 can function as oncogenes or as a tumor suppressor genes, depending on the mutations and the type of cancer. At least in some of these cases, the activity of the mutated protein (i.e., as an oncogene or as a tumor suppressor) depends on whether the mutation is a gain or a loss of function mutation. Additionally, mutations in genetic diseases may also have different consequences depending on the context. For example, while crippling mutations in a-globin gene usually result in alpha thalassemia, an a-globin mutation was found to decrease the globin chain imbalance and attenuate the severity of anemia in beta thalassemia major patients.

[0072] It is therefore clear that the additional information incorporated into the classifier system of the invention, as well as the additional annotations and explainability provided by the classifier system can be crucial to evaluating disease and treatment options.

[0073] Mutation pathogenicity classification methods

[0074] Accordingly, in some embodiments, there is provided a computer-implemented method for providing a pathogenicity classification of a variant of a protein of interest by a multi-module classifier system, the method including: a. providing a multi-module classifier system including: i. a sequence module based on a protein language model (PLM); ii. a structure module based on a graph neural network (GNN); iii. a property module; and iv. a unified head module including a machine learning model; b. inputting into the sequence module a wild-type sequence and a variant sequence of the protein of interest, and obtaining a sequence representation vector; c. inputting into the structure module a structure or structure prediction of the wildtype sequence and a structure or structure prediction of the variant sequence, and obtaining a structure representation vector; d. inputting into the property module numerical values associated with physical properties of the protein of interest, and obtaining a property representation vector; and e. processing the sequence representation vector, the structure representation vector, and the property representation vector by the unified head module, and obtaining a pathogenicity classification score.

[0075] It is appreciated that the inputting is not limited to outside input by a user and intends to describe addition of information into the respective module. The inputting may be carried out by any suitable method and from any adequate source, for example: manually by a user through a user interface, from an outside source (or manually) via a software layer (such as an application programming interface (API)), or by a different part or module of the classifier system itself.

[0076] Furthermore, the initial input provided by an outside source may be processed by at least one additional module or process not explicitly described herein to provide the required input for the sequence, structure, and / or property modules.

[0077] The minimal information required at each instance of running the classifier is a variant sequence (amino acid or nucleic acid), or a protein / gene identifier and an identity and position of a mutation. The classifier, together with any additional modules as needed, is capable of obtaining information not provided, including, for example, retrieving a wild-type protein sequence from an outside database (based on protein / gene identifier or on a variant sequence), building a variant sequence from a wild-type sequence and a mutation, obtaining a protein structure or providing a predicted structure of wild-type and variant proteins, and obtaining numerical values associated with physical properties of the protein of interest from outside databases. It is appreciated that obtaining sequences, structure, and other attributes as well as structure predictions are common practice and a person skilled in the art would easily be able to add such modules to the classifier.

[0078] Accordingly, in some embodiments, inputting a wild-type sequence of the protein of interest (in step b), inputting a structure or structure prediction of the wild-type sequence and a structure or structure prediction of the variant (step c), and / or inputting numerical values associated with physical properties of the protein of interest (step d), are conducted by at least one additional modules of the multi-module classifier system once a variant protein has been entered into the model, without the need to manually input this information. Such modules may be based on software such as Rosetta, PROVEAN, Swiss-PDB, PyMOL, and the Schrodinger suite.

[0079] In some embodiments, the computer- implemented method further includes at least one preprocessing step prior to inputting into the modules (in steps (b), (c), and (d)).

[0080] The term “representation vector”, as used herein, relates to a multi-dimensional vector that is the output of a hidden (intermediate) layer of a neural network. Other commonly used terms for a representation vector include “feature map”, “embedding”, “latent representation” or “hidden representation”.

[0081] The sequence module

[0082] As shown in Fig. 1, The sequence module is configured to accept sequence information including a wild-type sequence and a variant sequence, tokenize the sequences, apply a PLM transformer to the tokenized sequence, and provide a sequence representation vector which contains information relevant to the pathogenicity of the variant sequence.

[0083] It is appreciated that unless indicated otherwise, all references to sequences are to amino acid sequences.

[0084] In some embodiments, the wild-type sequence and the variant sequence, are each a single sequence. In some embodiments, the wild-type sequence is a single sequence and the variant sequence may be more than one sequence. In case multiple variant sequences (or mutation positions + identities) are provided, they are inputted into the sequence module in wild-type + variant pairs.

[0085] The term “wild-type sequence”, as used herein, relates to a version of a protein sequence which is used as reference for comparing to the variant sequence, and is a functional version of a protein not associated with pathogenicity or malfunction.

[0086] It is appreciated that the wild-type version may be any version of a protein which may be used as reference for comparing to the variant version and for which sufficient information is available for training the models of the invention. The term “variant”, as used herein relaters to a version of the entity it refers to, which is highly but not completely identical to that entity. For example, a variant of a certain protein is a protein having a sequence almost identical to the wild-type protein sequence but with at least one amin acid difference.

[0087] The term “variant sequence”, as used herein, relates to a sequence similar to the wild-type sequence (preferably derived from the same protein) but including at least one amino acid change compared to the wild-type sequence (indicated herein as “a variation position”). In some embodiments, the variant sequence includes a single amino acid change compared to the wild-type sequence. In some embodiments, the variant sequence includes more than a single amino acid change compared to the wild-type sequence. The term “variant” encompasses pathogenic mutations as well as silent changes and relates both to pathogenic and to non-pathogenic variants. The changes or mutations may be missense mutations, deletions, insertions, etc.

[0088] In some embodiments, the variant includes a mis sense mutation compared to the wild-type sequence. In some embodiments, the variant includes more than one missense mutations. In some embodiments, the only difference between the wild-type sequence and the variant sequence is a missense mutation. In some embodiments, the only differences between the wild-type sequence and the variant sequence are more than one missense mutations.

[0089] The term “missense mutation” relates to a replacement of an amino acid by a different amino acid.

[0090] The term “pathogenic” is used herein to indicate that a pathogenic variant or mutation is associated with disease, or causing disease. The term “benign”, as used herein, means non- pathogenic. The term “pathogenic” or “benign”, used herein as label, is used to classify a variant.

[0091] The term “gain of function (GOF)“, as used herein, relates to a variation or mutation which results in a function or a feature that is not present in the corresponding wild-type protein. Such a function may be, for example, lack of regulation, so that the GOF mutation causes the variant carrying it to be active when the wild-type protein is not active. Such mutations are typically dominant and in the realm of cancer are usually found in oncogenes.

[0092] The term “loss of function (LOF)”, as used herein, relates to a variation or a mutation which causes the protein not to be functional. Such mutations are typically recessive and in the realm of cancer are usually found in tumor-suppressor genes.

[0093] In some embodiments, the PLM is a model selected from Ankh, ProtBert-BFD, ProteinBert, ESMlv, and ESMlb. In some embodiments, the model is Ankh-large.

[0094] In some embodiments, the PLM is pre-trained on a large amount of sequence data, such as at least 100 million records. In some embodiments, the sequence inputted into the sequence module is a complete sequence of the protein. In some embodiments, the sequence inputted into the sequence module is a partial sequence of the protein.

[0095] In some embodiments, the wild-type sequence and the variant sequence are of the same length. However, in some embodiments, the variant sequence includes small insertions or deletions (i.e., of several amino acids) with respect to the wild-type sequence.

[0096] In some embodiments, the wild-type sequence and the variant sequence have the same length. In some embodiments, the length of the wild-type sequence does not defer from the length of the variant sequence by more than about 1, 2, 3, 4, 5, or 10 amino acids. In some embodiments, the length of the wild-type sequence does not defer from the length of the variant sequence by more than about 1%, 5%, or 10%.

[0097] In some embodiments, the wild-type sequence and / or the variant sequence have a length of at least about 50, 100, 250, 500, 1000, 1500, 2000, or 2500 amino acids. In some embodiments, the wild-type sequence and / or the variant sequence have a length of at least about 1000, 1022, 1050, or 1100 amino acids.

[0098] In some embodiments, the sequence module includes a randomized (or a moving) window of the maximal tokens limit for improving predictions at positions farther than the model limit.

[0099] In some embodiments, the computer- implemented method further includes at least one preprocessing step prior to inputting into the sequence module. Such a pre-processing step may include, for example, retrieving protein sequence from a database based on the identifier or preparing a variant sequence based on a wild-type sequence and a mutation, etc.

[0100] As explained above, the initial input provided from an outside source may be processed by at least one additional module before inputting into the sequence module. For example, a user may provide a protein or a gene identifier, and the additional module will retrieve the protein sequence from a database based on the identifier, for inputting into the sequence module; the user may provide, instead of a variant sequence, a wild-type sequence and a mutation with a position in the sequence, and a module will prepare the variant sequence based on the input. Some examples are provided below in more detail.

[0101] In some embodiments, the multi-module classifier system further includes a variant generation module capable of accepting as input a wild-type protein or nucleotide sequence, a replacement amino acid or nucleotide, and a position (or coordinate) in which the amino acid or nucleotide provided replaces the corresponding amino acid or nucleotide in the wild-type sequence, and providing a variant amino acid or nucleotide sequence including the replacement. Similarly, in some embodiments, the variant generation module is capable of adding or deleting an amino acid or a nucleotide at an indicated position, or replacing several amino acids or nucleotides, to provide a variant sequence.

[0102] In some embodiments, the method further includes a step of inputting a wild-type amino acid or nucleotide sequence, at least one replacement amino acid or nucleotide, and at least one replacement position, to the variant generation module, and obtaining a variant amino acid or nucleotide sequence including the sequence replacement. The variant sequence may be used as input for the sequence module, or for further processing prior to inputting into the sequence module.

[0103] In some embodiments, the wild-type sequence and the variant sequence are provided as raw sequences, and not as a sequence alignment. In some embodiments, a sequence alignment, or a multiple sequence alignment (MSA), is not used as input to the sequence module.

[0104] The sequence representation vector is a vector that the sequence (PLM) model outputs from its intermediate layers. Its size is a parameter of the neural network definition and not related to the input length.

[0105] In some embodiments, the sequence module is stored on at least one storage device. In some embodiments, the sequence representation vector is stored on at least one storage device.

[0106] The structure module

[0107] As shown in the embodiment presented in Fig. 1, The structure module is configured to accept 3D structure information including a wild-type structure and a variant structure, generate a 3D graph representation, apply GNN to the 3D graph representation, and provide a structure representation vector which contains information relevant to the pathogenicity of the variant structure.

[0108] In some embodiments, the GNN is selected from Graph Attention Network (GAT) and Equivariant Graph Convolution Network (EGCN).

[0109] The term “structure”, as used herein, relates to a 3D structure.

[0110] In some embodiments, a PLM is not used as part of the structure model. In some embodiments, a PLM is not used for structure prediction.

[0111] The GNN accept structural input in various formats, such as PDB translated into a 3D graph representation.

[0112] As explained above, the initial input provided from an outside source may be processed by at least one additional module before inputting into the structure module. Examples include a user providing a protein identifier and an additional module retrieving an available protein structure from a database, or a user providing a variant sequence and the additional module generating a predicted structure for the provided sequence. Some examples are provided below in more detail.

[0113] In some embodiments, the computer- implemented method further includes at least one preprocessing step prior to inputting into the structure module. Such a pre-processing step may include, for example, a structure prediction. Nonlimiting examples include calculating a predicted protein structure based on a protein sequence, and calculating a predicted variant protein structure based on a wild-type protein sequence and a mutation.

[0114] Such structure predictions may be calculated by a tool such as FoldX, Rosetta, PROVEAN, Swiss-PDB, PyMOL, or the Schrodinger suite, and the calculated structure prediction is then inputted into the structure module.

[0115] In some embodiments, the structure prediction has a Predicted Local Distance Difference Test (pLDDT) of less than about 60, 70, 80, 90, or 100. In some embodiments, the structure prediction has a pLDDT of less than about 70.

[0116] In some embodiments, the method further includes inputting a protein sequence into the protein structure modelling module and obtaining a predicted structure of the sequence for inputting into the structure module. In some embodiments, the method further includes inputting a protein variant sequence into the protein structure modelling module and obtaining a predicted structure of the variant for inputting into the structure module.

[0117] In some embodiments, the structure or structure prediction include at least one feature selected from relative atom distance, direction, orientation, and position, relative to atoms of neighboring residues.

[0118] The structure representation vector is a vector that the structure (GNN) model outputs from its intermediate layers. Its size is a parameter of the neural network definition and not related to the input length.

[0119] In some embodiments, the structure module is stored on at least one storage device. In some embodiments, the structure representation vector is stored on at least one storage device.

[0120] The property module

[0121] As shown in the embodiment presented in Fig. 1, The property module is configured to accept numerical properties including wild-type and variant physical properties, normalize the numerical properties, apply FCN to the normalized properties, and provide a property representation vector which contains information relevant to the pathogenicity of the variant properties.

[0122] In some embodiments, the property module is based on a model selected from a fully- connected neural network (FCN), Random Forest, and a gradient-boosting (XGBoost). In some embodiments, the property module is based on an FCN model.

[0123] In some embodiments, the property module is based on an Random Forest model.

[0124] In some embodiments, the numerical values are inputted in a tabular format.

[0125] In some embodiments, the numerical values are numerical biophysical properties of the protein of interest.

[0126] In some embodiments, the numerical values are selected from a free energy (AAG) difference between wild-type and mutant, a root mean square deviation (RMSD) of atoms, a BLOSUM matrix substitution values, population distribution of mutations, co-evolutionary information, and / or parameters presented in Table 1.

[0127] In some embodiments, the classifier further includes a biophysical properties calculation module for pre-calculating biophysical properties and inputting the calculated properties into the property module.

[0128] In some embodiments, at least some of the numerical values are pre-calculated by molecular dynamics software such as FoldX, BioPython, PyMOL, MAESTRO, and DUET.

[0129] In some embodiments, the method further includes inputting numerical features associated with the protein of interest into the biophysical properties calculation module and obtaining calculated properties for inputting into the property module.

[0130] The properties representation vector is a vector obtained by the tabular model, an intermediate representation from a hidden layer of the fully-connected neural network that processes the properties tabular features.

[0131] In some embodiments, the property module is stored on at least one storage device. In some embodiments, the property representation vector is stored on at least one storage device.

[0132] The unified head

[0133] As shown in the embodiment presented in Fig. 1, The unified head module is configured to accept representation vectors from the sequence, structure, and property modules, concatenate and apply multi-head attention pooling to the vectors, and provide a final representation vector and a pathogenicity score.

[0134] In some embodiments, the machine learning model of the unified head is selected from a deep learning network, a support vector machine (SVM), k-Nearest Neighbors (KNN), and other suitable algorithms.

[0135] In some embodiments, the unified head includes at least one attention layer and / or at least one fully-connected layer.

[0136] The at least one attention layer and / or at least one fully -connected layer of the unified head module operate on combined output from the sequence module, the structure module, and the property module. In some embodiments, the combined output is combined by concatenating the three separate representation vectors into a combined output.

[0137] In some embodiments, the unified head module includes at least three layers. In some embodiments, the unified head module includes at least three fully-connected layers. In some embodiments, the unified head module includes at least three attention layers.

[0138] In some embodiments, the at least one attention layer is selected from self attention, soft attention, hard attention, sparse attention, and cross attention.

[0139] In some embodiments, the unified head is capable of providing at least one representation vector. The unified representation vector is a vector that one of the hidden layers of the unified head outputs.

[0140] In some embodiments, the unified head is stored on at least one storage device. In some embodiments, the at least one representation vector is stored on at least one storage device. In some embodiments, the pathogenicity classification score is stored on at least one storage device.

[0141] The pathogenicity classification score may represent any desired classification that can be based on existing data. Specifically, the present invention demonstrates using classifications of a variant as a pathogenic (i.e., disease-causing or disease-related) or benign (non-pathogenic), or classification of a variant as having a GOF or LOF mutation. The ability to provide these classification is facilitated by using appropriate labels during training of the models.

[0142] The pathogenicity classification score is a likelihood score from 0-1, when 0 represents one classification option (e.g. benign) and 1 represents the other classification (e.g. pathogenic), and the classification score represents the likelihood of the variation to be either one of the two options.

[0143] Accordingly, in some embodiments, the pathogenicity classification score provides a classification of the variant, such as a pathogenic / benign variant, or a GOF / LOF variant.

[0144] In some embodiments, the multi-module classifier system is capable of distinguishing between a gain of function (GOF) and a loss of function (LOF) mutation with an accuracy of at least about 0.5, 0.6, 0.7, or 0.8. In some embodiments, the multi-module classifier system is capable of distinguishing between a gain of function (GOF) and a loss of function (LOF) mutation with an accuracy of at least about 0.7.

[0145] In some embodiments, the multi-module classifier system is capable of distinguishing between a gain of function (GOF) and a loss of function (LOF) mutation with a precision of at least about 0.5, 0.6, 0.7, or 0.8. In some embodiments, the multi-module classifier system is capable of distinguishing between a gain of function (GOF) and a loss of function (LOF) mutation with a precision of at least about 0.8. In some embodiments, the multi-module classifier system is capable of distinguishing between a gain of function (GOF) and a loss of function (LOF) mutation with an area under the curve (AUC) of a receiver operated calculation (ROC) of at least about 0.5, 0.6, 0.7, or 0.8. In some embodiments, the multi-module classifier system is capable of distinguishing between a gain of function (GOF) and a loss of function (LOF) mutation with an area under the curve (AUC) of a receiver operated calculation (ROC) of at least about 0.7.

[0146] Architecture and training of the multi-module classifier system

[0147] In some embodiments, the architecture of the multi-module classifier system is trained via back-propagation, with an optimizer such as Stochastic Gradient Descent (SGD) or Adaptive Moment Estimation (ADAM).

[0148] In some embodiments, the architecture of the multi-module classifier system includes Feature Pyramid Network elements, as described below.

[0149] In some embodiments, the sequence module PLM and / or the structure module GNN are trained by incorporating a scheme based on the Feature Pyramid Network.

[0150] In some embodiments, a representation of an amino acid at a variation position is pooled from more than one layer. This approach enables capturing both lower-level, higher-resolution as well as higher- level, lower-resolution embedded representations. A global (full- sequence) representation, to which this scheme was not applied, was concatenated to the specific amino acid representation that was pooled from more than one layer, for both the wild-type and the variant sequences, and the concatenated vectors were forwarded to the final linear classification layers of the specific module or of the unified head module.

[0151] Accordingly, in some embodiments, the representation vectors of the sequence model and / or the structure model include features pooled from more than one layer. In some embodiments, the features pooled from more than one layer relate to at least one amino acid which is different between the wild-type sequence and the variant sequence (i.e., a variation position). In some embodiments, the features pooled from more than one layer are pulled based on the variation position. In some embodiments, a sequence representation of an amino acid at a variation position is pooled from more than one layer and concatenated to the sequence representation vector of the protein. In some embodiments, a structure representation of an amino acid at a variation position is pooled from more than one layer and concatenated to the structure representation vector of the protein.

[0152] It is appreciated that each of the three modules can also be used as stand-alone models, and can provide classifications. Accordingly, in some embodiments, at least one of the sequence module, the structure module, and the property module is also capable of providing a classification score.

[0153] However, as shown in the present application, the multi-module unified classifier provides more accurate predictions compared to the separate modules.

[0154] Training of the multi-module classifier system is done by first training the three models separately, with a relatively large number of epochs and a high initial learning rate (at least about 10’3or 10’4), and then further training of the unified model for a smaller number of epochs and with a lower learning rate (at most about 10’6or 10’7). At the unified training, the final classification layer of the original standalone models is removed, and a tensor with unified predefined dimensions is then concatenated to the representation vectors and processed through the final classification layers, including attention layers and fully connected layers operating on the concatenated vectors.

[0155] In some embodiments, the PLM sequence model, the GNN structure model, and / or the property module machine learning model further incorporate labels retrieved from an annotated variant effect database, such as ClinVar, COSMIC, or HGMD.

[0156] The labels retrieved from an annotated variant effect database may include any desired labels, and non-limiting examples include “pathogenic” and “benign” (for a pathogenic vs. benigh classification), or “GOF” vs. “LOF” (for a GOF vs. LOF classification), or labels including information related to any of the mentioned terms.

[0157] In some embodiments, the labels retrieved from an annotated variant effect database during training of the models correspond to the desired classification. For example, the labels may include information relating to pathogenic vs. benign classification or to GOF vs. LOF classification.

[0158] In some embodiments, the wild-type sequence for the training is retrieved from a sequence database such as PDB, or UniProt. In some embodiments, the variant sequences for training were obtained from annotated sequence databases including variants, such as ClinVar, HGMD, COSMIC, or other published database.

[0159] In some embodiments, the sequence inputted into the sequence module is directly retrieved from a protein sequence database. In some embodiments, the sequence inputted into the sequence module is an amino acid sequence computed from a nucleic acid sequence retrieved from a nucleic acid sequence database prior to inputting into the sequence module.

[0160] In some embodiments, the wild-type structures for the training are obtained from the protein data base (PDB).

[0161] The multi-module classifier system may further include an explainability module for providing details on the classification. The explainability module is based on Gradient-weighted Class Activation Mapping (GradCAM), and generally involves computing gradients with respect to the feature map (representation vector, or embedding) of a specific layer (that may be selected and tuned by the user), the gradients representing the effect of changes in the feature map on the class score (classification output). The gradients are passed to global average pooling and interpolated to the input dimensions to retrieve residue-level explainability scores. The output of the explainability module is a vector with same length as that of the input sequence (e.g., the wildtype sequence or the variant sequence), where each value represents the actual contribution of the corresponding residue to the output classification.

[0162] Accordingly, in some embodiments, the multi-module classifier system further provides in step (e), in addition to the pathogenicity score, an explainability feature at an amino acid residue level, based on Gradient-weighted Class Activation Mapping (GradCAM), the explainability feature including a vector having a dimension which is the wild-type or the variant sequence length, representing an effect of each amino acid at a specific position on the pathogenicity classification.

[0163] In some embodiments, the method further includes displaying the explainability feature by a rendering computer software.

[0164] As explained below, models may sometimes need to be calibrated, to align their confidence score with their performance.

[0165] Accordingly, in some embodiments, the method further includes calibration of each of the separate models or of the complete classifier.

[0166] In some embodiments, the calibration includes methods such as adjusting the output layer or applying post-processing methods like temperature scaling.

[0167] It is appreciated that although the multi-module classifier system is described herein with reference to a specific implementation of prediction pathogenicity, it may be used for other tasks. For example, one may utilize the embeddings (representation vectors) from the sequence, structure, tabular, and / or unified modules for Retrieval Augmented Generation (RAG) tasks in the proteins learning domain.

[0168] Method of preparing the multi-module classifier system

[0169] In some embodiments, there is provided a computer-implemented method for generating a multi-module classifier system for predicting the pathogenic potential of protein variants, the method including: a. training a sequence module based on a protein language model (PLM) by inputting pairs of variant and wild-type protein sequences, each pair corresponding to a single protein, and obtaining sequence representation vectors; b. training a structure module based on a graph neural network (GNN) by inputting pairs of variant and wild-type protein structures or structure predictions, each pair corresponding to a single protein, and obtaining structure representation vectors; c. training a property module by inputting numerical values associated with physical properties of pairs of variant and wild-type proteins, each pair corresponding to a single protein, and obtaining property representation vectors; and d. processing the sequence representation vectors, the structure representation vectors, and the property module representation vectors by a unified head module based on a machine learning model, and further training the three modules together with the unified head to obtain a pathogenicity classification score.

[0170] Definitions and embodiments mentioned above and which may be relevant to the embodiments in this section also apply here, and vice versa. Some particularly relevant embodiments may be pointed out or explicitly repeated. For terms used herein, unless stated otherwise, their definition and embodiments are intended to be the same as above (mutatis mutandis).

[0171] In some embodiments, the trainings of the sequence module, the structure module, and the property module in steps (a)-(c) are conducted at a number of epochs and initial learning rate that are higher than the number of epochs and initial learning rate used for training the unified multimodule classifier system in step (d).

[0172] In some embodiments, the training of the sequence module PLM and / or of the structure module GNN included a scheme based on a Feature Pyramid Network.

[0173] In some embodiments, training of the sequence module includes pooling representations for at least one amino acid at a variation position of a protein sequence from more than one layer, and concatenating it to a sequence representation vector corresponding to the protein.

[0174] In some embodiments, training of the structure module includes pooling representations for at least one amino acid at a variation position of a protein sequence from more than one layer, and concatenating it to a structure representation vector corresponding to the protein.

[0175] In some embodiments, the training of the sequence module, the structure module, and / or the property module further includes inputting labels retrieved from an annotated variant effect database, such as ClinVar, COSMIC, or HGMD

[0176] In some embodiments, the labels retrieved from an annotated variant effect database during training of the models correspond to the desired classification. For example, the labels may include information relating to pathogenic vs. benign classification or to GOF vs. LOF classification. A computer storage medium / processor containing the classifier

[0177] In some embodiments, there is provided a computer-readable non-transitory storage medium, storing a computer program thereon, wherein, when the computer program is executed by a processor, a method as described herein is implemented.

[0178] In some embodiments, there is provided a processing circuitry configured to implement the methods described herein.

[0179] In some embodiments, there is provided a system including one or more processing units configured to execute the methods described herein.

[0180] Definitions and embodiments mentioned above and which may be relevant to the embodiments in this section also apply here, and vice versa. Some particularly relevant embodiments may be pointed out or explicitly repeated. For terms used herein, unless stated otherwise, their definition and embodiments are intended to be the same as above (mutatis mutandis).

[0181] The term “storage device”, used throughout the application, relates to a computer-readable non-transitory storage device or a computer-readable non-transitory storage medium for storing any of the modules, inputs, or outputs disclosed with respect to the methods and the systems disclosed herein. Nonlimiting examples for non-transitory storage media include hard drives, solid-state drives, USB flash drives, flash memory cards, optical disks, magnetic tapes, as well as cellular phone storage.

[0182] The multi-module classifier system

[0183] In some embodiments, there is provided a computer-implemented multi-module classifier system for providing a pathogenicity classification of a variant of a protein of interest, the classifier including: a. a sequence module based on a protein language model (PLM) configured to accept a wild-type sequence and a variant sequence of the protein of interest as input and to provide a sequence representation vector; b. a structure module based on a graph neural network (GNN) configured to accept a structure or structure prediction for the wild-type sequence and a structure or structure prediction of the variant sequence as input and to provide a structure representation vector; c. a property module configured to accept numerical values related to physical properties of the protein of interest as input and to provide a property representation vector; and d. a unified head module based on a machine learning model, configured to accept the representation vectors of the sequence module, the structure module and the property module, and to provide a pathogenicity classification score as output.

[0184] The multi-module classifier system described in this section is the same multi-module classifier system described with respect to the above methods and is intended to include all of the features described in above embodiments. For terms used herein, unless stated otherwise, their definition and embodiments are intended to be the same as above (mutatis mutandis).

[0185] In some embodiments, there is provided one or more computer-readable storage medium, storing the multi-module classifier system described herein.

[0186] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the invention pertains.

[0187] The term "a" and "an" refers to one or to more than one (i.e., to at least one, or to one or more) of the grammatical object of the article. By way of example, “an element” means one element or more than one element.

[0188] The term "about" when referring to a measurable value such as an amount, a ratio, and the like, is meant to encompass variations of ±10% of the indicated value, as such variations are also suitable to perform the disclosed invention. Any numerical values appearing in the application are intended to be construed as if preceded by “about”, unless indicated otherwise.

[0189] While certain embodiments of the invention have been illustrated and described, it will be clear that the invention is not limited to the embodiments described herein. Numerous modifications, changes, variations, substitutions, and equivalents will be apparent to those skilled in the art without departing from the spirit and scope of the present invention as described by the claims, which follow.

[0190] The following examples are presented in order to more fully illustrate some embodiments of the invention. They should in no way be construed, however, as limiting the broad scope of the invention. One skilled in the art can readily devise many variations and modifications of the principles disclosed herein without departing from the scope of the invention. EXAMPLES

[0191] Methods

[0192] Principles and Rationale

[0193] Overview ofTriVIAl

[0194] TriVIAl is a hybrid deep learning model that combines three distinct data types: (1) A fully- connected neural network (FCN) which models the tabular features encapsulating numeric attributes (obtained, for example form FoldX and BioPython), such as difference in free energy, the root mean square deviation (RMSD) of the proteins' atoms, BLOSUM substitution values, and additional parameters, as detailed in Table 1; (2) A transformer-based Protein Language Model (PLM) that has been pre-trained on a massive set of sequences; and (3) a Graph Neural Network (GNN) that captures and maps the molecular 3D aspects that were introduced by the mutation. Together, these modalities complement each other to gain synergistic effect.

[0195] Tabular features, such as energy and molecular properties, as well as population distribution and co-evolutionary information, are core components of many missense pathogenicity prediction tools. While it is clear that tabular features cannot capture the complex context of the proteins, they offer well-established features that can easily be computed and offer clear prediction interpretability. For instance, a significant change in AAG can be directly linked to destabilization of a protein, providing reasoning for pathogenicity prediction. It is assumed that AAG also enables the distinction of the nature of the pathogenic mutation, especially if it Gain-of-Function or Loss- of-Function.

[0196] Protein language Models (PLMs), successors of general Large Language Models (LLMs), are powerful models that learn meaningful representations with biophysical quantitative features of proteins in unsupervised process. Protein sequence data is vastly more abundant than structural or energy and co-evolution data, enabling PLMs to learn through the largest sets of known sequences such as UniProt and UniRef datasets, without homology information or labeled properties. However, the current PLMs have several limitations: First, most of them are limited to process sequences with maximal length of ~1K amino acids, raising the question of their utility for proteins with longer sequences, especially where the mutation occurs after the length limitation. Another concern is raised from the training procedure - LLMs are trained by masking specific tokens in a sequence (words in natural language, residues in protein sequences) and learning to predict the masked token; in other words, they are trained to predict what amino acid may appear in a specific location with what likelihood; hence, their ability to fully represent all of the physical and chemical interaction is not fully proven.

[0197] Structure-based models operate on proteins represented as augmented 3D graphs with features attached to each node - either at the residue level, atom level or combined hierarchical representations - and to each edge between adjacent nodes - either predefined k nearest neighbors or all neighbors under certain distance thresholds, usually 4.5-10 Angstrom. Structures are richer in information than sequences and allow models to derive insights directly from local chemical interactions. This enables structure-based models to utilize broader, deeper and richer knowledge encapsulated in atomic or residual level of interactions and by comparing wild-type to mutation structures they can learn how the physicochemical state of the mutation is implicitly expressed by the backdrop of structure itself, rather than against phylogenetic sequence data that merely implies structure.

[0198] To take advantage of all worlds, TriVIAl is designed as a triple-backbone neural network with a unified head. An FCN backbone operates on the tabular features; a pre-trained PLM transformer backbone processes the sequence information; and a GNN is deployed on the structural data.

[0199] An overview of the system is depicted in Fig. 1.

[0200] Classification Framework Design

[0201] Tabular model

[0202] To properly evaluate the tabular features’ prediction-based utility, three independent models were evaluated: Random Forest, gradient-boosting (XGBoost), and a fully -connected 7-layers neural network (NN). All hyper-parameters such as number of trees and shape and number of layers were determined per hyper-parameters grid search of 5-fold cross validation. The full configuration details are described below under Software and Frameworks.

[0203] Although classic machine learning (ML) models, namely random forest and XGBoost on their own perform better than the NN model on the tabular data, for the combined multi-modality model which is trained by back-propagation (e.g., stochastic gradient descent (SGD)), the neural network was integrated.

[0204] Sequence model

[0205] Five flavors of Protein Language Models were evaluated: Ankh, ProtBert-BFD, ProteinBert, ESMlv, and ESMlb. For each model, the learning was frozen on all but the top 3 layers, maintaining the original configuration of layers.

[0206] During the sequence model training, each pair of wild-type and variant sequences were forwarded through the layers, producing a final embedded representation vector. Then, inspired by the concept of feature pyramid networks, the representation of the amino acid in the mutation position was pooled from several layers. This approach enabled it to capture both lower-level, higher-resolution as well as higher- level, lower-resolution embedded representations. The global (full- sequence) representation was concatenated to the amino acid representation for both the wildtype and the mutation and were forwarded to final linear classification layers. For mutations occurring past the model limit (1022 amino acids for ProtBert-BFD and ESM), a randomized window of the maximal tokens limit, containing the mutation position, was generated per forward, enabling mild data augmentation.

[0207] Structure model

[0208] For balancing between precision and complexity, ProNet’s backbone-level representation (Wang et al., 2022, ComENet: Towards Complete and Efficient Message Passing for 3D Molecular Graphs, https: / / arxiv.org / abs / 2206.08515) was followed with additional residue-level features. The backbone is represented by the Alpha Carbon (Co) atom, with the corresponding amino (-NH2) group and carboxyl (-COOH) group. Both the radius-based edges (An edge between nodes was defined for all residues with any atom existing in a radius of 6 Angstrom) and / -nearest neighbors with / =30 were tested. The node features were augmented, in addition to residue ID as a one-hot encoding of 1x20 vector, with polarity as 1x4 vector (polar, apolar, negatively charged and positively charged), and residue charge (floating point single value), similar to Reau et al., 2023 (DeepRank-GNN: a graph neural network framework to learn patterns in protein -protein interfaces. Bioinformatics 39). As edge features, Ingraham et al., 2019 (Generative Models for Graph-Based Protein Design, in Advances in Neural Information Processing Systems) was followed, and relative distance, direction and orientation were used based on the spherical coordinates between neighboring backbone nodes, in addition to relative positional embedding, effectively combining structural and positional encoding.

[0209] Two flavors of Graph Neural Networks were investigated: Equivariant Graph Convolution Network (EGCN) and Graph Attention Network (GAT). The rotation- and translation-equivariant convolution or the attention layers were used as interaction blocks that update the nodes based on the edge features and mean-based summation function with several fully-connected layers used as the global readout function. The global representation is augmented, similarly to the sequence model implementation, with node -position pooling operation which accumulates the hidden representation of the mutation position over sampled subset of the intermediate layers, followed by attention pooling layer for adaptive and contextual aggregation. The full architecture details are described below under Software and Frameworks.

[0210] For each implementation, similar to the sequence-based model, different levels of feature embedding were obtained for the mutation node and edges, through pooling from different layers. The mutation specific embedding was concatenated to the global protein-wide readout feature before the final classification layers. The Graph Attention Network model performed slightly better, though being more computationally-heavy.

[0211] Multiple Modalities Integration Considerations

[0212] Numerous strategies exist for combining several representations of data. Specifically, for protein-related tasks, existing approaches vary from simple all-numeric features to sophisticated consolidated representation deep learning. Some methods create unified tabular features, leveraging the embedding from learned deep learning models such as PLMs either as-is or after pre-processing such as principal components analysis (PCA), retaining the top N principal components. Ensemble learning offers another straight-forward tactic, with the adjustment of the weights of the predictions retrieved from the discrete models by e.g. logistic regression. A more recent approach that is successfully applied to protein-protein interface (PPI) and protein function prediction adopts PLM’s embedding for each amino acid as residue-level node feature. While generalizable and showing competitive results, the inherent limitations of PLMs such as sequence length restriction apply to some extent. An additional contemporary technique integrates protein sequence with Gene Ontology (GO) annotations, fed as either graph-based features or language model embedding.

[0213] A recent promising method, AlphaMis sense, reverses the multi-input approach and instead uses sequence, sampled variants and multiple sequence alignment (MSA) data to create a multitasking model that predicts both the mutated structure and a missense pathogenicity score.

[0214] However, the method of the present invention highlights the advantages of using diverse data sources to deeply comprehend the multifaceted origins of pathogenicity, as expressed by each modality. This, together with explainability techniques outlined in the explainability section facilitates delving into the diverse mechanisms and underlying processes responsible for the predicted result.

[0215] To demonstrate the importance of combining various modalities, the error rate, defined as the mean absolute difference between predictions and ground truth values, was calculated in relation to both mutation position and the mean value of AlphaFold's pLDDT (Predicted Local Distance Difference Test) metric. The pLDDT metric estimates the per-residue confidence in the agreement between AlphaFold predictions and experimental structures. (Error Rate = ’ where y is the ground-truth label, x is the input and f is the prediction function).

[0216] The presented findings illustrate that a model relying solely on sequence information performs worse when mutations occur outside of the model's context window limitation, e.g., ProtBert has an actual context window of 1022 amino acids, making it challenging in dealing with proteins with longer sequences (Fig. 2E). Similarly, the performance of a structure-only model degrades as pLDDT values decrease (Figs. 3B, 3D). This trend may be attributed to the increased complexity of the structural context, which appears to underlie the decreased performance observed in both AlphaFold and the GNN models. Alternatively, it illustrates that the model's quality is directly affected by the input's quality.

[0217] Data Annotation and Preparation

[0218] Pathogenicity Classification Task

[0219] The training and validation data samples were collected from ClinVar (Landrum et al., 2018 Nucleic Acids Res. 46, D1062-D1067) and the Human Gene Mutation Database (HGMD) Pro 2022.2 hg38 (Stenson et al. 2014, Hum. Genet. 133, 1-9). Only the pathogenic / damaging and benign samples were used, opting to exclude the likely benign / likely pathogenic variants to ensure a higher quality of data. The wild-type protein database (PDB) structures were obtained from AlphaFold Protein Structure Database Proteome UP000005640, and FoldX software (https: / / foldxsuite.crg.eu / ) was used to create the mutated structures.

[0220] All variant effect prediction (VEP) algorithms rank scores were annotated using dbNSFP4.4. Pre-calculated AlphaMissense ranks cores were downloaded and annotated separately. Next, the tabular features were computed either directly by FoldX or computed using BioPython package, after superimposing the wild-type and mutation structure.

[0221] The test sets were obtained from Zhang et. al. 2022 (Nat Mach Intell 4, 1017-1028) and included deep mutation scan (DMS) ensemble of four genes from different sources: BRCA1, PTEN, TP53 and MSH2.

[0222] Loss of Function (LOF) and Gain of Function (GOF) Classification Task

[0223] A dataset derived from the COSMIC - Cancer Mutation Census (Tate et al., 2019, Nucleic Acids Res. 47, D941-D947) was developed, focusing on gain-of-function (GOF) and loss-of- function (LOF) variants in cancer genes. Guided by the understanding that GOF mutations are prevalent in oncogenes and LOF mutations in tumor suppressor genes, specific selection criteria were implemented. The analysis was limited to missense mutations ('Substitution - Missense') not found in the general population ('GNOMAD_GENOMES_AF' == 0.0) and reported at least three times in the COSMIC dataset ('COSMIC_SAMPLE_MUTATED' > 2). Genes with overlapping oncogenic and tumor suppressor functions were excluded. This approach yielded a dataset of 2970 TSG-associated LOF and 3323 oncogene-linked GOF variants across 430 distinct genes.

[0224] Training Procedure

[0225] For the combined model, the curated ClinVar + HGMD dataset was divided into a stratified 80% -20% train / validation sets, respectively. The split was done at the gene level to make sure that there is no data contamination.

[0226] Each model for each data modality is initially trained as a standalone model with a large number of epochs and relatively high initial learning rate. Next, a unified fine-tuning is conducted on all backbones together for a smaller number of epochs and with a lower learning rate than the initial learning rate. At the unified training, the final classification layer of the original standalone models is removed, and instead of the last embedding layer outputs (including the single classification outputs), a tensor with unified predefined dimensions is then concatenated to the other backbones’ outputs and processed through the final classification layers, including attention layers and fully connected layers operating on the concatenated vectors.

[0227] All experiments were done on Nvidia DGX platform with 4 A100 GPUs, each having 80GB of memory. A simple parallelization approach of running each modality on a different GPU was applied.

[0228] Software and Frameworks

[0229] TriVIAl was developed using Python3.9 with the following libraries and frameworks: BioPython for sequence analysis algorithms; BioPandas for efficient protein structure parsing; Scikit-Learn, PyTorch, Deep Graph Library (DGL), and Hugging Face Transformers for the models' development and training. For full details please refer to the public repository at GitHub.

[0230] Tabular Model Description

[0231] The tabular features as described in Table 1 were first transformed by scaling column-wise to a normalized range. The tabular model consists of seven linear layers with increased number of output features, and a final output layer with a size unified across all modalities, of 256 channels. All linear layers are followed by Leaky ReLU activations, and dropout with ratio = 0.25 was applied after the 1st, 2nd, 3rd, and 6thlayers. Table 1: Protein tabular features.

[0232] Features marked with a start (*) refer to features that were computed for both the wild-type and the mutant protein Structural Model Description

[0233] TriVIAl GNN module contains several blocks for updating the nodes features and a readout function that obtains the final graph-level representation. The readout function consists of fused edges and nodes readout followed by linear layers. In addition, the global readout is enhanced with an attention layer running on the concatenated pooling of the mutation across several intermediate layers.

[0234] E(3) Equivariant Graph Convolution Network (EGCN): In the EGCN backbone, the model is built upon nine graph convolutional layers, chosen due to their E(3) equivariance properties: These layers can handle translation, rotation, and reflections, making them particularly suitable for molecular data analysis, such as protein structures. The reflection equivariance is important for correctly interpreting chiral molecules, which have non- superimposable mirror images, a common characteristic in molecular biology. The network uses residual connections and attention operations for aggregated embedding as part of the equivariant message passing.

[0235] Graph Atention Network ( GAT):

[0236] The GAT backbone uses attention mechanism to dynamically determine the importance of each neighbor's features. In addition to the original GAT proposal which uses nodes features, the model is augmented with edge attention as well. The attention layer computes the coefficients between nodes and edges; these coefficients indicate the importance of one node's and edge's features to another. For each pair of nodes, and similarly for each pair of edges, an attention score is computed, using a shared learnable linear transformation applied to their features, followed by Leaky ReLU and softmax normalization. The GAT backbone is built with six GAT blocks, each with five heads for multi-head attention.

[0237] Example 1: Missense Pathogenicity Prediction Metrics

[0238] A summary of Area Under the Curve Receiver Operating Characteristics (AUC ROC) and Precision Recall Curve (PRC AUC) of the TriVIAl model, compared to other state-of-the-art models, is presented in Figs. 4-8 for the test set (based on 9352 DMS variants based on Zhang 2022, supra). The TriVIAl combined model performs on par with AlphaMis sense, and achieves the best PRC AUC, making it specifically well-suited for clinical decisions support.

[0239] An important aspect of classification models is calibration: deep learning model calibration refers to the process of aligning the confidence of a model's predictions with the actual likelihood of those predictions being correct. In an ideally calibrated model, if it assigns a confidence level of 80% to a series of predictions, then approximately 80% of these predictions should be correct. Calibration is crucial because deep learning models, especially those in critical applications like healthcare or autonomous driving, not only need to be accurate but also trustworthy in their uncertainty estimates. Without proper calibration, a model's confidence levels may not accurately reflect the true probabilities of outcomes. This means a model could be overconfident, assigning high probabilities to predictions that are not as likely as indicated, or underconfident, assigning lower probabilities to outcomes that are more likely; both can lead to misguided decisions based on the model's output.

[0240] Calibration techniques involve adjusting the output layer or applying post-processing methods like temperature scaling, where the confidence scores are systematically adjusted to better reflect reality. This adjustment helps in ensuring that the confidence levels are meaningful and reliable, making the model more useful and trustworthy for decision-making processes.

[0241] The Brier Score, also known as the Brier Proper Score, is a scoring or performance metric used in the evaluation of probabilistic predictions made by classification models, particularly in the context of binary classification tasks. It assesses the accuracy of these predicted probabilities.

[0242] In binary classification, there are typically two possible outcomes: positive and negative, corresponding to pathogenic and benign in the present case. The Brier Score measures the mean squared difference between the predicted probabilities and the actual labels.

[0243] A lower Brier Score indicates better performance, as it means that the predicted probabilities are closer to the actual outcomes. Again, TriVIAl achieves the best Brier score amongst the other algorithms, showing that it is well-calibrated (Fig. 8).

[0244] A pivotal aspect of understanding the implications of a given mutation, is distinguishing between GOF and LOF mutations. Distinguishing between these two types is a challenging task that remains a largely unsolved issue with active research efforts.

[0245] To assess the model's ability to differentiate between GOF and LOF mutations, t-Stochastic Neighbor Embedding (t-SNE) was initially applied to project the embedding from the last classification layer into 2D space and to color them by their LOF or GOF label (Fig. 9).

[0246] The t-SNE plot indicated that, to some extent, the model is capable of distinguishing between LOF and GOF mutations. The classifier specifically was then further fine-tuned for the task of LOF vs. GOF separation as explained below in Example 2.

[0247] Example 2: LOF and GOF Prediction Metrics

[0248] The following four consequent experiments were performed in order to discriminate gain vs loss effects of the analyzed mutations. Experiments were designed to: (1) fine-tune the pathogenicity classifier on LOF vs. GOF labels on the subset from ClinVar and HGMD, with annotations taken from Sevim Bayrak et al. 2021 (Am. J. Hum. Genet. 108, 2301-2318) (1150 samples, with 80%-20% split for train and validation respectively); (2) fine-tune the pathogenicity classifier on LOF vs. GOF labels on the data derived from COSMIC (4791 samples, with 80%- 20% split for train and validation respectively); (3) test the classifier obtained from step 1 on the COSMIC derived dataset, without additional training; and (4) test the classifier obtained from step 2 on the ClinVar and HGMD derived dataset, without additional training. These results as described in Table 2 demonstrate the capability of the model to discriminate between LOF and GOF mutations, and suggest that both germline and somatic mutations exhibit molecular properties with fundamental similarity that aids with this distinction.

[0249] Table 2: LOF vs. GOF training experiments results

[0250] Example 3: Explainability

[0251] To facilitate more comprehensive understanding of the predictions from both a clinical and structural biology perspective, model explainability techniques were utilized. These techniques facilitate identifying the most impactful residues. Specifically, a generalized version of Gradient- weighted Class Activation Mapping (GradCAM)-like explainability was implemented. This involved conducting a backward pass relative to the predicted classification to calculate the gradients. Subsequently, these gradients were combined with activations to create a Class Activation Map (CAM) for each attention head. The most active input residues were then mapped onto the protein’s 3D structure, highlighting connections between target residues and the mutated residue. For visualization, the proteins were rendered using the NGLView package, adjusting the connection lines' width to reflect each residue's relative contribution. This suggested approach not only provides an interpretable justification for the predictions but also ensures that the model captures meaningful biological characteristics of the proteins and the mutations. Figs. 10-17 present visualizations that illustrate the impact of missense mutations on four proteins associated with cancer. Analyses and interpretation of these mutations, namely BRAF p.V600E, JAK2 p.V617F, KRAS p.G12D, and VHL p.R161Q, are provided in Example 4 below.

[0252] Example 4: Specific cases of missense mutations

[0253] Below are presented four pathogenic, three GOF and one LOF mutations correctly annotated by the model, demonstrating the model’s capabilities in modelling the molecular properties involved in different pathogenicity mechanisms. These modeling also provides the end user with valuable insights into the etiology of the mutations and their implications. Understanding the mechanisms of mutations also paves the way for the development of more effective and targeted treatment strategies. BRAF p.V600E

[0254] The BRAF protein takes role in intracellular signaling pathways that regulate cell growth. It is known to be mutated in several human cancers, such as melanoma, non-small-cells lung carcinoma, adenocarcinoma of the lung, and glioblastoma among others.

[0255] The V600E mutation, a substitution of Valine with Glutamic acid, causes the protein to be phosphomimetic because of the negative charge of the acidic Glutamic acid. It mimics the phosphorylation of other residues in the activation segment - specifically Threonine (T) at position 599 and Serine (S) at position 602 which activate the wild-type protein. The V600E mutation disrupts the normal interaction between the protein's Glycine -rich loop (P-loop) and the activation loop (A-loop), which in their usual state help regulate the protein's activity. The loss of BRAF inhibition control is oncogenic, leading to poor suppression of tumor growth. Figs. 10 and 11 present visualizations showing the explanations of the sequence and structure classifiers for the "Pathogenic" outcome prediction. The sequence model uses mainly residues from the DFG motif (594 to 596) and from the ATP binding, such as 483, 501 and nearby residues (Fig. 10). The structure model finds mainly the residues from the activation loop (A-loop) - residues 594 to 600, and the Glycine -rich loop (P-loop) - residues 464 to 471 - as contributing the most to the classifier decision (Fig. 11).

[0256] JAK2p.V617F

[0257] The Janus Kinase 2 (JAK2) gene encodes a key protein in the signaling pathways of several growth factors and cytokines, particularly those involved in hematopoiesis (blood cell production).

[0258] The JAK2 V617F mutation is a missense mutation where Valine (V) is substituted by Phenylalanine (F) at the 617th amino acid position in the JAK2 protein.

[0259] This mutation is strongly associated with myeloproliferative neoplasms (MPNs), a group of disorders characterized by the excessive production of one or more types of blood cells. The most common MPNs associated with the JAK2 V617F mutation include polycythemia vera, essential thrombocythemia, and primary myelofibrosis.

[0260] In the wild-type form, the JAK2 protein is activated by the binding of cytokines to their receptors, leading to the controlled production of blood cells. However, the V617F mutation results in a structural change that causes the JAK2 protein to be constitutively active, meaning it signals for blood cell production even in the absence of cytokines. This leads to the overproduction of erythrocytes. The sequence model focuses on residues around 588-607, 683, 592, in JH1; residues around 883-885, 922-925, 935-947 in JH2 pseudokinase domain, which contribute to the stabilization of the JH2-JH1 interaction through salt bridging or van der Waals forces (Fig. 12). The structure model finds, besides residues 523 and 570 which are crucial to the auto -inhibition process of the wild-type form, those in the Phenyl ring of residues around 594 and the residues involved in the phosphorylation of JH2 domain and thus the SH2-JH1 interaction (Fig. 13).

[0261] These explanations are in-line with known experimental structural biology findings.

[0262] KRASp.G12D

[0263] GTPase KRas (KRAS) is a signal transducer protein, which plays an essential role in various cellular signaling events such as in regulation of cell proliferation, differentiation and survival. KRAS G12D mutation results in an amino acid substitution at codon 12 from Glycine (G) to Aspartic acid (D), contributing to uncontrolled cell growth and cancer development. The sequence model utilizes residues mainly from the allosteric sites in switch I and II, residues 30-40 and 58- 70, respectively, and the membrane association domain - C-terminal structural element, named the hypervariable region (HVR), which plays a crucial role in anchoring RAS to the membrane in residues 180-188. In addition, it utilizes lysine residues such as K101, K104, K128 and K147 which were previously observed to be acetylation sites (Fig. 14). The structure model focuses on the P-loop, in residues 10-17, and Switch I and II regions, which undergo conformational changes upon GTP or GDP binding. The G12D mutation alters the dynamics of these regions, affecting how KRAS interacts with GTPase-activating proteins (GAPs), Guanine nucleotide exchange factors (GEFs), and downstream effectors (Fig. 15)

[0264] VHL p.R161Q

[0265] The von-Hippel-Lindau gene (VHL) is a tumor suppression gene which plays a critical role in in regulating cell growth and division. The VHL protein encoded by this gene, pVHL, is involved in the degradation of hypoxia- inducible factors (HIFs) transcription factors that respond to change in the oxygen levels within the cellular environment. Under normal oxygen conditions, VHL binds to HIFs and targets them for degradation, preventing the activation of genes that promote cell survival and angiogenesis. pVHL forms a ternary complex with the elongin C and elongin B proteins (also known as VCB complex), which is essential for pVHL stability and function.

[0266] The R161Q mutation in pVHL isoform 30 results in the substitution of Arginine (R) with Glutamine (Q) at position 161 position of the VHL amino acid sequence.

[0267] The sequence model predicts the pathogenicity using residues from the P-domain of pVHL (residues 63-154 and 193-204), which are involved in the binding sites of both elongin-C and HIF (Fig. 16) The structure model focuses on residues from a-helical domain (residues 155-192) which directly contacts elongin-C (Fig. 17).

Claims

CLAIMS1. A computer-implemented method for providing a pathogenicity classification of a variant of a protein of interest by a multi-module classifier system, the method comprising: a. providing a multi-module classifier system comprising: i. a sequence module based on a protein language model (PLM); ii. a structure module based on a graph neural network (GNN); iii. a property module; and iv. a unified head module comprising a machine learning model; b. inputting into the sequence module a wild-type sequence and a variant sequence of the protein of interest, and obtaining a sequence representation vector; c. inputting into the structure module a structure or structure prediction of the wild-type sequence and a structure or structure prediction of the variant sequence, and obtaining a structure representation vector; d. inputting into the property module numerical values associated with physical properties of the protein of interest, and obtaining a property representation vector; and e. processing the sequence representation vector, the structure representation vector, and the property representation vector by the unified head module, and obtaining a pathogenicity classification score.

2. The computer-implemented method of claim 1, wherein at least one of inputting a wild-type sequence of the protein of interest, inputting a structure or structure prediction of the wild-type sequence and a structure or structure prediction of the variant, and inputting numerical values associated with the protein of interest, is performed by at least one additional module of the multi-module classifier system.

3. The computer-implemented method of claim 1 or 2, wherein the PLM is a model selected from Ankh, ProtBert-BFD, ProteinBert, ESMlv, and ESMlb.

4. The computer-implemented method of claim 3, wherein the model is Ankh-large.

5. The computer-implemented method of any one of claims 1-4, wherein the PLM is pre-trained on a large amount of sequence data.

6. The computer-implemented method of any one of claims 1-5, wherein the variant sequence comprises a single amino acid change compared to the wild-type sequence.

7. The computer-implemented method of any one of claims 1-6, wherein the GNN is selected from Graph Attention Network (GAT) and Equivariant Graph Convolution Network (EGCN).

8. The computer- implemented method of any one of claims 1-7, wherein the GNN accepts a PDB structure transformed into a 3D graph representation.

9. The computer-implemented method of any one of claims 1-8, wherein the structure prediction of the variant sequence is pre-calculated by a molecular dynamics tool such as FoldX, BioPython, PyMOL, MAESTRO, and DUET.

10. The computer-implemented method of any one of claims 1-9, wherein the structure or structure prediction comprises features selected from atom distance, direction, orientation, and position, relative to atoms of the neighboring residues.

11. The computer-implemented method of any one of claims 1-10, wherein the property module is based on a model selected from a fully-connected neural network (FCN), Random Forest, and a gradient-boosting (XGBoost).

12. The computer-implemented method of any claim 11, wherein the property module is based on a fully-connected neural network (FCN) model.

13. The computer-implemented method of any one of claims 1-12, wherein the numerical values are selected from a free energy (AAG) difference between wild-type and mutant, a root mean square deviation (RMSD) of atoms, a BEOSUM matrix substitution values, population distribution of mutations, co-evolutionary information, and / or parameters presented in Table 1.

14. The computer-implemented method of any one of claims 1-13, wherein the numerical values comprise pre-calculated biophysical properties of the protein of interest.

15. The computer-implemented method of any one of claims 1-14, wherein the machine learning model of the unified head module is selected from a deep learning network, a support vector machine (SVM), and k-Nearest Neighbors (KNN).

16. The computer-implemented method of any one of claims 1-15, wherein the unified head module comprises at least one attention layer and / or at least one fully-connected layer.

17. The computer-implemented method of any one of claims 1-16, wherein the sequence representation vector, the structure representation vector, and the property representation vector are combined in the unified head module by concatenating the three representation vectors.

18. The computer- implemented method of any one of claims 1-17, wherein the pathogenicity classification is selected from a classification of the variant as being pathogenic or benign, and a classification of the variant as comprising a gain of function (GOF) or a loss of function (LOF) mutation.

19. The computer- implemented method of any one of claims 1-18, wherein the multi-module classifier system is capable of distinguishing between a gain of function (GOF) and a loss of function (LOF) mutations with an accuracy of at least about 0.7, a precision of at least about 0.8, and / or an area under the curve (AUC) of a receiver operated calculation (ROC) of at least about 0.7.

20. The computer- implemented method of any one of claims 1-19, wherein the representation vectors of the sequence module and / or the representation vectors of the structure module comprise features pooled from more than one layer of the respective models.

21. The computer-implemented method of claim 20, wherein the features pooled from more than one layer relate to at least one amino acid which is different between the wild-type sequence and the variant sequence.

22. The computer- implemented method of any one of claims 1-21, wherein at least one of the sequence module, the structure module, and the property module is capable of providing a classification score.

23. The computer- implemented method of any one of claims 1-22, wherein the sequence module, the structure module, and / or the property module, train with labels retrieved from an annotated sequence database.

24. The computer-implemented method of claim 23, wherein the labels are ”GOF” and ”LOF” and / or ’’pathogenic” and ’’benign”, and / or provide information related to the definition of ”GOF” and ”LOF”, and / or ’’pathogenic” and ’’benign”.

25. The computer- implemented method of any one of claims 1-24, wherein the multi-module classifier system further provides in step (e), in addition to the pathogenicity score, anexplainability feature at an amino acid residue level, based on Gradient-weighted Class Activation Mapping (GradCAM), the explainability feature comprising a vector having a dimension which is the wild-type or the variant sequence length, representing, for each amino acid in the sequence, an effect of the amino acid on the pathogenicity classification.

26. The computer-implemented method of claim 25, wherein the amino acid at the specific position is the same in the wild-type and variant sequences.

27. The computer- implemented method of claim 26, wherein the method further comprises displaying the explainability feature by a rendering computer software.

28. The computer-implemented method of claim any one of claims 1-27, wherein the method further comprises calibrating the multi-module classifier system.

29. A computer-implemented multi-module classifier system for providing a pathogenicity classification of a variant of a protein of interest, the classifier comprising: a. a sequence module based on a protein language model (PLM) configured to accept a wild-type sequence and a variant sequence of the protein of interest as input and to provide a sequence representation vector; b. a structure module based on a graph neural network (GNN) configured to accept a structure or structure prediction for the wild-type sequence and a structure or structure prediction of the variant sequence as input and to provide a structure representation vector; c. a property module configured to accept numerical values related to physical properties of the protein of interest as input and to provide a property representation vector; and d. a unified head module based on a machine learning model, configured to accept the representation vectors of the sequence module, the structure module and the property module, and to provide a pathogenicity classification score as output.

30. A computer-implemented method for generating a multi-module classifier system for predicting the pathogenic potential of protein variants, the method comprising: a. training a sequence module based on a protein language model (PLM) by inputting pairs of variant and wild-type protein sequences, each pair corresponding to a single protein, and obtaining sequence representation vectors;b. training a structure module based on a graph neural network (GNN) by inputting pairs of variant and wild-type protein structures or structure predictions, each pair corresponding to a single protein, and obtaining structure representation vectors; c. training a property module by inputting numerical values associated with physical properties of pairs of variant and wild-type proteins, each pair corresponding to a single protein, and obtaining property representation vectors; and d. processing the sequence representation vectors, the structure representation vectors, and the property representation vectors by a unified head module based on a machine learning model, and further fine-tuning (training) the three modules together with the unified head to obtain a pathogenicity classification score.

31. The computer-implemented method of claim 30, wherein the trainings of the sequence module, the structure module, and the property module in steps (a)-(c) are conducted at a number of epochs and initial learning rate that are higher than the number of epochs and initial learning rate used for training the three models together with the unified head in step (d).

32. The computer-implemented method of claim 30 or 31, wherein training of the sequence module and / or the structure module comprises pooling representations for at least one amino acid at a variation position of a protein sequence from more than one layer, and concatenating it to a sequence and / or structure representation vector, respectively, corresponding to the protein.

33. The computer- implemented method of any one of claims 30-32, wherein the training both of the separate modules and of the modules together with the unified head further comprises inputting labels retrieved from an annotated variant effect database.

Images