Cell surface protein identification method

Abstract

The present invention relates to a computer-implemented method for obtaining, by training, a tuned model configured to predict whether a protein is present at least partially on the external surface of a cell, a computer-implemented method for determining whether a protein is at least partially present on the external surface of a cell, a computer-implemented method for forming a dataset for obtaining, by training, the tuned model, and a system comprising a processor coupled to a memory and preferably a graphical user interface, the memory having recorded thereon the computer program comprising instructions for performing any of the computer-implemented methods.

Description

CELL SURFACE PROTEIN IDENTIFICATION METHODFIELD OF INVENTION

[0001] The present invention relates to deep learning approaches applied to biology, specifically to deep learning models for protein sequence analysis. In particular, this invention relates to a computer-implemented method for training a model configured to predict whether a protein is present at least partially on the external surface of a cell. The invention further relates to a computer program comprising instructions for performing the method as well as a computer readable storage medium and / or a system comprising a processor having recorded on said computer program.BACKGROUND OF INVENTION

[0002] The positioning of the protein on the external surface of the cell is intricately linked to cellular function and adaptation to the surrounding milieu. Identifying whether a protein is present at least partially on the external surface of a cell is crucial for understanding disease mechanisms, and developing targeted therapies. Surface proteins (i.e., proteins present on the external surface of a cell) play pivotal roles in cell-cell communication, signaling pathways, immune response modulation, ligand recognition, endocytosis processes and pathogen recognition. However, despite their significance, there is a notable deficiency in tools and methodologies for comprehensive surface protein identification. This gap constraints the ability to fully unravel the complexities of cellular interactions and disease processes. Addressing this limitation is paramount for advancing biomedical research and improving diagnostic and therapeutic strategies. Developing tools for surface protein identification would empower researchers to explore intricate cellular networks, uncover novel biomarkers, and accelerate the development of precision medicine approaches, ultimately benefiting human health and advancing scientific knowledge. The present invention falls within this context.SUMMARY

[0003] This invention thus relates to a computer-implemented method for obtaining, by training, a tuned model configured to predict whether a protein is present at least partiallyon the external surface of a cell, said tuned model being configured to receive as input an amino acid sequence of a protein and to output a probability for presence of at least a part of said protein represented by said amino acid sequence on the external surface of a cell, said method comprising: obtaining: o a training datasetcomprising a plurality of entries each comprising an amino acid sequence representative of a protein and a corresponding indication of whether or not said protein is present at least partially on the external surface of a cell; and o a first protein language model (PLM) being configured to receive as input an amino acid sequence of any length and to output a probability for presence, on the external surface of a cell, of at least a part of the input protein represented by the input amino acid sequence, and tuning the obtained first PLM (i.e. pretrained PLM) by: o determining a plurality of values for a set of one or more training hyperparameters; and o for each value of said plurality of values for said set, tuning the first PLM using the obtained training dataset and the values for said set of one or more training hyperparameters (thereby obtaining a plurality of tuned PLMs).

[0004] Advantageously, the method for obtaining, by training, a tuned model using a training dataset as described in the present disclosure allows for processing amino acid sequences of any length and outputting a probability for presence of at least a part of the protein comprising a specific amino acid sequence on the external surface of any eukaryotic cell. Obtaining this information about the presence of the protein on the external surface of a cell is essential for understanding underlying biological processes, disease mechanisms, and developing targeted therapies in which the protein of interest might be involved. According to other advantageous aspects of the invention, the method for obtaining, by training, a tuned model comprises one or more of the features described in the following embodiments, taken alone or in any possible combination.

[0005] According to one or several embodiments, obtaining the first PLM comprises: receiving a pre-trained PLM configured to receive as input an amino acid sequence and to output a representation of the protein represented by the input amino acid; adapting the received pre-trained PLM so as to receive as input an amino acid sequence of any length, and adding a classification head to output said probability for presence of at least a part of said protein on the external surface of a cell.

[0006] According to one or several embodiments, the set of one or more training hyperparameters comprises one or more of: a learning rate, a batch size, and one or more parameters for inputting an amino acid sequence to the first PLM.

[0007] In one example, the set of one or more training hyperparameters comprises one or more of: a learning rate, a batch size, and one or more parameters for inputting an amino acid sequence to each tuned PLM.

[0008] According to one or several embodiments, the first PLM is configured to take as input the amino acid sequence of any length in the form of a plurality of portions, said one or more parameters for inputting comprising: a chunk size representing a size of said portions of said input amino acid sequence, and an overlap size representing an overlap between two consecutive portions of said plurality of portions of said input amino acid sequence.

[0009] In one example, each tuned PLM takes as input the amino acid sequence of any length in the form of a plurality of portions, said one or more parameters for inputting comprising: a chunk size representing a size of said portions of said input amino acid sequence, andan overlap size representing an overlap between two consecutive portions of said plurality of portions of said input amino acid sequence.

[0010] This invention further relates to a computer- implemented method for determining whether a protein is at least partially present on the external surface of a cell, the method for determining whether a protein is at least partially present on the external surface of a cell comprising: providing: o a tuned model configured to predict whether a protein is present at least partially on the external surface of a cell, said tuned model comprising one or more tuned PLMs of a plurality of the tuned PLMs, each tuned according to the method for training of the present invention; and o an amino acid sequence (of any length) representative of said protein; and determining a probability for presence of at least a part of said protein on the external surface of a cell by computing a probability using said tuned model.

[0011] According to one or several embodiments, the determining of said probability for presence comprises: computing one or more intermediate probabilities, each using a respective tuned PLM of the provided one or more tuned PLMs; determining said probability for presence as a function of said one or more computed intermediate probabilities.

[0012] According to one or several embodiments, the function comprises one or more of an average function, a max function, and a min function.

[0013] According to one or several embodiments, each tuned PLM takes as input the amino acid sequence of any length in the form of a plurality of portions, said plurality of portions being defined by: a chunk size representing a size of each portion of plurality of portions forming said input amino acid sequence, and an overlap size representing an overlap between two consecutive portions of said plurality of portions of said input amino acid sequence.

[0014] This invention further relates to a computer-implemented method for forming a dataset (i.e. training dataset) for obtaining, by training, a tuned model configured to predict whether a protein is at least partially present on the external surface of a cell, said method comprising: providing: o a first dataset of amino acid sequences in relation to cells (i.e., a first dataset of amino acid sequences of proteins which may be at least partially present on the external surface of cells); and o one or more second datasets with experimental validation of protein subcellular localizations; extracting: o a first set of proteins (i.e., a plurality of amino acid sequences, each amino acid sequence being associated to a protein) from the first dataset, using a first group of keys, the first group of keys indicating if a protein is at least partially present on the external surface a cell; and o a second set of proteins (i.e., a plurality of amino acid sequences, each amino acid sequence being associated to a protein) from the first dataset, using a second group of keys, the second group of keys indicating if a protein is not on the external surface of a cell; filtering: o the first extracted set based on the provided one or more second datasets; and o the second extracted set based on the provided one or more second datasets; clustering the filtered first extracted dataset and the filtered second extracted datasets based on an identity of two amino acid sequences in said datasets, thereby obtaining a first clustered dataset and a second clustered dataset; and forming the training dataset by aggregating the first clustered dataset and the second clustered dataset.

[0015] According to one or several embodiments, said method for forming a dataset comprises: providing: o a first dataset of amino acid sequences of proteins which may be at least partially present on the external surface of cells; and o one or more second datasets with experimental validation of protein subcellular localizations; extracting: o a first set of amino acid sequences of proteins from the first dataset, using a first group of keys, the first group of keys indicating if a protein is at least partially present on the external surface a cell; and o a second set of amino acid sequences of proteins from the first dataset, using a second group of keys, the second group of keys indicating if a protein is not on the external surface of a cell; filtering: o the extracted first set based on the provided one or more second datasets; and o the extracted second set based on the provided one or more second datasets; clustering the filtered extracted first set and the filtered extracted second set based on an identity of two amino acid sequences in said filtered extracted first and second sets, thereby obtaining a first clustered dataset and a second clustered dataset; and selecting a representant of each cluster from the first clustered dataset and selecting a representant of each cluster from the second clustered dataset and forming the training dataset using said selected representants.

[0016] Advantageously, the training dataset allows obtaining, by training, a tuned model configured to predict whether a protein is at least partially present on the external surface of a cell. In other words, the training dataset is adapted to obtain, by training, a tuned model to distinguish a cell surface protein (i.e. protein with at least one portion present on the external surface of a cell) from other proteins, and that for a protein of any length without the need for truncation of the protein. Furthermore, the training dataset (notablythe first group of keys) allows distinguishing cell surface proteins from other proteins, not cell membrane proteins (i.e., proteins that do not necessarily have an extracellular portion) from other proteins. The specific prediction of extracellular portion presence in a protein is of utmost importance for the application of such models in gene and cell therapy research.

[0017] According to one or several embodiments, at least one of the one or more second datasets comprises a confidence value for each entry, the confidence value representing a score of said entry being at the cell membrane.

[0018] According to one or several embodiments, the first dataset is extracted in part from UniProt.

[0019] In addition, the disclosure relates to a computer program comprising instructions for performing any of the methods compliant with any of the above execution modes when the program is executed by a processor.

[0020] The present disclosure further relates to a non-transitory tangible computer readable storage medium, tangibly embodying a computer program comprising instructions for performing any of the methods compliant with any of the above execution modes.

[0021] Such a non-transitory program storage device can be, without limitation, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor device, or any suitable combination of the foregoing. It is to be appreciated that the following, while providing more specific examples, is merely an illustrative and not exhaustive listing as readily appreciated by one of ordinary skill in the art: a portable computer diskette, a hard disk, a ROM, an EPROM (Erasable Programmable ROM) or a Flash memory, a portable CD-ROM (Compact-Disc ROM).

[0022] The present disclosure further relates to a system comprising a processor coupled to a memory (and preferably a graphical user interface), the memory having recorded thereon the computer program comprising instructions for performing any of the methods compliant with any of the above execution modes.DEFINITIONS

[0023] In the present invention, the following terms have the following meanings:

[0024] The terms “adapted” and “configured” are used in the present disclosure as broadly encompassing initial configuration, later adaptation or complementation of the present device, or any combination thereof alike, whether effected through material or software means (including firmware).

[0025] The term “processor” should not be construed to be restricted to hardware capable of executing software, and refers in a general way to a processing device, which can for example include a computer, a microprocessor, an integrated circuit, or a programmable logic device (PLD). The processor may also encompass one or more Graphics Processing Units (GPU), whether exploited for computer graphics and image processing or other functions. Additionally, the instructions and / or data enabling to perform associated and / or resulting functionalities may be stored on any processor- readable medium such as, e.g., an integrated circuit, a hard disk, a CD (Compact Disc), an optical disc such as a DVD (Digital Versatile Disc), a RAM (Random- Access Memory) or a ROM (Read-Only Memory). Instructions may be notably stored in hardware, software, firmware or in any combination thereof.

[0026] “Machine learning (ML)” designates in a traditional way computer algorithms improving automatically through experience, on the ground of training data enabling to adjust parameters of computer models through gap reductions between expected outputs extracted from the training data and evaluated outputs computed by the computer models.

[0027] A “hyper-parameter” presently means a parameter used to carry out an upstream control of a model construction, such as a remembering-forgetting balance in sample selection or a width of a time window, by contrast with a parameter of a model itself, which depends on specific situations. In ML applications, hyper-parameters are used to control the learning process.

[0028] “Datasets” are collections of data used to build an ML mathematical model, so as to make data-driven predictions or decisions. In “supervised learning” (i.e. inferring functions from known input-output examples in the form of labelled training data), threetypes of ML datasets (also designated as ML sets) are typically dedicated to three respective kinds of tasks: “training”, i.e. fitting the parameters, “validation”, i.e. tuning ML hyperparameters (which are parameters used to control the learning process), and “testing”, i.e. checking independently of a training dataset exploited for building a mathematical model that the latter model provides satisfying results.

[0029] A “neural network (NN)” designates a category of ML comprising nodes (called “neurons”), and connections between neurons modeled by “weights”. For each neuron, an output is given in function of an input or a set of inputs by an “activation function”. Neurons are generally organized into multiple “layers”, so that neurons of one layer connect only to neurons of the immediately preceding and immediately following layers. The above ML definitions are compliant with their usual meaning, and can be completed with numerous associated features and properties, and definitions of related numerical objects, well known to a person skilled in the ML field. Additional terms will be defined, specified or commented wherever useful throughout the following description. A “protein language model (PLM)” is a computational model designed to generate and understand sequences of amino acids in proteins. A PLM utilizes machine learning algorithms to learn patterns and relationships within protein sequences. These models can predict protein structures, functions, interactions, and properties based on the sequences of amino acids.

[0030] “Peptide, polypeptide, protein” are used interchangeably, and refer to a compound comprised of amino acid residues covalently linked by peptide bonds. A protein or peptide must contain at least two amino acids, and no limitation is placed on the maximum number of amino acids that can comprise a protein's or peptide's sequence. Polypeptides include any peptide or protein comprising two or more amino acids joined to each other by peptide bonds. As used herein, the term refers to both short chains, which also commonly are referred to in the art as peptides, oligopeptides and oligomers, for example, and to longer chains, which generally are referred to in the art as proteins, of which there are many types. “Proteins” include, for example, biologically active fragments, substantially homologous polypeptides, oligopeptides, homodimers, heterodimers, variants of polypeptides, modified polypeptides, derivatives, analogs,fusion proteins, among others. A polypeptide includes a natural peptide, a recombinant peptide, or a combination thereof.

[0031] A “cell surface protein ”, or also called “protein at least partially present on the external surface of a cell”, as used herein, is intended to refer to such protein which is at least partially present at the external surface of the corresponding cell. The term “external surface” refers to the part of the cell membrane (plasma membrane, a phospholipid bilayer membrane which is found in eukaryotic cells, prokaryotic cells and most archaebacteria) which is in contact with the extracellular space. Unless stated otherwise, the term may thus encompass, without limitation, any of such proteins having at least one amino acid, or a plurality of amino acids (e.g. 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 15, 20, 25, 30, 35, 40, 45, 50 or more than 50 amino acids), in contact with said external surface e.g. proteins characterized by at least one of the following:- the presence of one or more transmembrane domains, for example 1, 2, 3, 4, 5, 6, 7 or more than 7 transmembrane domains; or- the lack of transmembrane domains, for example membrane-associated proteins which do not possess a transmembrane domain but are characterized by their attachment to the external surface of the lipidic bilayer by non-covalent interactions with hydrophobic residues.

[0032] It will be understood by the skilled in the art that, conversely, all membrane- associated proteins, membrane proteins or even transmembrane proteins are not necessarily cell surface proteins. This may, for example, be the case that such proteins are not considered herein as cell surface proteins, when referring to strictly mitochondrial proteins, vesicle membrane-associated proteins or the like which are not necessarily present at the cell surface, and accessible to the surrounding environment, in a physiological manner. It may also be the case that some proteins are associated to the cell membrane, but are solely accessible to the internal surface of the corresponding cell (e.g. strictly cytosolic proteins).BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The present disclosure will be better understood, and other specific features and advantages will emerge upon reading the following description of particular and non-restrictive illustrative embodiments, the description making reference to the annexed drawings wherein:

[0034] Figure 1 is a flow chart showing steps of a computer-implemented method for obtaining, by training, a tuned model according to one or several embodiments compliant with the present disclosure;

[0035] Figure 2 is a flow chart showing steps of a computer-implemented method for determining whether a protein is at least partially present on the external surface of a cell according to one or several embodiments compliant with the present disclosure;

[0036] Figure 3 is a flow chart showing steps of a computer-implemented method for forming a dataset for obtaining, by training, a tuned model according to one or several embodiments compliant with the present disclosure;

[0037] Figure 4 shows an apparatus integrating the functions of at least one of the following devices: a device for obtaining, by training, a tuned model, a device for determining whether a protein is at least partially present on the external surface of a cell, and / or a device for forming a dataset according to one or several embodiments compliant with the present disclosure;

[0038] On the figures, the drawings are not to scale, and identical or similar elements are designated by the same references.ILLUSTRATIVE EMBODIMENTS

[0039] The present description illustrates the principles of the present disclosure. It will thus be appreciated that those skilled in the art will be able to devise various arrangements that, although not explicitly described or shown herein, embody the principles of the disclosure and are included within its scope.

[0040] All examples and conditional language recited herein are intended for educational purposes to aid the reader in understanding the principles of the disclosure and the concepts contributed by the inventor to furthering the art, and are to be construed as being without limitation to such specifically recited examples and conditions.

[0041] Moreover, all statements herein reciting principles, aspects, and embodiments of the disclosure, as well as specific examples thereof, are intended to encompass both structural and functional equivalents thereof. Additionally, it is intended that such equivalents include both currently known equivalents as well as equivalents developed in the future, i.e., any elements developed that perform the same function, regardless of structure.

[0042] Thus, for example, it will be appreciated by those skilled in the art that the block diagrams presented herein may represent conceptual views of illustrative circuitry embodying the principles of the disclosure. Similarly, it will be appreciated that any flow charts, flow diagrams, and the like represent various processes which may be substantially represented in computer readable media and so executed by a computer or processor, whether or not such computer or processor is explicitly shown.

[0043] The functions of the various elements shown in the figures may be provided through the use of dedicated hardware as well as hardware capable of executing software in association with appropriate software. When provided by a processor, the functions may be provided by a single dedicated processor, a single shared processor, or a plurality of individual processors, some of which may be shared.

[0044] It should be understood that the elements shown in the figures may be implemented in various forms of hardware, software or combinations thereof. Preferably, these elements are implemented in a combination of hardware and software on one or more appropriately programmed general-purpose devices, which may include a processor, memory and input / output interfaces.

[0045] The present disclosure will be described in reference to a particular functional embodiment of a computer-implemented method 1 for obtaining, by training, a tuned model 51 (i.e., a deep learning model), as illustrated on Figure 1.

[0046] The tuned model 51 may be configured to predict whether a protein is present at least partially on the external surface of a cell.

[0047] The tuned model 51 may be configured to receive as input an amino acid sequence of a protein and to output a probability for presence of at least a part of the protein comprising the amino acid sequence on the external surface of a cell.

[0048] The tuned model 51 may be adapted to threshold the probability value indicating the presence of at least a part of the protein comprising the amino acid sequence on the external surface of a cell and to output a binary value of : 1 to indicate the presence of at least a part of the protein on the external surface of the cell, or 0 to indicate the absence of the protein on the external surface of the cell.

[0049] The protein may reside permanently on the external surface of the cell as an integral component of the cell membrane, fulfilling crucial roles such as signaling, transport, and cell-cell recognition. In other cases, the protein may transiently reside on the external surface of the cell during cellular events like endocytosis and exocytosis.

[0050] The presence of at least a part of the protein (i.e. a sequence of amino acids) comprising the amino acid sequence on the external surface of the cell means that the other parts of the protein (the other parts of the proteins are amino acids sequences encoding parts of the protein that are not on the external surface of the cell) may be:- encoding an extracellular part of the protein and / or and extracellular part of the protein;- localized on the external surface of the cell, or on the internal surface of the cell.

[0051] The computer- implemented method 1 for obtaining, by training, a tuned model 51 may comprise a first step 11 of obtaining a training dataset 42, the dataset comprising a plurality of entries each comprising an amino acid sequence representative of a protein and a corresponding indication of whether or not said protein is present at least partially on the external surface of a cell; and obtaining a first protein language model (PLM) 41 being configured to receive as input an amino acid sequence of any length and to output a probability for presence, on the external surface of a cell, of at least a part of the input protein represented by the input amino acid sequence.

[0052] By an amino acid sequence representative of a (respective) protein it is meant that said amino acid sequence uniquely defines the (respective) protein. In other words, a protein is in a one-to-one relationship to its representative amino acid sequence.

[0053] The training dataset 42 may comprise an ensemble of amino acid sequences in which each amino acid sequence is representative of a protein. For each amino acid representative of a protein, a corresponding indication of whether or not said protein is present at least partially on the external surface of the cell may be associated. The corresponding indication may be a probability value, a binary value or a YES / NO indication (see table 1).Table 1: Correspondence between an amino acid sequence representative of a protein and an indication of whether or not the protein is present at least partially on the external surface of the cell.

[0054] For example, an entry in the training dataset may be a whole amino acid sequence, said whole amino acid sequence being the (global) sequence of amino acids determining said protein.

[0055] According to one embodiment, obtaining the first PLM 41 may comprise receiving a pre-trained PLM configured to receive as input an amino acid sequence and to output a representation of a protein comprising the input amino acid.

[0056] The received pre-trained PLM may be an ESM-2 (Evolutionary Scale Model), a ProtBERT (Bidirectional Encoder Representations from Transformers) or any other type of PLM that may be configured to receive as input an amino acid sequence. These pretrained PLM are pre-configured to receive as input an amino of predefined length.

[0057] Hence, according to this embodiment, the method is further configured to obtain the first PLM 41 by adapting the architecture of the received pre-trained PLM so as to be able to receive as input an amino acid sequence of any length, and adding a classification head to output said probability for presence of at least a part of said protein on the external surface of a cell.

[0058] The method 1 also comprises a second step 12 comprising tuning the obtained first PLM by: determining a plurality of values for a set of one or more training hyperparameters; and for each value of said plurality of values for said set, tuning the first PLM 41 using (e.g., at least a portion of) the obtained training dataset 42 and the values for said set of one or more training hyperparameters (thereby obtaining a plurality of tuned PLMs 51).

[0059] In other words, the tuning of the obtained first PLM may comprise using a transfer learning approach.

[0060] The tuned PLMs 51 correspond to the first PLM 41 that has been tuned / trained using the training dataset 42 therefore obtaining several versions of the tuned first PLM 41.

[0061] The set of one or more training hyperparameters may comprise one or more, in some cases all, of: a learning rate, a batch size (i.e., the number of training instances in the batch), and one or more parameters for inputting an amino acid sequence to each tuned PLM.

[0062] According to one or several embodiments, each tuned PLM advantageously takes as input the (i.e., whole) amino acid sequence of any length in the form of a plurality of portions, thanks to the one or more parameters for inputting that may comprise: a chunk size representing a size of said portions of said input amino acid sequence, andan overlap size representing an overlap between two consecutive portions of said plurality of portions of said input amino acid sequence.

[0063] In other words, the tuned PLM are configured to take as input the whole amino acid sequence of any length, wherein the data representing the whole amino acid sequence is first chunked if needed into at least two portions and secondly structured into a matrix. In one example, if the whole amino acid sequence length is smaller than the chunk size, chunking the amino acid sequence may not be necessary.

[0064] For example, an amino acid sequence may have a length of “N” (N being a positive integer representing the number of amino acids in the whole amino acid sequence) and may be divided into portions of smaller length. The length of the smaller portions (i.e. chunk size) may be constant or variable. In one example, the chunk size is constant and equal to “n” (n being a positive integer), where n<N. N may be a multipleN of n, meaning that the whole amino acid sequence may be represented by - portions of equal length n. If N is not a multiple of n, the whole amino acid sequence may be represented by portions of equal length n, and one portion of N — n isN the floor function applied to the ratio -. For example, n may be equal to 128, 256, or 512.

[0065] For example, a matrix may be used to store the information about the content of the portions of the whole amino acid sequence. The matrix may have a variable dimension. In an example, the dimensions of the matrix may correspond to the number of portions (for the number of rows) and to the chunk size (for the number of columns), or inversely. If the length of a portion is smaller than the chunk size, the portion in question may be padded (e.g., zero-padded).

[0066] The whole amino acid sequence may be divided into portions each of length n. In case the length N of the whole amino acid sequence is not divisible by n, it may be possible to obtain portions of the same length n by padding. For example, the first portion may be processed independently since there's no prior segment to overlap with. For the second portion of the amino acid sequence, an overlap of “b” amino acids may be maintained with the preceding segment (i.e. the first portion). When transitioning to thesecond portion, instead of commencing from the very next amino acid after the previous portion’s end (i.e. the end of the first portion, or the last amino acid of the first portion), a shift forward by “n-b” amino acids may be applied. This ensures that the last “b” amino acids of the prior portion (e.g., first portion) overlap with the initial “b” amino acids of the subsequent portion (e.g., second portion). This may be expressed as follows: the beginning of the i'11portion may be determined by: start; = (i-l).(n-b)+l; the end of the i'11portion may be determined by: end; = start; + n - 1.

[0067] Indeed, using the chunk size approach for dividing the whole amino acid sequence has many advantages like: allowing handling of sequences of varying lengths; reducing the memory footprint required for analysis particularly when dealing with limited memory resources or when processing multiple sequences simultaneously; and identifying local features within a sequence.

[0068] Furthermore, overlapping the portions of the amino acid sequence may allow to: capture information spanning across adjacent segments (i.e. portions), ensure that each portion retains some context from the preceding portion, and capture long-range dependencies in an amino acid sequence.

[0069] For each set of one or more training hyperparameters, a plurality of values may be determined and for each value of the plurality of values for each set, the first PLM 41 is tuned resulting in a plurality of tuned PLMs corresponding to the first PLM 41.

[0070] In other words, the tuned model 51 obtained with the computer-implemented method 1, may comprise one or more tuned PLMs.

[0071] For example, ensemble averaging may be applied to obtain the tuned model 51 with the computer-implemented method 1. The one or more PLMs may be trained using at least a portion of the obtained training dataset and at least two different sets of one or more training hyperparameters. Two sets are considered different if they have at least one training parameter with different values between them. Each PLM from the one or more PLMs may be used to output a prediction (i.e. probability for presence, on the externalsurface of the cell, of at least a part of the input protein). The predictions from the one or more PLMs may then be combined using ensemble averaging which involves averaging the predictions (i.e. the probabilities output from all the PLMs of the one or more PLMs). Advantageously, by applying ensemble averaging to the obtained plurality of tuned PLMs 51, the overall performance and robustness of the tuned model 51 may be enhanced, and consequently the tuned model 51 may provide improved predictions for the presence of the protein on the external surface of the cell.

[0072] For instance, the performance of the obtained tuned model 51 may be assessed using a performance measure such as: accuracy, recall, and / or precision (see table 2).Table 2: examples of performance assessment measurements (TP: True Positive, FP: False Positive, TN: True Negative, FN: False Negative).

[0073] For example, the representation of the protein may be an embedding, i.e. a mathematical representation of the amino acid sequence in a high-dimensional vector space. For example, with the ESM-2, the abstract representation (i.e. the representation of the protein) corresponds to a specific layer in the network of ESM-2.

[0074] The added classification head may be a feedforward neural network composed of one or more layer(s) of neurons, taking as input the abstract representation of the PLM and outputting the cell surface probability.

[0075] The method 1 for obtaining, by training, a tuned model may further comprise a validation step for evaluating the performance of the machine learning tuned model. Such a validation step may comprise using a K-fold cross validation approach: partitioning the training dataset into K equally sized folds, training the model on K-l folds, and evaluating it on the remaining fold. This process is repeated K times, with each fold serving as the validation set exactly once, and the results are averaged to obtain a robust estimate of the tuned model's performance.

[0076] The present disclosure further relates to a computer- implemented method 2 for determining whether a protein is at least partially present on the external surface of a cell (as illustrated on Figure 2), the method 2 comprising: providing 21 (i.e., obtaining) (i.e. step 21): o a tuned model 51 configured to predict whether a protein is present at least partially on the external surface of a cell, said tuned model 51 comprising one or more tuned PLMs of a plurality of the tuned PLMs, each tuned according to the method for training a tuned model 51 according to any the herein presented embodiments; and o an amino acid sequence 43 representative of said protein; and determining 22 (i.e. step 22) a probability 61 for presence of at least a part of said protein on the external surface of a cell by computing a probability using said tuned model 51.

[0077] According to one or several embodiments, in the method 2 for determining whether a protein is at least partially present on the external surface of a cell further, determining 22 the probability 61 for presence may further comprise: computing one or more intermediate probabilities, each using a respective tuned PLM of the provided one or more tuned PLMs; determining the probability 61 for presence as a function of said one or more computed intermediate probabilities.

[0078] In other words, the function of the one or more computed intermediate probabilities may comprise one or more of the following: average function (i.e. arithmetic mean): this operation calculates the arithmetic mean of the intermediate probabilities, thus the final probability for presence is calculated as the average of the intermediate probabilities; weighted average function: each intermediate probability is assigned a weight, the weights may be determined empirically or based on domain expertise; the final probability for presence is then calculated as a weighted average of the intermediate probabilities. For example, if one PLM has been known to be more accurate than others, it could be assigned a higher weight in the calculation; whereas a less accurate PLM may be assigned a lower weight;maximum (i.e. max) function: the final probability for presence is calculated as the maximum value among the intermediate probabilities; minimum (i.e. min) function: the final probability for presence is calculated as the minimum value among the intermediate probabilities.

[0079] The present disclosure further relates to a computer-implemented method 3 for forming a (training) dataset 42 for training a tuned model 51 configured to predict whether a protein is at least partially present on the external surface of a cell (as illustrated on Figure 3).

[0080] The method 3 may comprise a first step 31 for providing (i.e., obtaining) a first dataset 44 for amino acid sequences in relation to cells and one or more second datasets 45 with experimental validation of protein subcellular localizations.

[0081] The first dataset 44 for amino acid sequences in relation to cells may comprise a plurality of proteins, each protein being associated to its corresponding amino acid sequence and the one or more cells to which is related (e.g., wherein all types of eukaryotic cells are taken into consideration).

[0082] The one or more second datasets 45 may comprise a plurality of proteins (i.e., each protein being associated to its corresponding amino acid sequence), each protein being associated with information relating to experimental validation of protein subcellular localizations. This one or more second datasets 45 is introduced in the present method for reproducibility validation and / or robustness reinforcement of the first dataset 44.

[0083] The method 3 may further comprise extracting 32 (i.e. step 32): o a first set (i.e., Positive set) of proteins from the first dataset 44, using a first group (e.g., “transmembrane” AND “cell membrane”, or, “cell membrane” AND “receptor”) of keys (e.g., keywords in Uniprot), the first group of keys indicating if a protein is at least partially present on the external surface a cell; and o a second set (i.e., Negative set) of proteins from the first dataset 44, using a second group (e.g., NOT “membrane”, or (NOT (“cell membrane”) AND “membrane of any organelles inside the cell”)) of keys (e.g., keywords inUniprot), the second group of keys indicating if a protein is not on the external surface of a cell (e.g., none of the amino acid of the protein is at the external surface of the cell).

[0084] In other words, as a result of this step, the method extract from the same database (i.e., first dataset 44) a first set of amino acid sequences, each amino acid sequence representing a protein, that has been labelled in said database as being at least partially present on the external surface a cell, and a second set of amino acid sequences, each amino acid sequence representing a protein, that has been labelled in said database as totally absent from the external surface a cell.

[0085] The method 3 may further comprise filtering 33 (i.e. step 33): o the extracted first set based on the provided one or more second datasets 45; and o the second extracted set based on the provided one or more second datasets 45.

[0086] The method 3 may further comprise clustering 34 (i.e. step 34) the filtered extracted first set and the filtered extracted second set based on an identity of two amino acid sequences in the filtered extracted sets, thereby obtaining a first clustered dataset and a second clustered dataset. Said clustering step 34 may comprise grouping similar amino acid sequences together based on their degree of similarity. For instance, the first clustered dataset may comprise X clusters, each cluster being a group of similar proteins, and the second clustered dataset may comprise Y clusters, each cluster being a group of similar proteins. In one example, X and Y are superior to one.

[0087] The method 3 may further comprise forming 35 (i.e. step 35) the training dataset 42 by aggregating the first clustered dataset and the second clustered dataset. Alternatively, said method 3 may implement a step 35 comprising selecting a representant of each cluster from the first clustered dataset and selecting a representant of each cluster from the second clustered dataset and forming the training dataset using said selected representants. The representant of a cluster may be a protein within the cluster that is the most similar (e.g. highest degree of similarity) to all other proteins within the same cluster.

[0088] The clustering may be performed using the mmseqs2 tool, specifically through the "easy-cluster" command. The representant of a cluster may be determined by the mmseqs2 tool.

[0089] The clustering may be performed using any tool for comparing, clustering, and searching large sets of protein or nucleotide sequences by indexing sequences, prefiltering similar regions, and then performing detailed alignments only where needed, drastically speeding up large-scale comparisons without losing much accuracy.

[0090] For example, the clustering may be performed using CD-HIT (Cluster Database at High Identity with Tolerance), Linclust, UCLUST (from USEARCH).

[0091] The identity of two amino acid sequences (i.e., two proteins) may be, for example, based on a similarity between said two amino acid sequences. The method may measure said similarity using any known method in the field. The method may cluster the amino acid sequences of each of the filtered extracted first / second sets if any two amino acid sequences have a similarity measure above a threshold, for example, above 80% or preferably 90%.

[0092] Similarly to the training dataset, the method may form a validation and a testing dataset.

[0093] For example, the first dataset 44 may originate from a protein database that entails structural and functional properties of all the proteins found in eukaryotic cells. UniProt is an example of such a protein database.

[0094] In the first dataset 44, protein subcellular localizations may originate from either a single source or a compilation of various sources, whether publicly available or not. For example, protein subcellular localizations may originate from experimental data, inferences based on similarity to other proteins, or computational predictions.

[0095] The first set of proteins (i.e., first set of amino acid sequences of proteins, also called positive set) may comprise proteins that are at least partially present on the external surface a cell. A protein may be considered at least partially present on the external surface a cell if the protein is for example: “a transmembrane protein and associated with the cell membrane”, or “associated with a receptor and with the cell membrane”. Notethat a transmembrane protein may be spanning across an organelle membrane and not across the cell membrane.

[0096] The second set of proteins (i.e., second set of amino acid sequences of proteins, also called negative set) may comprise proteins that are not on the external surface of a cell. A protein may be considered as not on the external surface of a cell if the protein is for example: “not associated with a membrane”, or, “not associated with the cell membrane and associated with a membrane of any organelles inside the cell”. In other words, the negative set comprises proteins that are associated in the first dataset 44 to a label indicating that none of the amino acids of the protein’s amino acid sequence is located on the external surface of the cell membrane.

[0097] The one or more second datasets 45 with experimental validation of protein subcellular localizations may originate from reference protein databases that may be smaller than the protein database used to obtain the first dataset (e.g., UniProt) but more reliable. In other words, the one or more second datasets 45 is more reliable as they comprise pair of information “protein’s amino acid sequence”-“protein subcellular localizations” that have been experimentally validated. Reference protein databases may be:Knowledge based: obtained from scientific literature;Experiment based: obtained from experimental data such as the Human Protein atlas (HPA).

[0098] The one or more second datasets 45 may comprise high quality experimental protein subcellular localizations that may be used to validate entries in the first dataset 44. In an example, a label from the first dataset 44 may only be accepted if it also appears in at least one of the second datasets 45.

[0099] Filtering 33 the first / second extracted set based on the provided one or more second datasets may be achieved through a comparison between the content of the first / second extracted set and the content of the provided one or more second datasets 45. For example, the filtered first / second extracted dataset may comprise less data than the first / second extracted dataset after filtering out the data that were found not reliable.

[0100] The first clustered dataset and the second clustered dataset may be obtained using a clustering algorithm based on amino acids sequence identity. For example, the clustering algorithm may group similar sequences together based on their degree of similarity:- calculating pairwise sequence identities between all sequences in the filtered extracted first set and the filtered extracted second set (in other words: calculating pairwise sequence identities between each protein sequence in the filtered extracted first set; and calculating pairwise sequence identities between each protein sequence in the filtered extracted second set); wherein measuring / calculating sequence identity for each pair of amino acids sequences comprises measuring for example a percentage of identical amino acids or nucleotides at equivalent positions in the aligned sequences;- grouping sequences into clusters based on a clustering criterion (e.g., a threshold of sequence identity, above which sequences are considered sufficiently similar to be clustered together);- outputting the first clustered dataset and the second clustered dataset, wherein sequences within each cluster share a high degree of sequence identity with each other (e.g., 90% of the amino acids are exactly the same when aligning two sequences of the cluster).

[0101] Forming 35 the training dataset may be obtained by aggregating the first clustered dataset and the second clustered dataset. In other words, forming 35 the training dataset may comprise selecting a representant of each cluster from the first clustered dataset and selecting a representant of each cluster from the second clustered dataset and forming the training dataset using the selected representants. For example, only one representant of each cluster may be kept in the final training set (i.e. training dataset), as both the first and second clustered datasets may comprise several clusters. Since clusters in the first and second clustered dataset may be formed of similar proteins, this advantageously allows to avoid having similar proteins repeated numerous times in the training dataset as this could lead to over-representation. Thus, only one representant of each cluster may be selected from both the first clustered dataset and the second clustered dataset.

[0102] For example, a balance between the positive set and the negative set may be achieved in forming the training dataset. In case of a balanced training dataset, the samenumber of proteins may be extracted from both the first and the second clustered datasets. The smallest set may impose the final number of proteins by its size (i.e. the number of proteins it contains).

[0103] According to one or several embodiments, at least one of the one or more second datasets 45 may comprise a confidence value for each entry, the confidence value representing a score of the entry being at the cell membrane (i.e. at least partially on the external surface of the cell).

[0104] An entry may be a whole amino acid sequence. In this case the whole amino acid consists in the sequence of all amino acids constituting the protein.

[0105] The score may be a value from a range of values. For example, the higher the value of the score, the higher is the confidence in the entry being at the cell membrane. In an example, the score is set between the values: 0, 1, 2, 3, 4, 5; 0 corresponding to the lowest chance of the entry being at the cell membrane and 5 corresponding to the highest chance of the entry being at the cell membrane.

[0106] For example, during filtering 33 the first / second extracted set based on the provided one or more second datasets, the data from the one or more second datasets 45 that comprises the confidence value representing the score of 0 may have lower chances of being part of the filtered first / second extracted dataset; however the data from the one or more second datasets that comprises the confidence value representing the score of 5 may have higher chances of being part of the filtered first / second extracted dataset.

[0107] According to one or several embodiments, the first dataset 44 may be extracted in part from UniProt. For example, the first dataset 44 may only comprise human proteins and / or reviewed proteins (i.e. proteins that have been manually curated and reviewed by experts for accuracy and reliability).

[0108] The methods 1, 2 and 3 are computer-implemented. This means that steps (or substantially all the steps) of these methods are executed by at least one computer, or any system alike. Thus, steps of each of these methods are performed by the computer, possibly fully automatically, or, semi-automatically. For example, the triggering of at least some of the steps of one of these methods may be performed through user-computer interaction. The level of user-computer interaction required may depend on the level ofautomatism foreseen and put in balance with the need to implement user’s wishes. In examples, this level may be user-defined and / or pre-defined.

[0109] The computer-implemented methods 1, 2 and 3 can be carried out alternatively or in any cumulative way, other implementations within the scope of the present disclosure include methods having only parts of the present functionalities.

[0110] A typical example of computer-implementation of a method is to perform the method with a system adapted for this purpose. The system may comprise a processor coupled to a memory and preferably a graphical user interface (GUI), the memory having recorded thereon a computer program comprising instructions for performing the method. The memory may also store a database. The memory is any hardware adapted for such storage, possibly comprising several physical distinct parts (e.g. one for the program, and possibly one for the database).

[0111] The present disclosure further relates to a computer program comprising instructions for performing any of the herein disclosed computer-implemented methods. The computer program may comprise instructions executable by a computer, the instructions comprising means for causing the above system to perform any of the disclosed methods. The program may be recordable on any data storage medium, including the memory of the system. The program may for example be implemented in digital electronic circuitry, or in computer hardware, firmware, software, or in combinations of them. The program may be implemented as an apparatus, for example a product tangibly embodied in a machine-readable storage device for execution by a programmable processor. A method steps may be performed by a programmable processor executing a program of instructions to perform functions of the method by operating on input data and generating output. The processor may thus be programmable and coupled to receive data and instructions from, and to transmit data and instructions to, a data storage system, at least one input device, and at least one output device. The application program may be implemented in a high-level procedural or object-oriented programming language, or in assembly or machine language if desired. In any case, the language may be a compiled or interpreted language. The program may be a fullinstallation program or an update program. Application of the program on the system results in any case in instructions for performing the method.

[0112] The system may comprise at least one of the following devices: a device for obtaining, by training, a tuned model 51 (i.e. a device for implementing the method 1), a device for determining whether a protein is at least partially present on the external surface of a cell (i.e. a device for implementing the method 2), and / or a device for forming a dataset (i.e. a device for implementing the method 3), according to one or several embodiments compliant with the present disclosure.

[0113] Each of these devices may be advantageously an apparatus, or a physical part of an apparatus, designed, configured and / or adapted for performing the mentioned functions and produce the mentioned effects or results. In alternative implementations, any of the devices is embodied as a set of apparatus or physical parts of apparatus, whether grouped in a same machine or in different, possibly remote, machines. Any of the devices may e.g. have functions distributed over a cloud infrastructure and be available to users as a cloud-based service, or have remote functions accessible through an API. The devices may be integrated in a same apparatus or set of apparatus, and intended to same users. In other implementations, the structure of one of the devices may be completely independent of the structure of another one of the devices, and may be provided for other users.

[0114] A particular apparatus 9, visible on Figure 4, is embodying the system described above. It corresponds for example to a workstation, a laptop, a tablet, a smartphone, or a head- mounted display (HMD).

[0115] That apparatus 9 comprises the following elements, connected to each other by a bus 95 of addresses and data that also transports a clock signal:- a microprocessor 91 (or CPU);- a graphics card 92 comprising several Graphical Processing Units (or GPUs) 920 and a Graphical Random Access Memory (GRAM) 921- a non-volatile memory of ROM type 96;- a RAM 97;- one or several I / O (Input / Output) devices 94 such as for example a keyboard, a mouse, a trackball, a webcam; other modes for introduction of commands such as for example vocal recognition are also possible;- a power source 98; and- a radiofrequency unit 99.

[0116] According to a variant, the power supply 98 is external to the apparatus 9.

[0117] The apparatus 9 also comprises a display device 93 of display screen type directly connected to the graphics card 92 to display synthesized images calculated and composed in the graphics card. The use of a dedicated bus to connect the display device 93 to the graphics card 92 offers the advantage of having much greater data transmission bitrates and thus reducing the latency time for the displaying of images composed by the graphics card. According to a variant, a display device is external to apparatus 9 and is connected thereto by a cable or wirelessly for transmitting the display signals. The apparatus 9, for example through the graphics card 92, comprises an interface for transmission or connection adapted to transmit a display signal to an external display means such as for example an LCD or plasma screen or a video-projector. In this respect, the RF unit 99 can be used for wireless transmissions.

[0118] It is noted that the word "register" used hereinafter in the description of memories 97 and 921 can designate in each of the memories mentioned, a memory zone of low capacity (some binary data) as well as a memory zone of large capacity (enabling a whole program to be stored or all or part of the data representative of data calculated or to be displayed). Also, the registers represented for the RAM 97 and the GRAM 921 can be arranged and constituted in any manner, and each of them does not necessarily correspond to adjacent memory locations and can be distributed otherwise (which covers notably the situation in which one register includes several smaller registers).

[0119] When switched-on, the microprocessor 91 loads and executes the instructions of the program contained in the RAM 97.

[0120] As will be understood by a skilled person, the presence of the graphics card 92 is not mandatory, and can be replaced with entire CPU processing and / or simpler visualization implementations.

[0121] In variant modes, the apparatus 9 may include only the functionalities of one device and not another. In addition, at least one of the devices may be implemented differently than a standalone software, and an apparatus or set of apparatus comprising only parts of the apparatus 9 may be exploited through an API call or via a cloud interface.

Claims

CLAIMS1. A computer- implemented method (1) for obtaining, by training, a tuned model (51) configured to predict whether a protein is present at least partially on the external surface of a cell, said tuned model (51) being configured to receive as input an amino acid sequence of a protein and to output a probability for presence of at least a part of said protein, represented by said amino acid sequence, on the external surface of a cell, said method (1) comprising: obtaining (11): o a training dataset (42) comprising a plurality of entries each comprising an amino acid sequence representative of a protein and a corresponding indication of whether or not said protein is present at least partially on the external surface of a cell; and o a first protein language model (PLM) (41) being configured to receive as input an amino acid sequence of any length and to output a probability for presence, on the external surface of a cell, of at least a part of the input protein represented by the input amino acid sequence, and tuning (12) the obtained first PLM by: o determining a plurality of values for a set of one or more training hyperparameters; and o for each value of said plurality of values for said set, tuning the first PLM using the obtained training dataset and the values for said set of one or more training hyperparameters, thereby obtaining a plurality of tuned PLMs (51).

2. The method (1) of claim 1, wherein obtaining (11) the first PLM (41) comprises: receiving a pre-trained PLM configured to receive as input an amino acid sequence and to output a representation of a protein represented by the input amino acid; adapting the received pre-trained PLM so as to receive as input an amino acid sequence of any length, and adding a classification head to output said probability for presence of at least a part of said protein on the external surface of a cell.

3. The method (1) of any of claims 1 or 2, wherein the set of one or more training hyperparameters comprises one or more of: a learning rate, a batch size, and one or more parameters for inputting an amino acid sequence to the first PLM (41).

4. The method (1) of claim 3, wherein the first PLM (41) is configured to take as input the amino acid sequence of any length in the form of a plurality of portions, said one or more parameters for inputting comprising: a chunk size representing a size of said portions of said input amino acid sequence, and an overlap size representing an overlap between two consecutive portions of said plurality of portions of said input amino acid sequence.

5. A computer-implemented method (2) for determining whether a protein is at least partially present on the external surface of a cell, the method (2) for determining whether a protein is at least partially present on the external surface of a cell comprising: providing (21): o a tuned model (51) configured to predict whether a protein is present at least partially on the external surface of a cell, said tuned model (51) comprising one or more tuned PLMs of a plurality of the tuned PLMs (51), each tuned according to the method (1) of any of claims 1 to 4; and o an amino acid sequence (43), of any length, representative of said protein; and determining (22) a probability for presence (61 ) of at least a part of said protein on the external surface of a cell by computing a probability using said tuned model (51).

6. The method (2) of claim 5, wherein the determining (22) of said probability for presence (61) comprises:computing one or more intermediate probabilities, each using a respective tuned PLM of the provided one or more tuned PLMs (51); determining said probability for presence (61) as a function of said one or more computed intermediate probabilities.

7. The method (2) of claim 6, wherein the function comprises one or more of an average function, a max function, and a min function.

8. The method (2) according to any of claims 5 to 7, wherein each tuned PLM (51) takes as input the amino acid sequence of any length in the form of a plurality of portions, said plurality of portions being defined by: a chunk size representing a size of each portion of plurality of portions forming said input amino acid sequence, and an overlap size representing an overlap between two consecutive portions of said plurality of portions of said input amino acid sequence.

9. A computer-implemented method (3) for forming a training dataset (42) for obtaining, by training, a tuned model (51) configured to predict whether a protein is at least partially present on the external surface of a cell, said method (3) comprising: providing (31): o a first dataset (44) of amino acid sequences of proteins which may be at least partially present on the external surface of cells; and o one or more second datasets (45) with experimental validation of protein subcellular localizations; extracting (32): o a first set of amino acid sequences of proteins from the first dataset (44), using a first group of keys, the first group of keys indicating if a protein is at least partially present on the external surface a cell; and o a second set of amino acid sequences of proteins from the first dataset, using a second group of keys, the second group of keys indicating if a protein is not on the external surface of a cell; filtering (33):o the extracted first set based on the provided one or more second datasets; and o the extracted second set based on the provided one or more second datasets; clustering (34) the filtered extracted first set and the filtered extracted second set based on an identity of two amino acid sequences in said filtered extracted first and second sets, thereby obtaining a first clustered dataset and a second clustered dataset; and selecting a representant of each cluster from the first clustered dataset and selecting a representant of each cluster from the second clustered dataset and forming (35) the training dataset (42) using said selected representants.

10. The method (3) of claim 9, wherein at least one of the one or more second datasets (45) comprises a confidence value for each entry, the confidence value representing a score of said entry being at the cell membrane.

11. The method (3) of any of claims 9 or 10, wherein the first dataset (44) is extracted in part from UniProt.

12. A computer- implemented method (1) for obtaining, by training, a tuned model (51) configured to predict whether a protein is present at least partially on the external surface of a cell, said tuned model (51) being configured to receive as input an amino acid sequence of a protein and to output a probability for presence of at least a part of said protein represented by said amino acid sequence on the external surface of a cell, said method (1) comprising: obtaining (11): o a training dataset (42) formed by a computer-implemented method (3) according to any of claims 9 to 11; and o a first protein language model (PLM) (41) being configured to receive as input an amino acid sequence of any length and to output a probability for presence, on the external surface of a cell, of at least a part of the input protein represented by the input amino acid sequence, and tuning (12) the obtained first PLM by:o determining a plurality of values for a set of one or more training hyperparameters; and o for each value of said plurality of values for said set, tuning the first PLM using the obtained training dataset and the values for said set of one or more training hyperparameters, thereby obtaining a plurality of tunedPLMs (51).

13. A computer program comprising instructions for performing any of the methods according to any one of claims 1 to 12.

14. A computer readable storage medium having recorded thereon the computer program of claim 13.

15. A system comprising a processor coupled to a memory, the memory having recorded thereon the computer program of claim 13.

Images