Predicting nanobody stability and generating stability-modulated sequences

Abstract

In some embodiments, a computer-implemented method of generating one or more artificial amino acid sequences representing artificial proteins predicted to have a different level of stability than a protein represented by an input amino acid sequence is provided. A computing system receives the input amino acid sequence and uses a tuned base machine learning model to generate an encoding of the input amino acid sequence. The tuned base machine learning model was fine-tuned to encode characteristics of proteins having the different level of stability. The computing system generates the one or more artificial amino acid sequences by decoding the encoding of the input amino acid sequence.

Description

PREDICTING NANOBODY STABILITY AND GENERATING STABILITY-MODULATED SEQUENCESCROSS-REFERENCE(S) TO RELATED APPLICATION(S)

[0001] This application claims the benefit of Provisional Application No. 63 / 659747, filed June 13, 2024, the entire disclosure of which is hereby incorporated by reference herein for all purposes.BACKGROUND

[0002] Owning to the fundamental role that biomolecules play in the functioning and pathology of living cellular systems, a key demand in biomedical research and applications is to detect and manipulate any molecule or live cell population of interest. About 50% of cellular dry mass is composed of proteins and at least 3,000- 10.000 or more unique protein molecules are present across mammalian cell proteomes. Diverse proteins pose a significant source of substrate to target for biomedical research or applications. Despite the prevalence of proteins and their importance, there is currently no entirely generalizable strategy to systematically generate biosensors that can detect and / or manipulate a particular molecular target or cell population based on protein expression.

[0003] Antibody fragments from Camelid species, also called Nanobodies, have great potential to fill this gap in technology in biomedicine, due to their ideal size, amenability to be applied both extracellularly and intracellularly, and their high affinity / specificity for target molecules. Nanobodies (NBs), are single-domain antibody fragments derived from camelids. They are small (~14 kDa), stably folded, and can bind difficult epitopes. Recent advances in animal immunization, in vitro selection, and de novo design have greatly expanded their repertoire, with over 1.4 million NB sequences cataloged in the Observed Antibody Space (OAS) and more than 700 NB-target structures in the Protein Data Bank (PDB).

[0004] NBs were initially delivered as genetic constructs intracellularly to perform target imaging or target degradation in living cells. However, because most NBs were originally developed for extracellular contexts, genetically expressed NBs often exhibit aggregation or unregulated stability, generating non-specific signals whenexpressed in living cells. These caveats limit their utility in intracellular applications. This is a general problem that also extends to other protein binders. Further, as with any protein binder system, the stability of the binder upon expression poses a significant problem in biosensing, due to the presence of background noise independently of interaction with the target molecule. This problem may be addressed by rendering nanobodies conditionally stable upon antigen binding, creating a system of reagents referred to as conditionally stable nanobodies. However, mutations that create effective sensors with some nanobodies do not always transfer effectively to create sensors with other nanobodies. Also, sensor and effector proteins fused to nanobodies could further modulate the effectiveness of certain conditionally stable mutations.

[0005] To address this issue, a tool that deeply learns from the landscape of mutational and contextual information that influence nanobody stability and conditional stability’, and in turn makes useful predictions about stability based on sequence alone, and can generate recommendations that modulate the stability or conditionally stability’ of input sequences, is desired.SUMMARY

[0006] This summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This summary is not intended to identify key features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter.

[0007] In some embodiments, a computer-implemented method of training a machine learning model to predict stability of an amino acid sequence is provided. A computing system trains a base machine learning model to encode characteristics of proteins of a library of proteins using known amino acid sequences representing the proteins of the library of proteins as a first training data set. The computing system fine tunes the base machine learning model using amino acid sequences in a stability training data set that includes amino acid sequences labeled as representing stable proteins and amino acid sequences labeled as representing unstable proteins to create a stability prediction model that encodes characteristics of the stable proteins and the unstable proteins. The computing system trains a classifier model using the aminoacid sequences and labels of stability training data set to classify amino acid sequences as stable or unstable.

[0008] In some embodiments, a computer-implemented method of generating one or more artificial amino acid sequences representing artificial proteins predicted to have a different level of stability than a protein represented by an input amino acid sequence is provided. A computing system receives the input amino acid sequence, and uses a tuned base machine learning model to generate an encoding of the input amino acid sequence. The tuned base machine learning model was fine-tuned to encode characteristics of proteins having the different level of stability. The computing system generates the one or more artificial amino acid sequences by decoding the encoding of the input amino acid sequence.

[0009] In some embodiments, a non-transitory computer-readable medium having computer-executable instructions stored thereon is provided. The instructions, in response to execution by one or more processors of a computing system, cause the computing system to perform actions of a method as described above.

[0010] In some embodiments, a system for generating predictions related to amino acid sequences that represent proteins is provided. The system comprises: a user interface computing system and a model execution computing system. The user interface computing system is configured to generate a query interface configured to receive an input amino acid sequence that represents a protein. The model execution computing system is configured to receive the input amino sequence from the user interface computing system; provide the input amino acid sequence to one or more machine learning models as input to generate one or more outputs; and transmit the one or more outputs to the user interface computing system. The user interface computing system is further configured to generate a result interface to present the one or more outputs.BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The foregoing aspects and many of the attendant advantages of this invention will become more readily appreciated as the same become better understood by reference to the following detailed description, when taken in conjunction with the accompanying drawings, wherein:

[0012] FIG. 1A - FIG. 1C illustrate the use of nanobodies for live intracellular applications, according to various aspects of the present disclosure.

[0013] FIG. 2 is a block diagram that illustrates aspects of a non-limiting example embodiment of a nanobody prediction computing system according to various aspects of the present disclosure.

[0014] FIG. 3 is a schematic illustration of a non-limiting example embodiment of a data pipeline for predicting stability of a protein sequence, according to various aspects of the present disclosure.

[0015] FIG. 4 is a flowchart that illustrates a non-limiting example embodiment of a method of training a machine learning model to predict stability of nanobodies, according to various aspects of the present disclosure.

[0016] FIG. 5A - FIG. 5C illustrate results of a non-limiting example embodiment of creating a stability training data set and training a stability prediction model, according to various aspects of the present disclosure.

[0017] FIG. 6 is a schematic illustration of a non-limiting example embodiment of a data pipeline for generating sequences having desired stability characteristics, according to various aspects of the present disclosure.

[0018] FIG. 7 is a flowchart that illustrates a non-limiting example embodiment of a method of training machine learning models to generate nanobody sequences having desired stability changes, according to various aspects of the present disclosure.

[0019] FIG. 8 is a flowchart that illustrates a non-limiting example embodiment of a method of using machine learning models to generate sequences having desired changes in stability, according to various aspects of the present disclosure.

[0020] FIG. 9 is a chart that illustrates results of analysis of sequences generated using an embodiment of the method illustrated in FIG. 8.

[0021] FIG. 10A - FIG. 10F further illustrate validation of sequences generated by the techniques described in FIG. 8.

[0022] FIG. 11 is a block diagram that illustrates a non-limiting example embodiment of a system of computing devices configured to generate sequences having desired stability characteristics, according to various aspects of the present disclosure.

[0023] FIG. 12 is a flowchart illustrating a non-limiting example embodiment of a method of generating sequences having desired changes in stability, according to various aspects of the present disclosure.

[0024] FIG. 13 illustrates a non-limiting example embodiment of a query interface, according to various aspects of the present disclosure.

[0025] FIG. 14 is an illustration of a non-limiting example embodiment of a result interface, according to various aspects of the present disclosure.DETAILED DESCRIPTION

[0026] Recent advances in artificial intelligence (Al), particularly in neural network architectures, present a promising avenue for building a tool for predicting nanobody stability. Large language models (LLMs), which have become increasingly sophisticated, offer tools to quantify similarities in text. Similar to language text, protein sequences are encoded in sequences of letters (amino acids) from an “alphabet.’’ The contextual interaction between amino acids in a protein sequence demands non-linear models such as LLMs. Recently, a LLM called NanoRoberta was developed, training on camelid antibody sequences and fine-tuning the model to predict interface interactions on a nanobody sequence. Here, we integrated multiple streams of information, including experimental data on nanobody stability, nanobody interface information, predictive and generative abilities of LLMs, and web user interface development to create a novel process that addresses the problem of generalizing conditionally stable engineering of nanobodies.

[0027] The techniques described herein provide various technical benefits. For example, conventional NB workflows can take months per target due to laborious mutagenesis and testing. The deep learning LLMs described herein predict and modify NB stability from sequence data, while modifying only non-interface residues. These tools propose mutations to modulate intracellular NB stability within seconds rather than months. Further, mutations that alter stability can compromise target affinity when they occur in binding interfaces. By analyzing over 389 high quality X-ray NB-target crystal structures, 31 non-interface residues were identified, allowing the techniques disclosed herein to selectively engineer intracellular stability without disturbing target binding. Further, the currently disclosed techniques aregeneralizable. Existing NB modifications for intracellular applications often work only for a small set of NBs, limiting adoption by diverse NBs. Any laboratory can input NB sequences — whether newly discovered or preexisting — and obtain stability- modulated NB variants. This reduces the barrier for researchers across fields, who need robust intracellular binders.

[0028] FIG. 1A - FIG. 1C illustrate the use of nanobodies for live intracellular applications, according to various aspects of the present disclosure. NBs were initially delivered as genetic constructs intracellularly to perform target imaging or target degradation in living cells. However, because most NBs were originally developed for extracellular contexts, genetically expressed NBs often exhibit aggregation or unregulated stability, generating non-specific signals when expressed in living cells. These caveats limit their utility in intracellular applications. This is a general problem that also extends to other protein binders.

[0029] Techniques have been developed that enable NBs to be used for applications requiring target-triggered activities, such as detection and manipulation of specific target-expressing cells. The simplest such techniques involve generating conditionally stable NBs (CS-NBs) that, when expressed intracellularly, are actively degraded unless they bind their intracellular target. FIG. 1 A illustrates such a use of conditionally stable NBs. A nanobody fusion protein having a sequence "MAQVQLQE..." and a mutated nanobody fusion protein having a sequence “MKQVQLQE...’’ are illustrated, with the mutated nanobody exhibiting conditional stability.

[0030] This design eliminates high background signals by making NB stability dependent on target binding. The first CS-NB developed, dGBPl, arose through a cell-based mutagenesis screen and was applied across transgenic GFP animal lines to manipulate the genes selectively within GFP+ cells. 3 major destabilizing dGBPl mutations (3maj) were mapped to conserved NB backbone residues not involved in interface binding. By transferring these destabilizing mutations, new CS-NBs against mammalian targets such as LAP1 and p53 were created.

[0031] Applications with CS-NBs were demonstrated by fusing anti-HIV Capsid CS-NB to fluorescent proteins to detect live HI V-reactivated T cells (FIG. IB), and to Cas9 protein to conduct genome editing only in HIV Capsid+ cells (FIG. 1C). CS-NBs were found to retain conditional stability across eukaryotic systems (e.g., human cell lines 293T. HeLa; rodent models; zebrafish; fruit flies; cancer cell lines A549, U2OS), suggesting that a conserved proteolytic mechanism, the ubiquitin proteasome system, underlies their degradation.

[0032] Despite these successes, a central challenge remains: each NB’s backbone sequence and epitope-binding interface is unique and interacts with introduced destabilizing mutations to generate variability in outcomes, such as NB aggregation or conditional stability. Alternative approaches — such as inserting exogenous protein domains — face the same scalability problem, requiring extensive, ad hoc optimization for every novel NB or target.

[0033] Embodiments of the present disclosure provide a solution that addresses this limitation by applying high content imaging to quantify large NB-reporter libraries inside living cells, and using the data to train large language models (LLMs), resulting in computer models that modulate intracellular stability of any input NB sequence without affecting target binding. The platform predicts how to adapt any NB — regardless of its source — into a robust, low-background intracellular binder.

[0034] FIG. 2 is a block diagram that illustrates aspects of a non-limiting example embodiment of a nanobody prediction computing system according to various aspects of the present disclosure. The illustrated nanobody prediction computing system 210 may be implemented by any computing device or collection of computing devices, including but not limited to a desktop computing device, a laptop computing device, a mobile computing device, a server computing device, a computing device of a cloud computing system, and / or combinations thereof. In some embodiments, the components illustrated in the nanobody prediction computing system 210 may be separated into separate computing systems, including but not limited to the separation illustrated in FIG. 11 and described in further detail below. In some embodiments, the nanobody prediction computing system 210 is configured to train machine learning models to predict stability of nanobodies based on sequence information, and / or to train machine learning models to generate sequences having desired stability characteristics. In some embodiments, the nanobody prediction computing system 210 is configured to use trained machine learning models topredict stability of nanobodies, to generate sequences having desired stability’ characteristics, and / or for other purposes.

[0035] As shown, the nanobody prediction computing system 210 includes one or more processors 202, one or more communication interfaces 204, a model data store 208, a protein library data store 214, and a computer-readable medium 206.

[0036] In some embodiments, the processors 202 may include any suitable ty pe of general-purpose computer processor. In some embodiments, the processors 202 may include one or more special-purpose computer processors or Al accelerators optimized for specific computing tasks, including but not limited to graphical processing units (GPUs), vision processing units (VPUs), and tensor processing units (TPUs).

[0037] In some embodiments, the communication interfaces 204 include one or more hardware and or software interfaces suitable for providing communication links between components. The communication interfaces 204 may support one or more wired communication technologies (including but not limited to Ethernet, FireWire, and USB), one or more wireless communication technologies (including but not limited to Wi-Fi, WiMAX, Bluetooth, 2G, 3G, 4G, 5G, and LTE), and / or combinations thereof.

[0038] As shown, the computer-readable medium 206 has stored thereon logic that, in response to execution by the one or more processors 202, cause the nanobody prediction computing system 210 to provide a model training engine 212, a model execution engine 216, and a user interface engine 218.

[0039] As used herein, “computer-readable medium” refers to a removable or nonremovable device that implements any technology capable of storing information in a volatile or non-volatile manner to be read by a processor of a computing device, including but not limited to: a hard drive; a flash memory; a solid state drive; random-access memory' (RAM); read-only memory (ROM); a CD-ROM, a DVD, or other disk storage; a magnetic cassette; a magnetic tape; and a magnetic disk storage.

[0040] In some embodiments, the model training engine 212 is configured to use protein sequence information and stability information stored in the protein library' data store 214 to train one or more machine learning models to predict stability' of sequences and / or to generate sequences having desired stability characteristics, andto store the trained machine learning models in the model data store 208. In some embodiments, the model execution engine 216 is configured to retrieve trained models from the model data store 208, and to use the trained models to predict stability of protein sequences and / or to generate sequences having desired stability characteristics. In some embodiments, the protein library data store 214 may include multiple data stores, including but not limited to one or more publicly available sources of protein and / or sequence data (e.g., Protein Data Bank (PDB), Observed Antibody Space (OAS), etc.).

[0041] Further description of the configuration of each of these components is provided below.

[0042] As used herein, ‘‘engine” refers to logic embodied in hardware or software instructions, which can be written in one or more programming languages, including but not limited to C, C++, C#, COBOL, JAVA™, PHP, Perl, HTML, CSS, JavaScript, VBScript, ASPX, Go, and Python. An engine may be compiled into executable programs or written in interpreted programming languages. Software engines may be callable from other engines or from themselves. Generally, the engines described herein refer to logical modules that can be merged with other engines or can be divided into sub-engines. The engines can be implemented by logic stored in any type of computer-readable medium or computer storage device and be stored on and executed by one or more general purpose computers, thus creating a special purpose computer configured to provide the engine or the functionality thereof. The engines can be implemented by logic programmed into an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA). or another hardware device.

[0043] As used herein, “data store” refers to any suitable device configured to store data for access by a computing device. One example of a data store is a highly reliable, high-speed relational database management system (DBMS) executing on one or more computing devices and accessible over a high-speed network. Another example of a data store is a key -value store. However, any other suitable storage technique and / or device capable of quickly and reliably providing the stored data in response to queries may be used, and the computing device may be accessible locally instead of over a network, or may be provided as a cloud-based service. A data storemay also include data stored in an organized manner on a computer-readable storage medium, such as a hard disk drive, a flash memory, RAM. ROM, or any other type of computer-readable storage medium. One of ordinary skill in the art will recognize that separate data stores described herein may be combined into a single data store, and / or a single data store described herein may be separated into multiple data stores, without departing from the scope of the present disclosure.

[0044] FIG. 3 is a schematic illustration of a non-limiting example embodiment of a data pipeline for predicting stability of a protein sequence, according to various aspects of the present disclosure. The pipeline 300 uses a large language model trained to encode characteristics of nanobodies to make predictions about intracellular nanobody stability.

[0045] The pipeline 300 starts with an input sequence 302. The input sequence 302 may be any sequence of amino acid residues to be analyzed for stability, and typically represents a nanobody. The input sequence 302 then undergoes tokenization 304 to create an encoded input sequence 306. Tokenization is a task that is commonly used for processing natural language using a machine learning model, in which each word in an input string is mapped to a vocabulary. Unlike this more complex tokenization used for natural language, the tokenization 304 applied to the input sequence 302 may be a direct mapping method where each amino acid residue is represented by a unique identifier. As a non-limiting example, a vocabulary for the tokenization 304 may be defined as {“M”: 1, ‘R’’: 2, "‘H”: 3, “K”: 4, ‘"D”: 5, "‘E”: 6, “S”: 7, “T”: 8, “N”: 9}. With such a mapping, an input sequence 302 of “MKRTEN” may undergo tokenization 304 to create an encoded input sequence 306 of [1, 4, 2. 8, 6, 9], thereby facilitating efficient model processing.

[0046] The encoded input sequence 306 is provided as input to a base machine learning model 310. Any suitable architecture may be used for the base machine learning model 310. In some embodiments, a model architecture that encodes characteristics of the input, including contextual relationships between tokens of the input may be used. In some embodiments, a BERT (Bidirectional Encoder Representations from Transformers) model architecture may be used. In some embodiments, an improved version of a BERT model, such as a RoBERTa (Robustly Optimized BERT Pretraining Approach) model, may be used, which is similar toBERT but which uses better pretraining techniques. One non-limiting example of a suitable RoBERTa model architecture includes 12 hidden layers, 12 attention heads, a hidden dimension of 768, and a dimensionality of 3072 in the feed-forward neural network, though in other examples, other numbers of hidden layers, attention heads, hidden dimensions, and feed-forward dimensionality may be used. The base machine learning model 310 may be trained using a technique, as described below with respect to FIG. 4.

[0047] The output of the base machine learning model 310 is a token embedding 312. In some embodiments, the token embedding 312 represents a likelihood of each token in the vocabulary being present at each position in the sequence.

[0048] To create the stability prediction model 308. the token embedding 312 is provided to a classifier model 314. The classifier model 314 outputs a stability prediction 316, which indicates a probability regarding whether the input sequence 302 would be classified as a stable nanobody or an unstable nanobody. Any suitable architecture may be used for the classifier model 314. In some embodiments, a feedforward neural network with two nodes at the end representing the stable classification and the unstable classification, respectively, may be used. The classifier model 314 may be trained to generate stability predictions as described below with respect to FIG. 4.

[0049] FIG. 4 is a flowchart that illustrates a non-limiting example embodiment of a method of training a machine learning model to predict stability of nanobodies, according to various aspects of the present disclosure. In the method 400, the base machine learning model 310 is trained to encode characteristics of proteins, and the classifier model 314 is trained to classify nanobody sequences as being stable or unstable.

[0050] From a start block, the method 400 proceeds to block 402, where a model training engine 212 of a nanobody prediction computing system 210 gathers protein sequence data for a set of proteins. In some embodiments, the set of proteins is a set of nanobodies. For example, the model training engine 212 may obtain camelid antibody sequences from the Observed Antibody Space database provided by the University of Oxford. In other examples, the protein sequence data may be obtained from another source.

[0051] At block 404, the model training engine 212 tokenizes the protein sequence data. The tokenization of each protein sequence in the protein sequence data may proceed as discussed with respect to the tokenization 304 of FIG. 3. In some embodiments, tokenizing the protein sequence data may also include truncating sequences to a maximum sequence length. For example, a maximum sequence length for nanobodies may be set to a value between 140 and 160, such as 150. In a non-limiting example of a set of protein sequence data for camelid VHH sequences obtained from the Observed Antibody Space database, 99.996% of 1.6 million sequences retrieved were 150 residues long or less, meaning that a maximum sequence length of 150 only affected a trivial number of sequences.

[0052] At block 406, the model training engine 212 trains a base machine learning model 310 using the protein sequence data to encode characteristics of the proteins. In embodiments wherein a BERT or RoBERTa model architecture is used, the base machine learning model 310 may be trained on the masked language modeling task using the tokenized protein sequence data. Any suitable hyperparameters may be used for training the base machine learning model 310. In a non-limiting example embodiment, approximately 1.6 million sequences were obtained. The sequences were split into training, test, and validation sets with an 80 / 10 / 10 split, resulting in 1,281,308 sequences in the training set, 160,163 sequences in the validation set, and 160,164 sequences in the test set. The base machine learning model 310 was then trained on the masked language modeling task for 13,000 steps with a batch size of 96 on an Nvidia T4 GPU using the Adam optimizer. For hyperparameters, a learning rate of 5e-05, a weight decay of 0.01, and Adam epsilon of le-06, and an Adam beta2 of 0.98 were used. In other embodiments, other hyperparameters may be used, including but not limited to different training data set sizes and different numbers of steps and / or batch sizes.

[0053] At block 408, the model training engine 212 stores the trained base machine learning model 310 in a model data store 208 of the nanobody prediction computing system 210. At this point, the base machine learning model 310 has been trained to encode general characteristics of the proteins in the protein sequence data gathered at block 402. This trained base machine learning model 310 may be combined with other components and fine-tuned for various purposes, including but not limited tothe stability prediction model 308 illustrated in FIG. 3, and the generative model 608 illustrated in FIG. 6.

[0054] At block 410, the model training engine 212 gathers protein sequence data that includes stability classifications. In some embodiments, the model training engine 212 may obtain experimental results that indicate stability classifications for a plurality of the proteins in the protein sequence data gathered at block 402, or for other protein sequences. As a non-limiting example, nanobodies may be tested as TagBFP fusion reporters driven off the ubiquitous CAG promoter, or mCheriy fusions. Binary labels (i.e., 0 for unstable sequences, 1 for stable sequences) may be applied, with nanobodies having clearly soluble TagBFP or mCherry reporter expression under fluorescence microscopy being labeled stable, and nanobody reporters showing aggregation, partially soluble, or non-expressing phenotypes being labeled unstable.

[0055] At block 412 the model training engine 212 creates a stability training data set using the protein sequence data that includes stability classifications. In some embodiments, the stability training data set includes a plurality of training data pairs, wherein each training data pair includes a tokenized sequence and a classification label. In creating the stability training data set, the model training engine 212 may reserve portions of the data for test and for validation, and may sample the training data in order to balance various categories (e.g., types of sequence data, labels, etc.). Further details of a specific non-limiting example of the creation of a stability training data set are provided in the discussion of FIG. 5A.

[0056] At block 414, the model training engine 212 combines the trained base machine learning model 310 with a classifier model 314 to create a stability prediction model 308. Any suitable architecture may be used for the classifier model 314. As discussed above with respect to FIG. 3, in some embodiments, a feedforward neural network with two nodes at the end representing the stable classification and the unstable classification, respectively, may be used.

[0057] At block 416, the model training engine 212 trains the stability prediction model 308 using the stability training data set. Any suitable technique may be used to train the stability prediction model 308. As a non-limiting example, the stability prediction model 308 may be trained on the sequence classification task usingindustry-standard techniques (e.g.. an Adam optimizer, gradient descent, etc.) for 10 epochs with a batch size of 96. learning rate of 5e-05, and a weight decay of 0.001.

[0058] At block 418, the model training engine 212 stores the stability prediction model 308 in the model data store 208. The method 400 then proceeds to an end block and terminates.

[0059] In one non-limiting example, the following model training protocol may be used by applying a publicly available Transformer framework (e.g.. Hugging Face Transformers v 4.42) to the hyperparameter ranges and public dataset source described in the following table:

[0060] FIG. 5 A - FIG. 5C illustrate results of a non-limiting example embodiment of creating a stability training data set and training a stability prediction model, according to various aspects of the present disclosure. A RoBERT a-based base machine learning model 310 was trained on protein sequence data including ~1.4 million NB sequences from the Observatory Antibody Space (OAS) using masked language modeling (MLM). This pretrained model was fine-tuned using experimental results that indicate stability. A stability training data set including 1,391 NB sequences were assigned a binary label (O=unstable, l=stable). These nanobodies were almost exclusively tested as TagBFP fusion reporters driven off the ubiquitous CAG promoter in published studies, with 11 sequences being mCherry fusions. The dataset was derived from fluorescence reporter assays in 293T cells or HeLa cells. A combination of naturally derived NBs (n=100). rationally mutated variants (n=303). and sequences derived from viral mutagenesis + FACS screens for CS-NBs (n= 4 NBs, 988 variants) make up the dataset. Binary labels were used, whereby NBs with clearly soluble TagBFP or mCherry reporter expression under fluorescence microscopy were labeled stable, whereas NB-reporters showing aggregation, partially soluble or non-expressing phenotypes were labeled unstable.

[0061] In pilot tests, sub-sampling of stability labels from the fine-tuning dataset produced a dataset with balanced label categories (n = 651 sequences) that improved model performance (predictive balanced accuracy (Fl score): 0.75) relative to the full dataset with unbalanced categories (Fl: 0.48). Although it is difficult to compare Fl scores from balanced and unbalanced datasets, this highlights the importance ofcareful training set design. The base machine learning model 310 was subsequently fine-tuned on a balanced subsample of 583 NB sequences (Training: 466; Validation: 58; Test: 59 (39 unique NBs); 40 / 60% stable / unstable). To prevent excessive influence from the mutagenesis datasets, which are variants of 4 nanobodies, only 219 of 988 variants sequences were subsampled. Early stopping of training was implemented by identifying the epoch that produces the most optimal training and validation loss, before validation loss begins to reverse its trend and increase, a sign of model over-fitting (FIG. 5A). This strategy produced a model that distinguishes between stable and unstable sequences with a Fl score of 0.8 on unseen Test sequences (FIG. 5B). In contrast, a control model trained on shuffled labels produced a Fl score of 0.52, close to the expected random value of 0.5, while the pre-trained model (without fine-tuning) produced Fl of 0.61. Importantly, the expected stability’ score shifts were produced upon fine-tuning specifically on real, but not shuffled data (FIG. 5C). These results indicate a strong performance model able to generalize its predictions to unseen data.

[0062] Though binary classification between stable and unstable sequences based on the sequences themselves have been discussed, in some embodiments, these techniques may be enhanced. For example, in some embodiments, a larger library of phenotypically characterized NB variants may be used to increase the amount of available training data. The larger training space may improve the model’s ability to detect sequence patterns associated with various s tab i 1 i ty states and allow the classification thresholds to be refined by incorporating multi-class analyses that distinguish aggregated, partially stable, and fully cleared nanobodies. A numerical cutoff based on intensity distribution across NBs may be used to define Stable vs. Unstable labels. NBs having high intensity and low puncta may be labeled as stable, with the rest being unstable. A multi-task learning approach may be used, training the model on both puncta and intensity parameters using well characterized methods. By using such a multi-task learning approach that incorporates puncta and intensity metrics, the model may be trained to classify sequences as stable, unstable, or conditionally stable.

[0063] FIG. 6 is a schematic illustration of a non-limiting example embodiment of a data pipeline for generating sequences having desired stability characteristics,according to various aspects of the present disclosure. The pipeline 600 uses a tuned base machine learning model 610 and a decoder 614 to generate sequences having a desired change in stability compared to an input sequence 602.

[0064] The pipeline 600 starts with an input sequence 602, that undergoes tokemzation 604 to create an encoded input sequence 606. The input sequence 602, tokenization 604, and encoded input sequence 606 are similar to the input sequence 302, tokenization 304, and encoded input sequence 306 illustrated and described in FIG. 3, and so are not described again here for the sake of brevity. The input sequence 602 is a sequence for which a stability -modulated variant is desired. For example, the input sequence 602 may be a nanobody that is known to bind to a desired target, but that is more or less stable than desired for a particular application.

[0065] The encoded input sequence 606 is the provided to a generative model 608. where a tuned base machine learning model 610 processes the encoded input sequence 606 to generate a model output 612. As described in further detail below, the tuned base machine learning model 610 may be a version of the base machine learning model 310 that is fine-tuned on a set of training data that has the desired stability to encode characteristics of sequences having the desired stability. For example, the tuned base machine learning model 610 may be trained on protein sequences labeled as stable to encode characteristics of stable protein sequences, protein sequences labeled as unstable to encode characteristics of unstable protein sequences, or protein sequences labeled as conditionally stable to encode characteristics of conditionally stable protein sequences. The model output 612 of the tuned base machine learning model 610 may be an encoding that represents the characteristics of the desired stability. For example, the model output 612 may be a matrix having a first dimension that is the size of the vocabulary of the tokenization 604, and a second dimension of the length of the input sequence 602, where each cell of the matrix indicates a probability of the given amino acid residue being present at the given position in the sequence, assuming that the output sequence has the stability characteristic for which the tuned base machine learning model 610 was fine tuned.

[0066] The model output 612 is then provided to a decoder 614 to output an output sequence 616. The output sequence 616 is a sequence that represents a proteinhaving the desired level of stability. Any suitable decoding technique may be used for the decoder 614. One suitable technique that retains portions of the input sequence 602 that are relevant to the binding interface of the protein is illustrated in FIG. 8 and described in further detail below.

[0067] FIG. 7 is a flowchart that illustrates a non-limiting example embodiment of a method of training machine learning models to generate nanobody sequences having desired stability' changes, according to various aspects of the present disclosure. In the method 700. a base machine learning model 310 is fine tuned to encode a desired stability level, such that the resulting model can be used to generate encodings that may be decoded to create sequences having the desired stability level.

[0068] From a start block, the method 700 proceeds to block 702, where the model training engine 212 retrieves the trained base machine learning model 310 from the model data store 208. In some embodiments, the trained base machine learning model 310 may be trained using a method 400 such as the method 400 illustrated in FIG. 4 and described above.

[0069] At block 704, the model training engine 212 creates a stable training data set from the protein sequence data in the stability' training data set labeled as stable. The stable training data set may include the protein sequence data from the stability' training data set that is labeled as stable, while not including protein sequence data that is labeled as unstable. In some embodiments, the model training engine 212 may choose protein sequence data such that sources of the protein sequences (e.g., naturally derived, rationally mutated variants, sequences derived from viral mutagenesis and FACS screens for CS-NBs, etc.) are balanced in the stable training data set.

[0070] At block 706, the model training engine 212 fine tunes the base machine learning model 310 on a masked language modeling task using the stable training data set to create a stable generative model. To avoid confusion, the adjective “stable” in the term “stable training data set” and “stable generative model” refers to the fact that the stable training data set includes protein sequences labeled as stable, and the stable generative model encodes characteristics of such proteins. Similar techniques may be used to fine tune the tuned base machine learning model 610 to create the stable generative model as were used to fine tune the base machinelearning model 310, but with the stable training data set instead of the stability training data set. By using a model architecture such as BERT or RoBERTa. the tuned base machine learning model 610 is designed to account for each token’s context by considering both preceding and subsequent tokens, up to 75 positions away. This bidirectional contextual understanding enables the generation of the model output 612, which may be a matrix that signifies the probability distribution across the vocabulary of amino acids for each position in the sequence.

[0071] At block 708, the model training engine 212 creates an unstable training data set from protein sequence data in the stability training data set labeled as unstable, and at block 710, the model training engine 212 fine tunes the base machine learning model on a masked language modeling task using the unstable training data set to create an unstable generative model. Again, to avoid confusion, the adjective "unstable'’ in the term "‘unstable training data set” and “unstable generative model” refers to the fact that the unstable training data set includes protein sequences labeled as unstable, and the unstable generative model encodes characteristics of such proteins. Similar techniques may be used to fine tune the tuned base machine learning model 610 to create the unstable generative model as were used to fine tune the base machine learning model 310, but with the unstable generative model instead of the stability’ training data set.

[0072] At block 712, the model training engine 212 stores the stable generative model and the unstable generative model in the model data store 208. The method 700 then proceeds to an end block and terminates.

[0073] The method 700 describes the training of a stable generative model and an unstable generative model. However, the training of these two specific models should not be seen as limiting. In other embodiments, only one of a stable generative model or an unstable generative model may be trained. In still other embodiments, a generative model with a different stability goal may be trained instead of or in addition to the stable generative model and the unstable generative model. For example, a conditional generative model may be trained to generate sequences having conditional stability, using a training data set of protein sequences labeled as being conditionally stable.

[0074] FIG. 8 is a flowchart that illustrates a non-limiting example embodiment of a method of using machine learning models to generate sequences having desired changes in stability, according to various aspects of the present disclosure. In the method 800, a tuned base machine learning model 610 representing the desired level of stability is used to generate an encoding, which is then decoded to create a generated sequence having the desired level of stability.

[0075] From a start block, the method 800 proceeds to block 802, where a model execution engine 216 of the nanobody prediction computing system 210 receives an input sequence (e.g., input sequence 602) and an indication of a desired stability change. The indication of the desired stability change may indicate that a generated sequence that is more stable than the input sequence 602, less stable than the input sequence 602, more conditionally stable than the input sequence 602, or any other stability characteristic encoded in a tuned base machine learning model 610 is desired.

[0076] At block 804, the model execution engine 216 retrieves a generative model (e.g., generative model 608) from the model data store 208 associated with the desired stability change. For example, if the indication of the desired stability change indicated a desire for a sequence that is more stable, then the model execution engine 216 may retrieve the stable generative model trained at block 706 of method 700. As another example, if the indication of the desired stability change indicated a desire for a sequence that is more unstable, then the model execution engine 216 may retrieve the unstable generative model trained at block 710 of method 700.

[0077] At block 806, the model execution engine 216 provides the input sequence as input to the generative model 608 to generate an embedding as a model output (e.g., model output 612). Though not illustrated, in some embodiments, the model execution engine 216 may conduct tokenization 604 on the input sequence 602 prior to providing the input sequence to the generative model 608.

[0078] The method 800 then proceeds to a for-loop defined between for-loop start block 808 and for-loop end block 816, wherein each sequence position of the input sequence is processed. For example, if the input sequence 602 begins “MKRTEN,” then the first iteration through the for-loop will process the first position (i.e., ‘"M”),the second iteration through the for-loop will process the second position (i.e., “K”), and so on.

[0079] From the for-loop start block 808, the method 800 advances to decision block 810. where a determination is made regarding whether the amino acid residue at the sequence position in the input sequence 602 is binding related. Any suitable technique may be used to determine whether an amino acid residue at a sequence position is antigen-binding related. As one non-limiting example, binding-related sequence positions may be determined by mining Protein Data Bank (PDB) and Proteins, Interfaces, Structures, Assemblies (PISA) data for NB-target structures. More than 1,000 such structures are available, and the analysis may focus on 389 high-resolution X-ray structures. Any conserved NB backbone residues with nonzero binding roles across any NB may be filtered out as binding-related sequence positions, leaving 31 sequence positions for potential mutation. As another nonlimiting example, the interface of an input NB may be predicted, as NB primary sequences encode structural patterns. A stringent cutoff may be used to maximize preservation of target binding.

[0080] If it is determined that the amino acid residue at the sequence position is binding related, then the result of decision block 810 is YES, and the method 800 proceeds to block 812. At block 812, the model execution engine 216 uses an amino acid residue from the sequence position in the input sequence 602 for the sequence position in an output sequence 616. For example, if the amino acid residue at the sequence position in the input sequence 602 is “M,” then the amino acid residue at the sequence position in the output sequence 616 would also be “M.” The method 800 then proceeds to for-loop end block 816. By reusing the amino acid residues from the input sequence 602 for binding related sequence positions, the method 800 avoids affecting the binding characteristics of the generated sequence while still potentially making updates to other sequence positions.

[0081] Returning to decision block 810. if it is determined that the amino acid residue at the sequence position in the input sequence 602 is not binding related, then the result of decision block 810 is NO, and the method 800 proceeds to block 814. At block 814, the model execution engine 216 selects an amino acid residue based on the embedding for the sequence position in the output sequence 616. Any suitabletechnique may be used to select the amino acid residue based on the embedding. The embedding (i.e., the model output 612) represents a probability that each of the amino acid residues in the vocabulary would be present at the sequence position. Accordingly, in some embodiments, the model execution engine 216 may select an amino acid residue with the greatest probability for the sequence position. In some embodiments, the model execution engine 216 may select an amino acid residue using a top-k sampling strategy, wherein k amino acid residues having the top probabilities for the sequence position are considered, the probabilities of those k amino acid residues are re-normalized to sum to 1, and then an amino acid residue is randomly sampled from those k amino acid residues based on the re-normalized probabilities. In some embodiments, the model execution engine 216 may select an amino acid using a top-p sampling strategy, wherein amino acid residues having a cumulative probability less than the threshold p are filtered out, and the amino acid residue is randomly selected from the remaining amino acid residues.

[0082] The method 800 then proceeds to the for-loop end block 816. If further sequence positions remain to be processed, then the method 800 returns to for-loop start block 808 to process the next sequence position. Otherwise, if all of the sequence positions have been processed, then the method 800 advances from for-loop end block 816 to block 818. At block 818, the model execution engine 216 provides the output sequence 616 as a sequence having the desired change in stability7. The method 800 then proceeds to an end block and terminates.

[0083] FIG. 9 is a chart that illustrates results of analysis of sequences generated using an embodiment of the method illustrated in FIG. 8. Generated sequences contained a median of 3-4 mutations per sequence (range 1 to 7 mutations), comparable to those identified from past mutagenesis screens for conditional stability, as well as naturally unstable NBs. Importantly, FIG. 9 shows that modelgenerated sequences were distinct from training sequences and more similar to input “Self’ sequences, suggesting against model overfitting.

[0084] FIG. 10A - FIG. 10F further illustrate validation of sequences generated by the techniques described in FIG. 8. FIG. 10A shows a schema (left) of a NB- TagBFP-luc2 reporter. The reporter is inducible by Target co-expression (right). FIG. 10B illustrates a schematic for model validation. FIG. 10C indicates that thetransfected luciferase reporter shows model-predicted intracellular stability changes, with a 2-tailed paired t-test, with an adjusted p<0.05. n=7 median values, and 3 replicate per NB. FIG. 10D shows that intracellular stability changes are negatively correlated with conditional stability. For this chart, the p is the probability that the slope is zero. FIG. 10E and FIG. 10F show the results of a test of a randomly selected set of input / output NBs (3K74) for NB-TagBFP-luc2 protein level by Western Blot of transfected 293T cel lysate, confirming protein stability changes and dependency on ubiquitin proteasome (MG132 proteasome blocker). The Transfection-normalized densitometry quantification used 3 independent experiments.

[0085] Past publications by the inventors of the present disclosure confirm that NB stability patterns in 293T cells reproduce across various eukaryotes (e.g., human cells, mouse, zebrafish, fruit flies). Thus, 293T cells provide a gold-standard to validate NB intracellular stability for broader applications. To validate the model’s predictions, model-generated NB sequences from the Test set were randomly sampled. The sequences were codon-optimized for mammalian expression, synthesized as DNA, and genetically fused to a TagBFP-luciferase reporter via Gibson Assembly into the pCAG-TagBFP-Luc2 vector. The TagBFP-luc2 reporter was validated to detect conditional stability of the original CS-NB: dGBPl (FIG. 10 A). These plasmids were transfected into 293T cells and evaluated normalized luciferase activity 1 day later (FIG. 10B). The model modulated normalized luciferase reporter activity in the expected direction, with room for improvement (FIG. 10C). Importantly, stability changes in model-generated sequences correlated with changes in conditional stability, measured by change in fold induction of luciferase reporter activity upon target molecule co-expression (FIG. 10E). This result also indicates that model-modulated NBs retain their target binding properties. To confirm stability’ changes at the protein level and that model-generated stability modulation is dependent on the ubiquitin proteasome, 293T cells were transfected with either a C AG-driven input (3K74 - anti-DHFR NB) or model-generated variant NB fused to TagBFP-luc2, along with a pCAG-mCherry transfection control. The input sequence was validated to be intracellularly stable and the output sequence designed to be less stable. 20 hours post-transfection, additional wells of transfectedcells were treated with 10 uM MG132 for 6 hours prior to sample harvest. Western Blot with luciferase and mCherry antibodies, followed by densitometry, showed that luciferase protein levels correlated with luciferase reporter patterns, confirming that the model can modulate intracellular NB stability in the desired direction (FIG. 10E - FIG. 10F). The results further confirm that degradation is mediated by the ubiquitin proteasome (FIG. 10E - FIG. 10F). Thus, the method’s potential was validated as a rapid in silico platform for generating NBs suitable for intracellular application.

[0086] Although prior studies have employed ad hoc or low-throughput approaches to generate a handful of conditionally stable nanobodies, no existing platform systematically integrates (i) LLM-based predictive mutation design, (ii) structural residue mapping of non-interface positions across hundreds of NB-target complexes, and (iii) scalable, high-content imaging assays for real-time phenotyping. The combination of the model architectures described herein and use of a decoder that avoids introducing mutations into binding-related sequence positions allow for the drastic reduction of design times from months to days, eliminates guesswork, and expands feasibility to any newly discovered NB. This approach provides a scalable, adaptable strategy for modifying NB sequences for intracellular applications.

[0087] Though not illustrated, the techniques described above were physically verified in-cell in order to demonstrate a physical transformation. DNA that encoded three model-generated nanobody variants was chemically synthesized, cloned into plasmid DNA for reporter fusion, and transiently transfected into HEK-293T cells as luciferase-fusion reporters. Across seven biological replicates each, the variants produced statistically significant (p < 0.05, paired two-tailed t-test) changes in reporter activity relative to the cognate parental nanobody. Western blot of one nanobody confirmed corresponding protein changes, with degradation of the destabilized model output sequence being mediated by ubiquitin proteasome across three replicates. These results confirmed that the techniques described above yield a measurable alteration in the biochemical state of living matter.

[0088] FIG. 11 is a block diagram that illustrates a non-limiting example embodiment of a system of computing devices configured to generate sequences having desired stability characteristics, according to various aspects of the present disclosure. By splitting at least some of the functionality of the nanobody predictioncomputing system 210 between multiple, separated computing systems, technical benefits with respect to traffic control, maintenance, and security may be obtained.

[0089] As shown, the system includes an end user computing system 1102 that communicates with the nanobody prediction computing system 210. The nanobody prediction computing system 210 has been deployed using a user interface computing system 1104, a model execution computing system 1106, and one or more devices for providing the model data store 208.

[0090] In some embodiments, the end user computing system 1102 may include one or more computing devices that are capable of communicating with the nanobody prediction computing system 210 and presenting a user interface generated by the nanobody prediction computing sy stem 210. Any suitable types of computing devices may be used for the end user computing system 1102. including but not limited to laptop computing devices, desktop computing devices, tablet computing devices, and smartphone computing devices.

[0091] In some embodiments, the user interface computing system 1104 may include one or more computing devices that collectively provide the user interface engine 218 of the nanobody prediction computing system 210. In some embodiments, the user interface engine 218 generates the user interface (e.g., a web interface) that is presented by the end user computing system 1102. In some embodiments, the user interface engine 218 may provide an application programming interface that provides access to the functionality of the nanobody prediction computing system 210 to a stand-alone application running on the end user computing system 1102. Any type of computing device may be used to provide the user interface computing system 1104, including but not limited to a server computing device, a desktop computing device, a laptop computing device, or one or more computing devices of a cloud computing system.

[0092] In some embodiments, the model execution computing system 1106 may include one or more computing devices that collectively provide the model execution engine 216. The model execution computing system 1106 may be configured to load models to be executed from the model data store 208, in which the trained models have already been stored, and to execute the models to service requests received by the user interface engine 218. Any type of computing device may be used to providethe model execution computing system 1106, including but not limited to a server computing device, a desktop computing device, a laptop computing device, or one or more computing devices of a cloud computing system. In some embodiments, the model execution computing system 1106 may also provide the model training engine 212, while in other embodiments, pretrained models may be provided by the model data store 208, and so a model training engine 212 may not be present in the nanobody prediction computing system 210, or may be present but inactive.

[0093] Deploying the techniques described herein using the network topology illustrated in FIG. 11 provides various technical benefits. For example, trained model weights are kept private to the model data store 208 and model execution computing system 1106. As these trained model weights are never exposed to the user interface computing system 1104, the security of this sensitive information can be maintained. As another example, storing and executing the models separately from the end user computing system 1102 allows low-cost edge devices (such as the end user computing system 1102) to be used for interaction, while still providing the opportunity for massive storage in model data store 208 and massive computing power in model execution computing system 1106. if desired.

[0094] FIG. 12 is a flowchart illustrating a non-limiting example embodiment of a method of generating sequences having desired changes in stability, according to various aspects of the present disclosure. The method 1200 is an illustration of how the techniques described herein may be implemented using the network topolog}' illustrated in FIG. 11.

[0095] From a start block, the method 1200 proceeds to block 1202. where a user interface computing system 1104 causes a query interface to be presented by an end user computing system 1102. In some embodiments, the user interface computing system 1104 may provide a web interface to the end user computing system 1102, and the end user computing system 1102 may render the web interface on a display device to accept input from a user.

[0096] FIG. 13 illustrates a non-limiting example embodiment of a query interface, according to various aspects of the present disclosure. As shown, the query interface is fairly simple. A first input interface element allows a user to select a nanobody organism (e.g., Llama glama, Vicugna pacos, Camelius dromedarius, etc.). A secondinput interface element allows a user to specify a top number of mutation combinations to list in the output. A third input interface element allows the user to select a prediction type (e g., a generated sequence with increased stability, a generated sequence with increased instability, or both). A fourth input interface element allows the user to select one or more types of models to be executed (e.g.. stability prediction model, generative model, etc.). In some embodiments, additional types of models may also be available for execution, such as models that predict an interface of either the input sequence or the output sequence. A fifth input interface element allows the user to specify an input sequence. In some embodiments, the fifth input interface element may allow the user to select the input sequence from a database. In some embodiments, the fifth input interface element may allow the user to type or cut-and-paste an input sequence from another source. The reduced amount of information presented on the query interface simplifies options for input values and avoids large amounts of complex texts. This user-centric approach avoids common problems in existing tools, in which complex query and result interfaces are challenging for users to interact with.

[0097] Returning to FIG. 12, the method 1200 proceeds from block 1202 to block 1204, where the user interface computing system 1104 receives, from the end user computing system 1102, an input sequence 602 and an indication of a desired stability. The input sequence 602 may be provided via the fifth input interface element, and the indication of the desired stability may be determined based on the entries in the third input interface element and / or the fourth input interface element.

[0098] At block 1206, the user interface computing system 1104 transmits the input sequence 602 and the indication of the desired stability to a model execution computing system 1106. At block 1208, the model execution computing system 1106 retrieves one or more relevant models from the model data store 208. For example, if the indication of the desired stability indicates that the user desires an output sequence with increased stability, the model execution computing system 1106 may retrieve the stable generative model from the model data store 208, and so on. At block 1210, the model execution computing system 1 106 uses the one or more relevant models to generate one or more sequences having the desired stability. The retrieval use of a relevant model to generate output sequences having desired changesin stability is illustrated in FIG. 8 and described above. In the method 1200, since multiple generative models may be specified, the method 800 may be executed once for each specified model. In some embodiments, if the stability prediction model is selected, then the model execution computing system 1106 may also retrieve and execute the stability prediction model illustrated in FIG. 3. In some embodiments, if an interface prediction model is selected, such a model may also be retrieved from the model data store 208 and executed.

[0099] At block 1212. the model execution computing system 1106 transmits the generated one or more sequences to the user interface computing system 1104. and at block 1214, the user interface computing system 1104 causes a result interface to be presented by the end user computing system 1102 to present the one or more sequences. In some embodiments, additional information such as the predicted interface or the stability’ prediction of the input sequence may also be shown.

[0100] FIG. 14 is an illustration of a non-limiting example embodiment of a result interface, according to various aspects of the present disclosure. In the illustrated result interface, a stability prediction generated by the stability prediction model is shown, as is a predicted interface for the input sequence, and a plurality of output sequences generated by the selected generative model. As shown, the result interface may also provide the ability to download FASTA sequences or other machine- readable versions of the generated output sequences, such that the output may be integrated into downstream tools.

[0101] Returning to FIG. 12, at optional block 1216, the end user computing system 1102 causes synthesis of at least one of the generated one or more sequences. Any system and / or technique for synthesizing the generated sequences known to those of ordinary skill in the art may be used, and the resulting synthesized proteins may be used for any purpose. The actions of optional block 1216 are illustrated and described as optional because in some embodiments, the generated output sequences may be stored or transferred to another device or software component without synthesizing the proteins.

[0102] The method 1200 then proceeds to an end block and terminates.

[0103] While illustrative embodiments have been illustrated and described, it will be appreciated that various changes can be made therein without departing from thespirit and scope of the invention. As a non-limiting example, the discussion above relates primarily to sequences that represent nanobodies. In some embodiments, the analyzed and generated sequences may represent one or more of nanobodies; fusion proteins; spacer proteins; or proteins that are combinations of at least two of nanobodies, fusion proteins, and spacer proteins.EXAMPLES

[0104] The following paragraphs provide a numbered list of non-limiting example embodiments of the present disclosure.

[0105] Example 1: A computer-implemented method of training a machine learning model to predict stability of an amino acid sequence, the method comprising: training, by a computing system, a base machine learning model to encode characteristics of proteins of a library of proteins using known amino acid sequences representing the proteins of the library' of proteins as a first training data set; fine- tuning, by the computing system, the base machine learning model using amino acid sequences in a stability training data set that includes amino acid sequences labeled as representing stable proteins and amino acid sequences labeled as representing unstable proteins to create a stability prediction model that encodes characteristics of the stable proteins and the unstable proteins; and training, by the computing system, a classifier model using the amino acid sequences and labels of stability training data set to classify amino acid sequences as stable or unstable.

[0106] Example 2: The computer-implemented method of example 1, wherein the proteins of the library of proteins are camelid antibodies.

[0107] Example 3: The computer-implemented method of any one of examples 1-2, wherein the known-stable proteins and the known-unstable proteins include one or more of nanobodies; fusion proteins; spacer proteins; or proteins that are combinations of at least two of nanobodies, fusion proteins, and spacer proteins.

[0108] Example 4: The computer-implemented method of any one of examples 1-3, wherein the base machine learning model uses a BERT architecture.

[0109] Example 5: The computer-implemented method of example 4, wherein the BERT architecture is a RoBERTa architecture.

[0110] Example 6: The computer-implemented method of any one of examples 1-5, wherein the classifier model uses a feed-forward neural network architecture.

[0111] Example 7: The computer-implemented method of any one of examples 1-6, further comprising: tokenizing, by the computing system, the known amino acid sequences representing the library proteins.

[0112] Example 8: The computer-implemented method of any one of examples 1-7, wherein an Fl score is used as a loss metric.

[0113] Example 9: A computer-implemented method of generating one or more artificial amino acid sequences representing artificial proteins predicted to have a different level of stability’ than a protein represented by an input amino acid sequence, the method comprising: receiving, by a computing system, the input amino acid sequence; using, by the computing system, a tuned base machine learning model to generate an encoding of the input amino acid sequence, wherein the tuned base machine learning model was fine-tuned to encode characteristics of proteins having the different level of stability'; and generating, by the computing system, the one or more artificial amino acid sequences by decoding the encoding of the input amino acid sequence.

[0114] Example 10: The computer-implemented method of example 9, wherein the encoding represents probabilities of each amino acid residue in a vocabulary appearing at each position in the artificial amino acid sequence.

[0115] Example 11 : The computer-implemented method of example 10, wherein generating the one or more artificial amino acid sequences by decoding the encoding of the input amino acid sequence includes, for each of the one or more artificial amino acid sequences: for each position in the artificial amino acid sequence, selecting an amino acid residue from the vocabulary for the position based at least on the probabilities.

[0116] Example 12: The computer-implemented method of example 11. wherein selecting the amino acid residue from the vocabulary for the position based at least on the probabilities includes at least one of: selecting the amino acid residue from the vocabulary' having a highest probability7; selecting the amino acid residue from the vocabulary7using a top-k sampling strategy: or selecting the amino acid residue from the vocabulary using a top-p sampling strategy.

[0117] Example 13: The computer-implemented method of any one of examples 10- 11. wherein the input amino acid sequence includes one or more antigen binding- related positions; and wherein generating the one or more artificial amino acid sequences by decoding the encoding of the input amino acid sequence includes, for each of the one or more artificial amino acid sequences: for each position in the artificial amino acid sequence: in response to determining that the position is not an antigen binding-related position, selecting the amino acid residue from the vocabulary for the position based at least on the probabilities; and in response to determining that the position is an antigen binding-related position, selecting the amino acid residue from the input amino acid sequence at the position.

[0118] Example 14: The computer-implemented method of any one of examples 9-13, wherein the different level of stability is a higher level of stability or a higher level of instability.

[0119] Example 15: The computer-implemented method of any one of examples 9-14, wherein the artificial proteins and the protein represented by the input amino acid sequence include one or more of nanobodies; fusion proteins; spacer proteins; or proteins that are combinations of at least two of nanobodies, fusion proteins, and spacer proteins.

[0120] Example 16: The computer-implemented method of any one of examples 9-15, wherein the tuned base machine learning model is a BERT model.

[0121] Example 17: The computer-implemented method of example 16, wherein the BERT model is a RoBERTa model.

[0122] Example 18: The computer-implemented method of any one of examples 9- 17. wherein the different level of stability is an enhanced level of stability, and wherein the method further comprises: using the artificial amino acid sequence to generate the artificial protein having the enhanced level of stability in vivo.

[0123] Example 19: The computer-implemented method of example 18. wherein the artificial protein is used as a biosensor in vivo.

[0124] Example 20: The computer-implemented method of any one of examples 9- 19. wherein the different level of stability is a reduced level of stability, and wherein the method further comprises: using the artificial amino acid sequence to generate the artificial protein having the reduced level of stability in vivo.

[0125] Example 21 : The computer-implemented method of any one of examples 9-20. wherein the different level of stability is a conditional stability, and wherein the method further comprises: using the artificial amino acid sequence to generate the artificial protein having conditional stability in vivo.

[0126] Example 22: The computer-implemented method of any one of examples 9-21, wherein the machine learning model is trained using a method as recited in any one of example 1 to example 8.

[0127] Example 23: A non-transitory computer-readable medium having computerexecutable instructions stored thereon that, in response to execution by one or more processors of a computing system, cause the computing system to perform actions of a method as recited in any one of example 1 to example 8.

[0128] Example 24: A non-transitory computer-readable medium having computerexecutable instructions stored thereon that, in response to execution by one or more processors of a computing system, cause the computing system to perform actions of a method as recited in any one of example 9 to example 21.

[0129] Example 25: A system for generating predictions related to amino acid sequences that represent proteins, the system comprising: a user interface computing system; and a model execution computing system; wherein the user interface computing system is configured to: generate a query interface configured to receive an input amino acid sequence that represents a protein; wherein the model execution computing system is configured to: receive the input amino sequence from the user interface computing system; provide the input amino acid sequence to one or more machine learning models as input to generate one or more outputs; and transmit the one or more outputs to the user interface computing system; and wherein the user interface computing system is further configured to: generate a result interface to present the one or more outputs.

[0130] Example 26: The system of example 25, wherein the model execution computing system is further configured to: retrieve the one or more machine learning models from a model data store.

[0131] Example 27: The system of example 26, wherein the model execution computing system and the model data store are separate computing systems.

[0132] Example 28: The system of any one of examples 25-27, wherein the user interface computing system and the model execution computing system are separate computing systems.

[0133] Example 29: The system of any one of examples 25-28, wherein providing the input amino acid sequence to one or more machine learning models as input to generate one or more outputs includes one or more of: providing the input amino acid sequence to an interface prediction model configured to generate an output that indicates characteristics of an interface of the protein represented by the input amino acid sequence; providing the input amino acid sequence to a stability prediction model configured to generate an output that indicates a predicted stability of the protein represented by the input amino acid sequence; providing the input amino acid sequence to a stable generative model to generate a first encoding of the input amino acid sequence based on characteristics of amino acid sequences that represent known-stable proteins, and generating an output that includes a higher stability amino acid sequence by decoding the first encoding of the input amino acid sequence; or providing the input amino acid sequence to an unstable generative model to generate a second encoding of the input amino acid sequence based on characteristics of amino acid sequences that represent known-unstable proteins, and generating an output that includes a higher instability amino acid sequence by decoding the second encoding of the input sequence.

Claims

CLAIMSThe embodiments of the invention in which an exclusive property or privilege is claimed are defined as follows:

1. A computer-implemented method of training a machine learning model to predict stability of an amino acid sequence, the method comprising: training, by a computing system, a base machine learning model to encode characteristics of proteins of a library of proteins using known amino acid sequences representing the proteins of the library of proteins as a first training data set; fine-tuning, by the computing system, the base machine learning model using amino acid sequences in a stability training data set that includes amino acid sequences labeled as representing stable proteins and amino acid sequences labeled as representing unstable proteins to create a stability’ prediction model that encodes characteristics of the stable proteins and the unstable proteins; and training, by the computing system, a classifier model using the amino acid sequences and labels of stability training data set to classify amino acid sequences as stable or unstable.

2. The computer-implemented method of claim 1, wherein the proteins of the library of proteins are camelid antibodies.

3. The computer-implemented method of claim 1, wherein the known-stable proteins and the known-unstable proteins include one or more of nanobodies; fusion proteins; spacer proteins; or proteins that are combinations of at least two of nanobodies, fusion proteins, and spacer proteins.

4. The computer-implemented method of claim 1, wherein the base machine learning model uses a BERT architecture.

5. The computer-implemented method of claim 4, wherein the BERT architecture is a RoBERTa architecture.

6. The computer-implemented method of claim 1, wherein the classifier model uses a feedforward neural network architecture.

7. The computer-implemented method of claim 1. further comprising:tokenizing, by the computing system, the known amino acid sequences representing the library proteins.

8. The computer-implemented method of claim 1, wherein an Fl score is used as a loss metric.

9. A computer-implemented method of generating one or more artificial amino acid sequences representing artificial proteins predicted to have a different level of stability than a protein represented by an input amino acid sequence, the method comprising: receiving, by a computing system, the input amino acid sequence; using, by the computing system, a tuned base machine learning model to generate an encoding of the input amino acid sequence, wherein the tuned base machine learning model was fine-tuned to encode characteristics of proteins having the different level of stability; and generating, by the computing system, the one or more artificial amino acid sequences by decoding the encoding of the input amino acid sequence.

10. The computer-implemented method of claim 9, wherein the encoding represents probabilities of each amino acid residue in a vocabulary appearing at each position in the artificial amino acid sequence.

11. The computer-implemented method of claim 10, wherein generating the one or more artificial amino acid sequences by decoding the encoding of the input amino acid sequence includes, for each of the one or more artificial amino acid sequences: for each position in the artificial amino acid sequence, selecting an amino acid residue from the vocabulary for the position based at least on the probabilities.

12. The computer-implemented method of claim 11, wherein selecting the amino acid residue from the vocabulary for the position based at least on the probabilities includes at least one of: selecting the amino acid residue from the vocabulary having a highest probability’; selecting the amino acid residue from the vocabulary using a top-k sampling strategy; or selecting the amino acid residue from the vocabulary using a top-p sampling strategy.

13. The computer-implemented method of claim 10, wherein the input amino acid sequence includes one or more antigen binding-related positions; and wherein generating the one or more artificial amino acid sequences by decoding the encoding of the input amino acid sequence includes, for each of the one or more artificial amino acid sequences: for each position in the artificial amino acid sequence: in response to determining that the position is not an antigen binding- related position, selecting the amino acid residue from the vocabulary for the position based at least on the probabilities; and in response to determining that the position is an antigen binding- related position, selecting the amino acid residue from the input amino acid sequence at the position.

14. The computer-implemented method of claim 9, wherein the different level of stability is a higher level of stability or a higher level of instability.

15. The computer-implemented method of claim 9, wherein the artificial proteins and the protein represented by the input amino acid sequence include one or more of nanobodies; fusion proteins; spacer proteins; or proteins that are combinations of at least two of nanobodies, fusion proteins, and spacer proteins.

16. The computer-implemented method of claim 9, wherein the tuned base machine learning model is a BERT model.

17. The computer-implemented method of claim 16, wherein the BERT model is a RoBERTa model.

18. The computer-implemented method of claim 9, wherein the different level of stability is an enhanced level of stability, and wherein the method further comprises: using the artificial amino acid sequence to generate the artificial protein having the enhanced level of stability in vivo.

19. The computer-implemented method of claim 18, wherein the artificial protein is used as a biosensor in vivo.

20. The computer-implemented method of claim 9, wherein the different level of stability is a reduced level of stability, and wherein the method further comprises:using the artificial amino acid sequence to generate the artificial protein having the reduced level of stability in vivo.

21. The computer-implemented method of claim 9, wherein the different level of stability is a conditional stability, and wherein the method further comprises: using the artificial amino acid sequence to generate the artificial protein having conditional stability in vivo.

22. The computer-implemented method of claim 9, wherein the machine learning model is trained using a method as recited in any one of claim 1 to claim 8.

23. A non-transitory computer-readable medium having computer-executable instructions stored thereon that, in response to execution by one or more processors of a computing system, cause the computing system to perform actions of a method as recited in any one of claim 1 to claim 8.

24. A non-transitory computer-readable medium having computer-executable instructions stored thereon that, in response to execution by one or more processors of a computing system, cause the computing system to perform actions of a method as recited in any one of claim 9 to claim 21.

25. A system for generating predictions related to amino acid sequences that represent proteins, the system comprising: a user interface computing system; and a model execution computing system; wherein the user interface computing system is configured to: generate a query interface configured to receive an input amino acid sequence that represents a protein; wherein the model execution computing system is configured to: receive the input amino sequence from the user interface computing system; provide the input amino acid sequence to one or more machine learning models as input to generate one or more outputs; and transmit the one or more outputs to the user interface computing system; and wherein the user interface computing system is further configured to: generate a result interface to present the one or more outputs.

26. The system of claim 25, wherein the model execution computing system is further configured to: retrieve the one or more machine learning models from a model data store.

27. The system of claim 26, wherein the model execution computing system and the model data store are separate computing systems.

28. The system of claim 25, wherein the user interface computing system and the model execution computing system are separate computing systems.

29. The system of claim 25, wherein providing the input amino acid sequence to one or more machine learning models as input to generate one or more outputs includes one or more of: providing the input amino acid sequence to an interface prediction model configured to generate an output that indicates characteristics of an interface of the protein represented by the input amino acid sequence; providing the input amino acid sequence to a stability prediction model configured to generate an output that indicates a predicted stability of the protein represented by the input amino acid sequence; providing the input amino acid sequence to a stable generative model to generate a first encoding of the input amino acid sequence based on characteristics of amino acid sequences that represent known-stable proteins, and generating an output that includes a higher stability ammo acid sequence by decoding the first encoding of the input ammo acid sequence; or providing the input amino acid sequence to an unstable generative model to generate a second encoding of the input amino acid sequence based on characteristics of amino acid sequences that represent known-unstable proteins, and generating an output that includes a higher instability amino acid sequence by decoding the second encoding of the input sequence.

Images