Deep learning for de novo antibody affinity maturation (modification) and property improvement

Abstract

Controlling the affinity and expression of antibodies is key for clinical applications. High affinity antibodies are associated with higher specificity and can be used at lower doses. Currently, antibody maturation is addressed by directed evolution methods. In this case, an initial library of mutagenized binders is seeded into the process and the affinity is improved through multiple rounds of mutagenesis and selection. However, the present disclosure employs a machine learning approach to compute mature antibody sequences using a similar process to directed evolution. These antibody sequences can be manufactured into physical antibodies after they are computed and validated. Additionally, the present method has the potential to outperform directed evolution in targeting specific affinities and is applicable to protein-protein interactions in general.

Description

[0001] Cross-reference to related applications

[0002] This application claims the benefit of U.S. Provisional Application No. 63 / 057,376, filed July 28, 2020. The entire teachings of the aforementioned application are incorporated herein by reference. Background Technology

[0003] Antibody maturation is the process of increasing the affinity of a given antibody for its antigen. In immunology, an antigen is a toxin or foreign substance that triggers an immune response. An example of such an immune response is the production of antibodies that bind to the antigen to fight it. Antibodies can also be designed using a variety of methods. Summary of the Invention

[0004] Controlling antibody affinity and expression is crucial for clinical applications. High-affinity antibodies are associated with higher specificity and can be used at lower doses. Currently, antibody maturation is addressed through directed evolution. In this approach, an initial library of mutant conjugates is seeded into the process, and affinity is increased through multiple rounds of mutation and selection. However, this disclosure employs a machine learning approach to compute mature antibody sequences using a process similar to directed evolution. These antibody sequences, after computation and validation, can be fabricated into physical antibodies. Furthermore, this method has the potential to outperform directed evolution in targeting specific affinities and is applicable to general protein-protein interactions.

[0005] In this embodiment, computational directed evolution (CDE) can be used in multi-objective and multi-model scenarios, such as when two or more features are selected for improvement substantially simultaneously. In this case, one or more models can be used. Each of these models can have one or more features to be optimized. These one or more models can be combined into a single objective function, and CDE can optimize this objective function. This objective function is optimized using an optimization procedure (in this case, computational directed evolution). When the objective function is optimized, it produces antibody sequences that are improved in one or more antibody features of interest.

[0006] In one embodiment, a method for determining antibody sequences with improved properties includes training a machine learning model based on a first set of antibody sequences. Each antibody sequence in a database is labeled with one or more corresponding properties. The method further generates a fine-tuned machine learning model by training the machine learning model based on a second plurality of antibody sequences, each of which is labeled with a corresponding property. The second plurality of antibody sequences are associated with an antigen of interest. The method further includes generating antibody sequences based on the fine-tuned machine learning model.

[0007] Those skilled in the art will recognize that fine-tuning a machine learning model (also known as fine-tuning a machine learning model) is a machine learning model that is first trained on a general dataset and then fine-tuned on a more specific dataset. Fine-tuning can also be described as the process of taking a machine learning model that has been trained for a given task and training it to perform a second task.

[0008] In one embodiment, a method for determining antibody sequences with improved properties includes generating a score for each of a plurality of machine learning models. Each machine learning model is trained based on a corresponding plurality of antibody sequences. Each antibody sequence in the plurality of antibody sequences is labeled with a property corresponding to that plurality of antibody sequences and a value for that property. Generating a score for each machine learning model yields a corresponding score for each of the plurality of machine learning models, the corresponding score indicating the contribution used to predict the property corresponding to the corresponding machine learning model. The method further includes generating antibody sequences using the plurality of machine learning models by weighting the output of each machine learning model according to each generated score and combining the weighted outputs into a weighted sum.

[0009] In various embodiments, contributions include one or more of the following: the importance of the property to manufacturing, protein expression, patient immunogenicity, expression, exploitability, interaction with other models, orthogonality with other models, and empirical derivations obtained by adjusting the generation process.

[0010] In this embodiment, generating an antibody sequence further includes selecting an antibody sequence from a proposed distribution based on a fine-tuned machine learning model. Generating an antibody sequence further includes determining whether the selected antibody sequence has an acceptance probability exceeding a specific threshold; if so, analyzing the antibody; otherwise, selecting the next antibody sequence from the proposed distribution.

[0011] In one embodiment, generating the antibody sequence further includes comparing a first characteristic value, determined by a fine-tuned machine learning model, of an antibody selected from the proposed distribution with a second characteristic value, determined by the fine-tuned machine learning model, of the antibody having the best characteristic value in the current search. If the first characteristic value is greater than the second characteristic value, the method replaces the antibody with the best characteristic value with the antibody selected from the proposed distribution.

[0012] In an embodiment, training the machine learning model further includes providing a set of amino acid sequences labeled with at least one characteristic. Training the machine learning model further includes masking a portion of the amino acid sequence set to provide a masked set of amino acid sequences. The remainder of the amino acid sequence set is an unmasked set of amino acid sequences. Training the machine learning model further includes training the machine learning model to estimate, based on (1) at least one characteristic labeled for each masked amino acid sequence in the masked set and (2) the labeled characteristic of the unmasked set of amino acid sequences and each unmasked amino acid sequence.

[0013] In an embodiment, generating a fine-tuned machine learning model further includes weighting each sequence feature in the second plurality of antibody sequences. Generating a fine-tuned machine learning model further includes using the machine learning model to determine optimal model parameters for generating the second plurality of antibody sequences. Generating a fine-tuned machine learning model further includes applying the optimal model parameters to the machine learning model. The model obtained after applying the optimal model parameters is the fine-tuned machine learning model.

[0014] In this embodiment, the corresponding property is affinity (e.g., binding affinity) or expression. In other embodiments, examples of properties (e.g., function values) may be one or more of the following: binding affinity, binding specificity, catalytic (e.g., enzyme) activity, fluorescence, solubility, thermal stability, conformation, immunogenicity, protein aggregation, proteolytic stability, expression, off-target effects, and any other functional property of the biopolymer sequence. This process is applicable to any protein for which we have a (small) starting set of examples with the property of interest. Based on this, we can then use the process described herein to modify the property of interest to better meet our needs. This can be an increase in value (e.g., increased catalytic rate), a decrease in property value (e.g., decreased immunogenicity), or reaching a target value or range of values ​​(e.g., achieving a specific binding affinity).

[0015] In one embodiment, the method includes selecting candidate antibody sequences from a proposed distribution based on a fine-tuned machine learning model, the antibody sequences being within defined acceptance criteria. The method further includes replacing the best-discovered antibody sequence with the candidate antibody sequence if the characteristics of the candidate antibody sequence are superior; otherwise, ignoring the candidate antibody sequence.

[0016] In one embodiment, the method further includes generating an antibody having the generated antibody sequence. In another embodiment, the method may further include providing a manufactured antibody having the generated antibody sequence and determining the properties of the antibody.

[0017] In one embodiment, a system for determining antibodies with improved properties includes a processor and a memory having computer code instructions stored thereon. The processor and the memory having the computer code instructions are configured to train a machine learning model based on a first plurality of antibody sequences. Each antibody sequence in a database is labeled with a corresponding property. The processor is further configured to generate a fine-tuned machine learning model by training the machine learning model based on a second plurality of antibody sequences and their corresponding properties. The second plurality of antibody sequences are associated with an antigen of interest. The processor is further configured to generate antibody sequences based on the fine-tuned machine learning model.

[0018] In an embodiment, an antibody maturation method may include providing a first antibody sequence to the system or method described above, and obtaining the generated antibody sequence from the system.

[0019] In the embodiments, the isolated antibody can be generated by the method described above. In the embodiments, the isolated antibody is generated by recombinant synthesis. In the embodiments, the isolated antibody is chemically synthesized.

[0020] In one embodiment, a method includes determining or generating antibody sequences with improved properties, the determination or generation being performed by fine-tuning a machine learning model. The fine-tuning machine learning model can be generated by: (1) training a machine learning model based on a first plurality of antibody sequences, each antibody sequence in the first plurality of antibody sequences labeled with a corresponding property, and (2) generating a fine-tuning machine learning model by training a machine learning model based on a second plurality of antibody sequences, each antibody sequence in the second plurality of antibody sequences labeled with a corresponding property, the second plurality of antibody sequences being associated with an antigen of interest. Optionally, training the machine learning model, generating the fine-tuning machine learning model, or both can be performed by a third party independently of generating the antibody sequences.

[0021] As used herein, an antibody sequence refers to an ordered amino acid sequence that can be stored digitally or in other formats. An antibody refers to the physical manifestation of an antibody. Those skilled in the art will recognize that antibody sequences generated using the systems and methods disclosed herein can be manufactured or otherwise produced as antibodies.

[0022] In one embodiment, a method for determining antibody sequences with improved properties includes training multiple machine learning models. Each machine learning model is trained based on a corresponding initial plurality of antibody sequences. Each antibody sequence in the initial plurality of antibody sequences is labeled with a corresponding property. Those skilled in the art will understand that each corresponding machine learning model can be trained with different plurality of antibody sequences. The method then generates multiple fine-tuned machine learning models. Each fine-tuned machine learning model is generated by training each machine learning model with a second plurality of antibody sequences. Each antibody sequence in the second plurality of antibody sequences is labeled with a corresponding property, and the second plurality of antibody sequences are associated with a target antibody. The method then uses the multiple fine-tuned machine learning models, weighted by corresponding hyperparameters, based on an objective function to generate antibody sequences.

[0023] In the embodiments, one or more of the antibody sequences are associated with the antigen of interest.

[0024] In one embodiment, a method for determining antibody sequences with improved properties includes providing scores for each of a plurality of machine learning models, each machine learning model being trained based on a corresponding plurality of antibody sequences. Each of the corresponding plurality of antibody sequences is labeled with a property corresponding to that plurality of antibody sequences and a value for that property. Each corresponding score of each of the plurality of machine learning models indicates a contribution for predicting the property corresponding to that corresponding machine learning model. The method further includes generating antibody sequences using the plurality of machine learning models by weighting the output of each machine learning model according to each provided score and combining the weighted outputs into a weighted sum.

[0025] In one embodiment, a method for determining antibody sequences with improved properties includes generating antibody sequences using multiple machine learning models. The generation of antibody sequences is performed by weighting the output of each machine learning model based on a score of each of the multiple machine learning models. Each machine learning model is trained based on a corresponding plurality of antibody sequences. Each antibody sequence in the corresponding plurality of antibody sequences is labeled with a property corresponding to that plurality of antibody sequences and a value for that property. Each corresponding score of each machine learning model in the plurality of machine learning models indicates a contribution for predicting the property corresponding to that corresponding machine learning model. The method further includes combining the weighted outputs into a weighted sum. Attached Figure Description

[0026] This patent or application document contains at least one color drawing. Upon request and after payment of the necessary fees, a copy of this patent or patent application publication containing one or more color drawings will be provided by the Patent Office.

[0027] The foregoing will become apparent from the more specific description of the exemplary embodiments that follows, as illustrated in the accompanying drawings, in which the same reference numerals denote the same parts in different figures. The drawings are not necessarily to scale, but rather focus on illustrating the embodiments.

[0028] Figure 1A This is a block diagram illustrating an example embodiment of the method disclosed herein.

[0029] Figure 1B This is a flowchart illustrating an example embodiment of the process used in this disclosure.

[0030] Figure 2 This is a diagram illustrating random walk sequences in the antibody sequence space used for computational directed evolution to discover antibody sequences.

[0031] Figure 3 This is a graph illustrating the improvement of the generated antibody sequences compared to existing datasets.

[0032] Figure 4 The illustration depicts a computer network or similar digital processing environment in which embodiments of the invention can be implemented.

[0033] Figure 5 yes Figure 4 A diagram illustrating an example of the internal structure of a computer (e.g., a client processor / device or a server computer) in a computer system. Detailed Implementation

[0034] The example embodiment is described below.

[0035] Figure 1A This is a block diagram 100 illustrating an example embodiment of the method described in this disclosure. This disclosure employs a de novo approach for closed-loop antibody affinity maturation. The iterative process begins with unsupervised pre-training of a deep machine learning model on a large dataset of relevant protein sequences and their properties (102). In an embodiment, this includes unsupervised pre-training of a language model on a dataset containing n antibody sequences. This enables the model to learn the potential statistical relationships between amino acids in the antibody sequences.

[0036] The process then fine-tunes the pre-trained model on a smaller supervised learning task using data specific to the desired antibody-antigen pair (104). This fine-tuned machine learning model is jointly trained with both affinity and expression as data specific to the desired antibody-antigen pair, but in practice, any number or type of additional features can be used during the fine-tuning process. Once the fine-tuned model is trained (104), it is used downstream to perform affinity maturation by solving an optimization problem further described below. After fine-tuning, a constrained search is performed on the antibody sequence space using directed evolution to obtain high-affinity antibody sequences for selected targets. The solution to this optimization problem is one or more affinity-matured antibody sequences (e.g., sequences predicted by the machine learning model to have higher affinity than observed in the supervised training task) (106). The method then constructs these affinity-matured antibody sequences (or improved antibody sequences). The method further experimentally determines the affinity and expression of the constructed antibody sequences. New data from the determined sequences are incorporated into the supervised learning dataset, and process 100 is repeated until suitable candidates are found (102). In an embodiment, the limiting condition for high affinity candidates may be an equilibrium dissociation constant (K0) of less than 10 picomoles (pM). D (For example, in the case of antifluorescein antibodies).

[0037] As used herein, an "antibody" refers to an immunoglobulin molecule capable of specifically binding to a target (such as carbohydrates, polynucleotides, lipids, peptides, etc.) through at least one antigen recognition site located in the variable region of an immunoglobulin molecule. As used herein, the term "antibody" encompasses not only complete (i.e., full-length) monoclonal antibodies, but also their antigen-binding fragments (such as Fab, Fab', F(ab')2, Fv), single-chain variable fragments (scFv) and mutants, fusion proteins containing antibody moieties, humanized antibodies, chimeric antibodies, biantibodies, linear antibodies, single-chain antibodies, single-domain antibodies (e.g., camel or llama VHH antibodies), multispecific antibodies (e.g., bispecific antibodies), and any other modified conformation of an immunoglobulin molecule containing an antigen recognition site of desired specificity, including glycosylated variants of antibodies, amino acid sequence variants of antibodies, and covalently modified antibodies.

[0038] As used in this article, the term "K" D Also known as the "binding constant," "equilibrium dissociation constant," or "affinity constant," it is a measure of the degree of reversible association between two molecular species (e.g., antibody and target protein) and includes both actual binding affinity and apparent binding affinity. Binding affinity can be determined using methods known in the art, including, for example, by measuring surface plasmon resonances, such as using the BIAcore system and assays.

[0039] In some respects, antibodies and K D Less than 10 -4 M, 10 -5 M, 10 -6 M, 10 -7 M, 10 -8 M, 10 -9 M, 10 -10 M, 10 -11 M or 10 -12 M binds to its target protein. For example, an antibody can bind to K. D Less than 1000 nM, or alternatively less than 900 nM, or alternatively less than 800 nM, or alternatively less than 700 nM, or alternatively less than 600 nM, or alternatively less than 500 nM, or alternatively less than 400 nM, or alternatively less than 300 nM, or alternatively less than 200 nM, or alternatively less than 100 nM, or alternatively less than 90 nM, or alternatively less than 80 nM, or alternatively less than 70 nM, or alternatively less than 60 nM, or alternatively less than 50 nM, or alternatively less than 40 nM, or alternatively... The target protein binds to a concentration of less than 30 nM, or less than 20 nM, or less than 15 nM, or less than 10 nM, or less than 9 nM, or less than 8 nM, or less than 7 nM, or less than 6 nM, or less than 5 nM, or less than 4 nM, or less than 3 nM, or less than 2 nM, or less than 1 nM, or less than 1000 pM, or less than 100 pM, or less than 10 pM, or less than 1 pM.

[0040] In some aspects, the antibody sequence identified by the methods disclosed herein corresponds to an antibody that binds to a target protein having an affinity of at least 70%, or alternatively at least 75%, or alternatively at least 80%, or alternatively at least 85%, or alternatively at least 90%, or alternatively at least 95%, or higher than that of a reference antibody. In some aspects, the antibody binds to a target protein having a higher affinity than that of a reference antibody. The reference antibody may be, for example, an antibody with the highest reported (e.g., published) affinity for a given target.

[0041] In practice, Figure 1A The process described herein rapidly increases antibody affinity after one round, generating a fluorescein antibody sequence with an affinity greater than 10 pM. This process can reduce the cost and time required to generate better therapeutic antibodies by reducing the number of constructs that need to be manufactured and tested. The process is described in detail below.

[0042] In the initial experiments, the process employed two datasets: a large set of unsupervised antibody sequences and a smaller set of sequence affinity pairs for the antibodies of interest, used to compute bootstrapping maturation. Specifically, pre-training used a complete antibody library taken from a single individual. This dataset is described in more detail in Bryan et al., “Commonality despite exceptional diversity in the baseline human antibody repertoire,” *Nature* 566.7744(2019):393 (hereinafter referred to as “Bryan”), which is incorporated herein by reference in its entirety. Unsupervised Dataset It contains approximately 300 million antibody sequences. For fine-tuning, high-throughput fluorophore antibody data from Adams, Rhys M. et al., “Measuring the sequence-affinity landscape of antibodies with massively paralleltitration curves,” *eLife* 5(2016):e23156 (hereinafter referred to as “Adams”), is used, which is incorporated herein by reference in its full text. This example supervised dataset Composed of 2803 sequences, these sequences were paired with expression and affinity data and evaluated in triplicate. (Source: This dataset) Examples are tuples that contain protein sequences, affinity measurements, and expression measurements, respectively.

[0043] The machine learning model is trained to act as a feature oracle for affinity and expression properties. The model used in this process is called "Omniprot," an adaptation of the BERT masked language model, which is described in more detail in Devlin, Jacob, et al., "Bert: Pre-training of deep bidirectional transformers for language understanding," arXiv:1810.04805 (2018) (hereinafter referred to as "Devlin"), which is incorporated herein by reference in its entirety. However, in principle, any model that can be pre-trained in an unsupervised manner can be used. Omniprot is a deep transformer model that learns to reconstruct masked portions of protein sequences. maskedDuring training, 15% of the amino acids in the protein sequence are randomly masked. The training task can be described as follows:

[0044]

[0045] Where θ is a vector of Omniprot parameters, and These represent the network parameters, which are the optimal solution to the unsupervised pre-training problem. In other words, Omniprot is evaluated based on the accuracy with which it guesses the masked amino acids. The training objective teaches Omniprot to model statistical regularities in the basic protein sequence space. In this case, as a form of transfer learning, Omniprot is applied to an unsupervised antibody sequence dataset. The model is trained on top of this to improve its performance on smaller downstream supervised tasks. Devlin further discloses various aspects of the unsupervised pre-training of BERT-like models and their architectures.

[0046] Once the model parameter set is obtained from the pre-training objective... This allows for fine-tuning the model using a smaller supervised dataset. In this example embodiment, fine-tuning refers to solving a new optimization problem using a smaller dataset after initializing the model with pre-trained parameters at the start of training, thus adapting the model to the new task. Therefore, the fine-tuning optimization problem can be expressed as:

[0047]

[0048] in, These are hyperparameters that weight each sequence feature of interest, and Given a training dataset The optimal set of network weights. The optimal parameters, together with the model architecture, constitute the oracle, which can now be used for downstream optimization of antibody affinity.

[0049] Once the model is trained, it can be used to find antibody sequences with improved affinity. This process is called sequence maturation. In an example embodiment, the fine-tuned Omniprot model can predict the affinity and expression of candidate sequences and search within the effective domain of possible antibody sequences. The optimization problem can be expressed as:

[0050]

[0051] in, This is a fine-tuned Omniprot model mapping from sequence to affinity. This optimization problem can be easily solved in various ways by applying different optimization methods. However, an important consideration is limiting the domain (e.g., protein sequences) to a set of effective antibody sequences. To solve this problem while taking into account the desired sequence constraints, an optimization procedure called computational directed evolution (CDE) can be applied. CDE is a Markov chain Monte Carlo (MCMC) technique applied to protein optimization. The MCMC process of CDE uses a proposal distribution specified by the marginal probabilities of mutations in the MSA of the training set. This process demonstrates how each mutation improves the affinity predicted by the expected model. As iterating, more mutations accumulate in the original sequences that improve the predicted binding affinity, ultimately producing strong candidates for testing in the laboratory. More specifically, CDE is an implementation of the Metropolis-Hastings method. In this example embodiment, CDE is a biased random walk of the effective protein sequence space, where the highest affinity antibody found is returned at a selected end of the walk.

[0052] To implement the Metropolis-Hastings approach, a collection of objects is used as input:

[0053] a) Initial starting antibody sequence

[0054] b) Suggest a distribution p(s_{t+1)|s_t), and

[0055] c) Acceptance Criteria

[0056] in, It is the set of all antibody sequences in the training set of the Omniprot fine-tuning process, and It is the set of all sequences that can be extracted from the marginal distribution of amino acids in the aligned training set. Therefore, the acceptance criterion g is a mapping from a pair of sequences (e.g., an antibody sequence and a next candidate sequence) to a unit interval indicating the probability of accepting the candidate sequence as the next step in a random walk. In this example embodiment of the method, the acceptance criterion is defined as:

[0057]

[0058] Where s and s c Let be the current sequence and the candidate sequence, respectively, and α be a positive constant. In principle, many such acceptance criteria can be used and are effective; however, in practice, the criteria described above have been found to be effective.

[0059] In view of the above, the CDE method is as follows (note that this is one implementation of Metropolis-Hastings):

[0060] a) Starting with the initial antibody sequence:

[0061] b) Initialize the current best sequence:

[0062] c) Extract new candidate sequences from the proposed distribution:

[0063] d) Probability: g(s, s) c ),set up

[0064] e) If but

[0065] f) Return to step (c) until n > max_iterations

[0066] Designers may wish to optimize multiple properties of an antibody sequence. To do this, models can be used to estimate each property of the antibody sequence, and the objective function uses hyperparameters to combine the weighted results of the various models. Hyperparameters and their usage are described below.

[0067] Considering the scenario of optimizing two objectives, those skilled in the art will recognize that any two properties can be analyzed. However, for this example, binding affinity and antibody solubility are used for illustration. For instance, a first model determines the binding affinity of the sequence to our target (m1(s)). A second model determines the antibody solubility (m2(s)). Both models take the sequence(s) as input and return a scalar representing the measured property. A joint objective function (e.g., an objective function) can combine the two scalar outputs as follows:

[0068] objective(s):=(1-α)m1(s)+αm2(s)

[0069] Here, α lies on a closed interval between 0 and 1. Determining this hyperparameter α is part of the practice of finding a good objective function, which may require testing. Typically, the objective can be represented as a weighted sum of a set of models. Optimizing multiple models involves determining how to weight each model when combining models to obtain the desired antibody sequence. This optimization can be done manually, as antibody designers may prioritize certain properties over others. Hyperparameters can represent one or more of the following: the importance of a property to manufacturing (e.g., manufacturability), protein expression, patient immunogenicity, exploitability, interaction with other models, orthogonality with other models, and empirical derivations derived by adjusting the generation process. Manufacturability is a factor based on the ease with which a protein sequence (e.g., a drug) can be created using standard biochemical techniques. Manufacturability factors include: ease of protein expression, likelihood of protein aggregation, protein stability, etc. These concerns all relate to the cost and feasibility of production. Exploitability is a factor based on properties related to the clinical success of a protein sequence (e.g., a drug). Exploitability factors include: ease of protein expression, likelihood of protein aggregation, protein stability, target specificity, etc.

[0070] Typically, for n models, the objective function for a multi-model type is expressed as:

[0071]

[0072] Wherein, weight w i Take a value between 0 and 1, and have all weights w i The sum is 1. When only one feature is optimized, n = 1 and w i =1. However, the designers configured the weights for the multi-model objective function.

[0073] Figure 1B This is a flowchart 150 illustrating an example embodiment of the process employed in this disclosure. The process begins with the selection of an initial antibody s0 (152). The initial antibody s0 is the antibody to be improved through a machine learning process. For initialization, the method first selects the best-discovered antibody s0. b Set it to s0. Then, the method uses the proposed distribution as defined above. Extracting new sequence s from c (156). In other words, this proposed distribution selects a new antibody from the current sequence s0 using a single sequence. Given the proposed distribution... New sequences are proposed by selecting mutation sites using a uniform distribution of mutated locations. Once a site is selected, several methods exist for choosing specific point mutations. These methods include:

[0074] a) Randomly select new amino acids evenly from the selected locations (from a set of 20 natural amino acids).

[0075] b) By using a first-order distribution induced by multiple sequence alignment of antibodies in the training set, new amino acids are selected at positions of interest, or

[0076] c) Select new amino acids at the sites of interest by sampling on the zero-order distribution of amino acids found in the antibody training set.

[0077] The first-order distribution induced by multiple sequence alignment is the empirical distribution of possible amino acids seen at a specific position in the training set. The zero-order distribution is the empirical distribution of all amino acids, independent of position. The zero-order distribution is analogous to extracting individual residues from each protein in the training set, placing the extracted residues into a bag, and sampling from the bag without replacement. The zero-order distribution does not preserve position, while the first-order distribution considers the distribution at every position in the protein. The process described above induces the distribution of possible sequences, as well as the conditional distribution given a starting sequence.

[0078] If the probability g(s,s) c If the threshold is exceeded, then s will be set to s. c (158). In one embodiment, the configurable threshold can be set by the design user.

[0079] In another embodiment, the configurable threshold can be set automatically by at least one factor. Whether automatic or manual, the following factors can be considered when setting the configurable threshold: the suggested acceptance rate and the mixing rate of the MCMC procedure. If the threshold is set too low, the acceptance rate will be high, but the mixing rate will be low, and convergence will be slow. If the threshold is set too high, the mixing rate will be high, but the acceptance rate will be low, so convergence will also be slow. Therefore, an intermediate value that balances the suggested acceptance rate and the mixing rate is ideal for maximizing the algorithm's performance. As described above, g(s,s) c ) represents a mapping from a pair of sequences (e.g., an antibody sequence and a next candidate sequence) to a unit interval that indicates the probability of accepting the candidate sequence as the next step in a random walk.

[0080] Then, the method determines based on Check if the characteristic value is less than The characteristic value (160). In other words, this method compares s b The affinity value of s is determined; however, those skilled in the art will recognize that other properties besides affinity can also be evaluated in this step. If s b If the evaluation characteristic is less than s, then s is set to s. b(For example, the sequence under consideration is evaluated as the best sequence to date) (162). The method then checks whether it has iterated a sufficient number of times or if there are other indicators indicating that the method has completed (164). If so, the method outputs s. b As output (166). If not, the method extracts another sequence s from the proposed distribution. c (156). If s b If the evaluation property is greater than or equal to s, then the method checks whether it has completed (164) and continues the operation as described above from there.

[0081] Figure 2 Figure 200 illustrates a random walk sequence in the antibody sequence space used for discovering antibody sequences using computational directed evolution. Antibody affinity is optimized using CDE along with a fine-tuning model. Seed antibody sequences s0 and 202 are one or more initial antibody sequences. The optimal sequence s0 from these sequences is selected. b Initialization is performed in sequence 204, where the optimal sequence is the one with the highest affinity and expression properties. Then, new candidate sequences 206 are drawn from the proposal distribution. Finally, the set of sequences s is set to s. c When fine-tuning the model for s b When the characteristic value result is less than s, s b Set it to 's'. Then, repeat the process until a suitable antibody is found.

[0082] The results of the method described in this paper provide improved antibody sequences. b The walk is iterated to generate as many sequences as possible required for the test. In various embodiments, the walk can be random, guided, or a combination of methods. The antigen affinity and expression of these sequences are then determined. The data can then be fed back into a fine-tuning process and can be repeated as needed until clinically relevant antibody sequences are generated. Using the method described herein, antibody sequences with improved luciferin antibody affinity, exceeding the affinity of the highest-affinity antibody seen in the dataset by an order of magnitude, can be generated. In this case, pre-training and fine-tuning used the dataset described above, but other datasets can also be used.

[0083] Figure 3Figure 300 illustrates the improvement of the generated antibody sequence 302 compared to existing datasets. This figure illustrates the fluorophore antibody sequence generated using de novo antibody affinity maturation. The de novo antibody affinity maturation process generated a fluorophore antibody sequence with sub-100 pM affinity. In Figure 300, the antibody sequences are plotted based on affinity (x-axis) and expression (y-axis). The upper right quadrant includes the generated antibody sequence 302, which possesses the highest values ​​for both properties. Those skilled in the art will recognize that these generated antibody sequences 302 (shown in red) possess the properties required for this process.

[0084] Figure 4 The illustration depicts a computer network or similar digital processing environment in which embodiments of the invention can be implemented.

[0085] Multiple client computers / devices 50 and multiple server computers 60 provide processing devices, storage devices, and input / output devices for executing applications, etc. The multiple client computers / devices 50 can also be linked to other computing devices, including other client devices / processes 50 and multiple server computers 60, via a communication network 70. The communication network 70 can be a remote access network, a global network (e.g., the Internet), a global computer collection, a local area network (LAN) or a wide area network (WAN), and can use the appropriate protocol (TCP / IP, etc.). It is part of a gateway that allows communication between devices (such as computers, etc.). This also applies to other electronic device / computer network architectures.

[0086] Figure 5 yes Figure 4 This diagram illustrates an example internal structure of a computer (e.g., client processor / device 50 or server computer 60) in a computer system. Each computer 50, 60 includes a system bus 79, where a bus is a set of hardware lines used to transfer data between components of a computer or processing system. The system bus 79 is essentially a shared channel connecting different components of the computer system (e.g., processor, disk storage, memory, input / output ports, network ports, etc.), enabling the transfer of information between components. Attached to the system bus 79 is an I / O device interface 82 for connecting various input and output devices (e.g., keyboard, mouse, monitor, printer, speakers, etc.) to the computer 50, 60. A network interface 86 allows the computer to connect to a network (e.g., ...). Figure 5Various other devices (network 70). Memory 90 provides volatile storage for computer software instructions 92 and data 94 used to implement embodiments of the present invention (e.g., the machine learning model module and fine-tuning machine learning model module code detailed above). Disk storage 95 provides non-volatile storage for computer software instructions 92 and data 94 used to implement embodiments of the present invention. Central processing unit 84 is also attached to system bus 79 and provides execution of computer instructions.

[0087] In one embodiment, processor routine 92 and data 94 are computer program products (generally referred to as 92) comprising a non-transitory computer-readable medium (e.g., one or more removable storage media such as DVD-ROM, CD-ROM, floppy disk, magnetic tape, etc.) providing at least a portion of the software instructions for the system of the present invention. Computer program product 92 can be installed by any suitable software installer as known in the art. In another embodiment, at least a portion of the software instructions can also be downloaded via wired communication and / or wireless connection. In other embodiments, the program of the present invention is a computer program propagation signal product embodied on a propagation medium (e.g., radio waves, infrared waves, laser waves, sound waves, or electromagnetic waves propagating over a global network (such as the Internet or other networks). Such a carrier medium or signal can be used to provide at least a portion of the software instructions for the routine / program 92 of the present invention.

[0088] All patents, published applications, and references cited herein are incorporated in their entirety by reference.

[0089] While exemplary embodiments have been specifically shown and described, those skilled in the art will understand that various changes in form and detail may be made therein without departing from the scope of the embodiments as covered by the appended claims.

Claims

1. A method for determining an antibody sequence with improved properties, the method comprising: A score is generated for each of a plurality of machine learning models, each machine learning model being trained based on a plurality of corresponding antibody sequences, each of the plurality of antibody sequences being labeled with a characteristic corresponding to the plurality of antibody sequences and a value of the characteristic corresponding to the antibody sequence, thereby obtaining a corresponding score for each of the plurality of machine learning models, the corresponding score indicating the contribution of the characteristic corresponding to the plurality of antibody sequences used to train the machine learning model; as well as Antibody sequences are generated using the multiple machine learning models by weighting the output of each machine learning model based on each generated score and combining the weighted outputs into a weighted sum. Each antibody sequence is the amino acid sequence of the antibody.

2. The method as described in claim 1, wherein, Generating the antibody sequence further includes: Based on the aforementioned multiple machine learning models, antibody sequences are selected from the proposed distribution; and Determine whether the selected antibody sequence has an acceptance probability exceeding a certain threshold. If so, analyze the antibody sequence; otherwise, select the next antibody sequence from the suggested distribution.

3. The method as described in claim 1, wherein, Generating the antibody sequence further includes: The first characteristic value of the antibody sequence selected from the suggested distribution, determined by a function of the plurality of machine learning models, is compared with the second characteristic value of the antibody sequence having the best characteristic value in the current search, determined by the plurality of machine learning models; and If the first characteristic value is greater than the second characteristic value, then the antibody sequence having the optimal characteristic value is replaced with an antibody sequence selected from the suggested distribution.

4. The method according to any one of claims 1 to 3, further comprising: The machine learning models in the plurality of machine learning models are trained based on a plurality of first antibody sequences, each antibody sequence in the plurality of first antibody sequences being labeled with a corresponding characteristic; as well as A fine-tuned machine learning model is generated by training one of the plurality of machine learning models based on a second plurality of antibody sequences, each antibody sequence in the plurality of antibody sequences being labeled with a corresponding characteristic, and the second plurality of antibody sequences being associated with an antigen of interest. The generation of the fine-tuned machine learning model further includes: Weight each sequence characteristic in the second plurality of antibody sequences; The optimal model parameters for generating the second plurality of antibody sequences are determined using the machine learning models among the plurality of machine learning models; and The optimal model parameters are applied to the machine learning model among the plurality of machine learning models to produce the resulting model, which is the fine-tuned machine learning model.

5. The method according to any one of claims 1 to 3, wherein, The properties corresponding to the respective plurality of antibody sequences are at least one of affinity, expression, protein aggregation, proteolytic stability, and off-target effects.

6. The method of claim 1, further comprising: Based on the multiple machine learning models, candidate antibody sequences are selected from the proposed distribution, wherein the candidate antibody sequences are within the defined acceptance criteria. as well as If the characteristics of the candidate antibody sequence are greater than those of the best-discovered antibody sequence, then the best-discovered antibody sequence is replaced with the candidate antibody sequence; otherwise, the candidate antibody sequence is ignored.

7. The method of any one of claims 1 to 3, wherein the method further comprises: To produce antibodies with the generated antibody sequence.

8. The method of any one of claims 1 to 3, wherein the method further comprises: Provide manufactured antibodies with the generated antibody sequence; as well as The properties of the manufactured antibodies are determined.

9. The method according to any one of claims 1 to 3, further comprising: One or more of the plurality of machine learning models are trained, each of the plurality of machine learning models being trained based on the corresponding plurality of antibody sequences, each of the corresponding plurality of antibody sequences being labeled with the characteristic corresponding to the corresponding plurality of antibody sequences and the value of the characteristic corresponding to the antibody sequence.

10. The method of claim 9, wherein, Training each of the one or more machine learning models further includes: Provide a set of amino acid sequences labeled with at least one characteristic; A portion of the amino acid sequence set is masked to provide a masked amino acid sequence set, wherein the remainder of the amino acid sequence set is an unmasked amino acid sequence set; and The one or more machine learning models are trained to estimate each amino acid sequence in the masked amino acid sequence set based on: (1) labeling at least one characteristic of each masked amino acid sequence, and (2) the unmasked amino acid sequence set and labeling at least one characteristic of each unmasked amino acid sequence.

11. The method according to any one of claims 1 to 3, wherein, The antibody sequence was generated using MCMC sampling.

12. The method according to any one of claims 1 to 3, wherein, The corresponding multiple antibody sequences are associated with the antigen of interest.

13. The method according to any one of claims 1 to 3, wherein, The contribution is used to predict at least one of the following characteristics to be improved: At least one characteristic is important for manufacturing the antibody sequence. Immunogenicity of antibodies in patients Antibody expression level, Developability Interaction with other models, Orthogonality with other models, and Empirical derivation obtained by adjusting the generation process.

14. A system for determining an antibody sequence having improved properties, the system comprising: processor; as well as The processor and the memory having computer code instructions stored thereon are configured to cause the system to: A score is generated for each of a plurality of machine learning models, each machine learning model being trained based on a plurality of corresponding antibody sequences, each of the plurality of antibody sequences being labeled with a characteristic corresponding to the plurality of antibody sequences and a value of the characteristic corresponding to the antibody sequence, thereby obtaining a corresponding score for each of the plurality of machine learning models, the corresponding score indicating the contribution of the characteristic corresponding to the plurality of antibody sequences used to train the machine learning model; as well as Antibody sequences are generated using the multiple machine learning models by weighting the output of each machine learning model based on each generated score and combining the weighted outputs into a weighted sum. Each antibody sequence is the amino acid sequence of the antibody.

15. The system of claim 14, wherein, To generate the antibody sequence, the system further comprises: Based on the aforementioned multiple machine learning models, antibody sequences are selected from the proposed distribution; as well as Determine whether the selected antibody sequence has an acceptance probability exceeding a certain threshold. If so, analyze the antibody sequence; otherwise, select the next antibody sequence from the suggested distribution.

16. The system as claimed in claim 14 or 15, wherein, To generate the antibody sequence, the system further comprises: The first characteristic value of the antibody sequence selected from the suggested distribution, determined by a function of the plurality of machine learning models, is compared with the second characteristic value of the antibody sequence that has the best characteristic value in the current search, determined by the plurality of machine learning models. and If the first characteristic value is greater than the second characteristic value, then the antibody sequence having the optimal characteristic value is replaced with an antibody sequence selected from the suggested distribution.

17. The system as claimed in claim 14 or 15, wherein, Further, the system: The machine learning models in the plurality of machine learning models are trained based on a plurality of first antibody sequences, each antibody sequence in the plurality of first antibody sequences being labeled with a corresponding characteristic; as well as A fine-tuned machine learning model is generated by training one of the plurality of machine learning models based on a second plurality of antibody sequences, each antibody sequence in the plurality of antibody sequences being labeled with a corresponding characteristic, and the second plurality of antibody sequences being associated with an antigen of interest. The generation of the fine-tuned machine learning model further includes: Weight each sequence characteristic in the second plurality of antibody sequences; The optimal model parameters for generating the second plurality of antibody sequences are determined using the machine learning models among the plurality of machine learning models; and The optimal model parameters are applied to the machine learning model among the plurality of machine learning models to produce the resulting model, which is the fine-tuned machine learning model.

18. The system as claimed in claim 14 or 15, wherein, The property corresponding to the respective plurality of antibody sequences is at least one of affinity and expression.

19. The system of claim 14, wherein, Further, the system: Based on the multiple machine learning models, candidate antibody sequences are selected from the proposed distribution, wherein the candidate antibody sequences are within the defined acceptance criteria. as well as If the characteristics of the candidate antibody sequence are greater than those of the best-discovered antibody sequence, then the best-discovered antibody sequence is replaced with the candidate antibody sequence; otherwise, the candidate antibody sequence is ignored.

20. The system as claimed in claim 14 or 15, wherein, Further, the system: One or more of the plurality of machine learning models are trained, each of the plurality of machine learning models being trained based on the corresponding plurality of antibody sequences, each of the corresponding plurality of antibody sequences being labeled with the characteristic corresponding to the corresponding plurality of antibody sequences and the value of the characteristic corresponding to the antibody sequence.

21. The system of claim 20, wherein, Training each of the one or more machine learning models includes: Provide a set of amino acid sequences labeled with at least one characteristic; A portion of the amino acid sequence set is masked to provide a masked amino acid sequence set, wherein the remainder of the amino acid sequence set is an unmasked amino acid sequence set; and The one or more machine learning models are trained to estimate each amino acid sequence in the masked set based on: (1) labeling at least one characteristic of each masked amino acid sequence and (2) the unmasked amino acid sequence set and labeling at least one characteristic of each unmasked amino acid sequence.

22. The system as claimed in claim 14 or 15, wherein, The antibody sequence was generated using MCMC sampling.

23. The system as claimed in claim 14 or 15, wherein, The corresponding multiple antibody sequences are associated with the antigen of interest.

24. The system as claimed in claim 14 or 15, wherein, The contribution is used to predict at least one of the following characteristics to be improved: At least one characteristic is important for manufacturing the antibody sequence. Antibody sequence expression, Immunogenicity of antibodies in patients Antibody expression level, Developability Interaction with other models, Orthogonality with other models, and Empirical derivation obtained by adjusting the generation process.

25. A method for antibody maturation, comprising: Provide a first antibody sequence to the system as described in any one of claims 14 to 24; as well as The generated antibody sequence is obtained from the system.

26. A method for antibody maturation, comprising: Provide a first antibody sequence to the method as described in any one of claims 1 to 13; as well as The generated antibody sequence is obtained from the method described above.

27. A method for determining an antibody sequence having improved properties, the method comprising: For each of the multiple fine-tuned machine learning models trained based on corresponding initial multiple antibody sequences, each antibody sequence in the corresponding initial multiple antibody sequences is labeled with a corresponding characteristic, and each fine-tuned machine learning model is further generated by training each machine learning model with a second multiple antibody sequence, each antibody sequence in the second multiple antibody sequence is labeled with a corresponding characteristic, and the second multiple antibody sequence is related to the antigen of interest; as well as Antibody sequences are generated based on an objective function using multiple fine-tuned machine learning models weighted by corresponding hyperparameters; Each antibody sequence is the amino acid sequence of the antibody.

28. A method for determining an antibody sequence having improved properties, the method comprising: A machine learning model is trained based on a plurality of antibody sequences, each of which is labeled with a corresponding characteristic. A fine-tuned machine learning model is generated by training the machine learning model based on the first plurality of antibody sequences based on the second plurality of antibody sequences, each of the second plurality of antibody sequences being labeled with a corresponding characteristic, and the second plurality of antibody sequences being associated with the antigen of interest. as well as Antibody sequences are generated based on the fine-tuned machine learning model. Each antibody sequence is the amino acid sequence of the antibody.

29. A method for determining an antibody sequence having improved properties, the method comprising: A score is provided for each of a plurality of machine learning models, each machine learning model being trained based on a plurality of corresponding antibody sequences, each of the plurality of antibody sequences being labeled with a characteristic corresponding to the plurality of antibody sequences and a value of the characteristic corresponding to the antibody sequence; the score provided for each of the plurality of machine learning models indicates the contribution of the characteristic corresponding to the plurality of antibody sequences used to train the machine learning model. as well as Antibody sequences are generated using the multiple machine learning models by weighting the output of each machine learning model according to each provided score and combining the weighted outputs into a weighted sum. Each antibody sequence is the amino acid sequence of the antibody.

30. A method for determining an antibody sequence with improved properties, the method comprising: Antibody sequences can be generated using multiple machine learning models in the following ways: The output of each of the plurality of machine learning models is weighted based on a score for each of the plurality of machine learning models, each machine learning model being trained on a plurality of corresponding antibody sequences, each antibody sequence being labeled with a characteristic corresponding to the corresponding antibody sequence and a value for the characteristic corresponding to the antibody sequence, and the score of each of the plurality of machine learning models indicating the contribution for predicting the characteristic corresponding to the corresponding antibody sequences used to train the machine learning model, and The weighted outputs are combined into a weighted sum; Each antibody sequence is the amino acid sequence of the antibody.

Images