Optimizing capsid monomer sequences using machine learning

Abstract

Methods, computer systems, and apparatus, including computer programs encoded on computer storage media, for optimizing capsids for a predefined property. The system maintains candidate monomer data characterizing a population of one or more candidate capsid monomers. At each of a plurality of iterations, the system updates (i) the candidate monomer data and (ii) current values of model parameters of a first machine-learning model. After performing the plurality of iterations, the system selects one or more target capsid monomers from the population of candidate capsid monomers, and outputs data specifying the selected target capsid monomers.

Description

[0001] OPTIMIZING CAPSID MONOMER SEQUENCES USING MACHINE LEARNING

[0002] CLAIM OF PRIORITY

[0003] This application claims priority to U.S. Provisional Application No. 63 / 595,610, filed on November 2, 2023, and European Application No. 23315432.7, filed on November 21, 2023, the contents of which are hereby incorporated by reference.

[0004] BACKGROUND

[0005] This specification relates to optimizing viral capsid monomer sequences using machine learning.

[0006] A virus can refer to a submicroscopic infectious agent that can replicate inside living cells of an organism, and a capsid can refer to a protein shell of a virus that encloses the genetic material of the virus.

[0007] A capsid can include an assembly of repeating protein structural units referred to as “capsid monomers” or simply “monomers.” A capsid can include any appropriate number of monomers, e.g., 20 monomers, 50 monomers, 100 monomers, etc. A capsid monomer can include a sequence (chain) of amino acids, where each amino acid is selected from a set of possible amino acids, e.g.. the standard set of 20 alpha amino acids, e.g.. glycine, alanine, valine, etc. The amino acid sequence of a capsid monomer can have any appropriate length (i.e., where the “length” of amino acid sequence refers to the number of positions, occupied by amino acids, in the amino acid sequence), e.g., a length of 10, 20, 30, etc.

[0008] An “adeno-associated virus” (AAV) is a small virus that infects humans but that causes a mild immune response and is not currently known to cause disease. AAVs have properties making them appropriate candidates for creating viral vectors for gene therapy, e.g., their apparent non-pathogenicity, their ability to infect non-dividing cells, and their capacity to stably integrate into host cell genomes at specific sites. AAV capsids can include a mixture of capsid monomers VP1, VP2, and VP3, totaling 60 monomers, which jointly assemble to form a capsid with icosahedral symmetry. A machine-learning model is a computational model that leams patterns and relationships in data, and then uses that knowledge to make predictions or decisions on new data. Neural networks are machine-learning models that employ one or more layers of nonlinear units to predict an output for a received input. The output of each hidden layer is used as input to the next layer in the network, i.e., the next hidden layer or the output layer. Each layer of the network generates an output from a received input in accordance with current values of a respective set of parameters.

[0009] SUMMARY

[0010] This disclosure describes methods, computer systems, and apparatus, including computer programs encoded on computer storage media, for optimizing capsids for a predefined property. The predefined property can refer to any appropriate property of the capsid, e.g., the manufacturability of the capsid.

[0011] In this specification, the "manufacturability" of a capsid characterizes how efficiently the capsid can be manufactured. In particular, the manufacturability of a capsid can characterize a rate at which the capsid can be manufactured when plasmids coding the capsid are transfected into one or more producer cells. More specifically, the manufacturability of a capsid can be based at least in part on a number of instances of the capsid that are generated when plasmids coding for the capsid are transfected into one or more cells. The manufacturability of a capsid indirectly characterizes various structural and biochemical properties of the capsid that affect its manufacture, e.g., how readily the capsid monomers of the capsid can assemble into the three-dimensional structure of the capsid, the stability of the capsid, etc.

[0012] In this specification, an “embedding” can refer to an ordered collection of numerical values, e.g., a vector, matrix, or other tensor of numerical values.

[0013] In one aspect, this disclosure provides a method for optimizing capsid monomer sequences. The method can be implemented by a system including one or more computers. The system maintains candidate monomer data characterizing a population of one or more candidate capsid monomers. The candidate monomer data includes, for each candidate capsid monomer in the population, (i) data defining a respective amino acid sequence of the respective candidate capsid monomer and (ii) data specifying a respective fitness score that characterizes a predefined property of a capsid including the respective candidate capsid monomer. The system initializes the population of candidate capsid monomers using (i) one or more initial capsid monomer sequences and (ii) a corresponding fitness score for each respective initial capsid monomer sequence. The system initializes current values of model parameters of a first machine-learning model. At each of a plurality of iterations, the system updates (i) the candidate monomer data and (ii) the current values of the model parameters of the first machine-learning model. To update (i) the candidate monomer data and (ii) the current values of the model parameters of the first machine-learning model, the system generates, using the first machine-learning model according to the current values of the model parameters and based on the respective amino acid sequences of one or more of the candidate capsid monomers in the population, data representing a set of query capsid monomers, uses a second machine-learning model to process respective data representing each of the query capsid monomers to generate a respective output specifying a respective benchmark fitness score for the respective query capsid monomer, updates the current values of the model parameters of the first machine-learning model using the benchmark fitness scores for the query capsid monomers, adds one or more of the query capsid monomers into the population of candidate capsid monomers, and updates the candidate monomer data by including, for each of the set of query7capsid monomers, (i) data defining the respective amino acid sequence of the respective query capsid monomer and (ii) data specifying the respective benchmark fitness score of the respective query capsid monomer. After performing the plurality of iterations, the system selects one or more target capsid monomers from the population of candidate capsid monomers, and outputs data specifying the selected target capsid monomers.

[0014] In some implementations, to select one or more target capsid monomers, the system selects a target capsid monomer having a highest fitness score from the population of candidate capsid monomers.

[0015] In some implementations, the predefined property' characterizes a manufacturability of capsids. For example, the manufacturability of the capsid can be defined based on a number of instances of the capsid that are generated when a plasmid coding for the capsid is transfected into one or more cells. The number of instances of the capsid that are generated when a plasmid coding the capsid is transfected into one or more cells can be normalized by a number of generated wild-type sequences. In some implementations, the predefined property characterizes a capacity of viruses that include the capsid to avoid neutralization.

[0016] In some implementations, the predefined property characterizes an immunoreactivity of the capsid.

[0017] In some implementations, the predefined property of the capsid characterizes a predicted capacity of viruses that include the capsid to penetrate a target tissue.

[0018] In some implementations, the predicted fitness score characterizes a packing capacity of the capsid.

[0019] In some implementations, the predicted fitness score characterizes a predicted capacity of viruses that include the capsid to integrate into host genomes.

[0020] In some implementations, the capsid corresponds to a capsid of an adeno-associated virus (AAV).

[0021] In some implementations, the system further physically generates one or more viruses that include a protein having one of the selected target capsid monomer. The system can further apply the generated viruses to a subject to achieve a therapeutic effect in the subject.

[0022] In some implementations, the second machine-learning model includes: an embedding neural network configured to process an input defining the amino acid sequence of the respective query' capsid monomer to generate a respective embedding of the amino acid sequence, and a fitness prediction machine-learning model configured to process the respective embedding of the amino acid sequence to generate an output characterizing a respective predicted fitness score for the respective query capsid monomer. The second machine-learning model can further include an error prediction machine-learning model configured to process an input representing the amino acid sequence of the respective query capsid monomer to generate a respective fitness error, wherein the respective fitness error is an estimate of an error in the respective predicted fitness score generated by the fitness prediction machine-learning model. The respective benchmark fitness score can be computed by combining: (i) the respective predicted fitness score generated by the fitness prediction machine-learning model, and (ii) the respective fitness error generated by the error prediction machine-learning model. In some implementations, the fitness prediction machine-learning model has a different model architecture than the error prediction machine-learning model.

[0023] In some implementations, the embedding neural network has been trained using selfsupervised learning on a set of training capsid monomers. For example, the self-supervised learning can include training the embedding neural network on the set of training capsid monomers to perform a token unmasking task. In another example, the self-supervised learning can include training the embedding neural network on the set of training capsid monomers to perform a next token prediction task.

[0024] In some implementations, the fitness prediction machine-learning model has been trained on a set of training examples, wherein each training example corresponds to a respective training capsid monomer and includes: (i) an embedding of the training capsid monomer, and (ii) a target fitness score for the training capsid monomer.

[0025] In some implementations, generating the data representing the set of query capsid monomers includes: generating data representing a set of mutant capsid monomers that are mutated from one or more of the candidate capsid monomers; initializing the set of query capsid monomers by selecting from the set of mutant capsid; and updating the set of query capsid monomers using the first machine-learning model.

[0026] In some implementations, the first machine-learning model is configured to process a first input characterizing an input capsid monomer to predict an adjusted fitness score of the input capsid monomer.

[0027] In some implementations, the first input further characterizes one or more mutated mutations of the input capsid monomer from a corresponding wild-type capsid monomer that the input capsid monomer is mutated from. In some implementations, the adjusted fitness score measures a combination of (i) a predicted fitness that characterizes the predefined property7of input capsid monomer, and (ii) a mutation distance between the input capsid monomer and the corresponding wild-type capsid monomer.

[0028] In some implementations, the first machine-learning model includes: an encoder network configured to process data specifying a respective input capsid monomer fragment that neighbors each mutation of the input capsid monomer from the corresponding wild-type capsid monomer to generate a respective feature vector, a pooling network layer configured to combine the respective feature vectors generated from the input capsid monomer fragments into a combined feature vector, and a decoder network configured to process the combined feature vector to generate the predictive score of the input capsid monomer.

[0029] In some implementations, updating the set of query capsid monomers using the first machine-learning model includes: at each of a plurality of iterations, performing: predicting, using the first machine-learning model, the adjusted fitness scores of a subset of query capsid monomers, selecting, based on the predicted adjusted fitness scores, one or more seed capsid monomers from the subset of query' capsid monomers, generating data representing an additional set of mutant capsid monomers that are mutated from the selected seed capsid monomers, and updating the set of query capsid monomers by including one or more of the additional set of mutant capsid monomers.

[0030] This disclosure also provides a system including one or more computers and one or more storage devices storing instructions that, when executed by the one or more computers, cause the one or more computers to perform the optimizing method described above.

[0031] This disclosure also provides one or more computer storage media storing instructions that when executed by one or more computers, cause the one or more computers to perform the optimizing method described above.

[0032] The subject matter described in this disclosure can be implemented in particular embodiments so as to realize one or more advantages.

[0033] Capsid monomer engineering is widely used in the field of virology7and gene therapy to modify the structure and function of viral capsid proteins. The capsid is the outer protein shell of a virus. In the context of gene therapy, viral capsids, such as those derived from adeno-associated viruses (AAVs), are often used as vectors to deliver therapeutic genes into target cells.

[0034] Gene therapy has the potential to transform healthcare, e.g., by enabling diseases (such as hemophilia, leukemia, melanoma, etc.) to be addressed with single-dose treatments of therapeutic viral vectors (e.g., AAV vectors). However, gene therapy treatments are often expensive, in some cases costing $1 million or more to treat a single patient. High costs are a major barrier to the full realization of the potentially transformative effects of gene therapy. A major contributor to the cost of gene therapy is the cost of manufacturing capsids for viral vectors.

[0035] To optimize a property (e.g., the manufacturability) of a capsid, one approach is to perform high-throughput screening of a large number of synthesized candidate capsid mutants to identify candidate capsid mutants with desirable properties. However, this approach can pose significant challenges. For example, the AAV VP1 capsid monomer sequence consists of 735 amino acids. Screening all possible mutants across the whole 735 lengths is not feasible. For example, even considering only substitution mutations, one would have to search through 20735possible sequences. When insertions and deletions are included, the search space becomes significantly greater. Even if one were to focus on a small region of 14-aa length, the number of possible sequences still reaches 2014i.e. ~ 1018. Practical cloning restrictions limit the number of unique candidates that can be produced at the plasmid level to <108, which is further reduced to a fraction of that (<107) as the viral library is packaged. Consequently, more than 99.999999999% of potential candidates are lost.

[0036] Machine learning methods can be applied to assist the engineering of virus capsid mutants. For example, instead of using high-throughput screening, prediction models trained on large-scale AAV mutants can be used to predict the fitness score that characterizes a predefined property of a novel sequence of an AAV mutant. However, such an approach is also associated with significant challenges.

[0037] First, the accuracy and robustness of a machine-learning model can be limited by the amount and quality of training data that is available for training the machine-learning model. In many cases, generating capsid fitness training data for training the fitness prediction system involves running laboratory7experiments, e.g., by measuring the number of instances of capsids that are generated by transfecting cells with plasmids coding for the capsids. These laboratory experiments can be expensive and time consuming, and can generate “noisy” (e.g., inaccurate or inconsistent) data.

[0038] Moreover, the qualify (e.g., manufacturability) of a capsid can result from complex interactions of biology, chemistry, and physics, and thus in order to predict capsid qualify, a machine-learning model must perform sophisticated implicit reasoning. The reasoning capacity of a machine-learning model can depend on the number of parameters of the model, e.g., such that increasing the number of parameters of the machine-learning model can increase its reasoning capacity. Certain machine-learning models that perform complex prediction tasks can include large numbers of parameters, e.g.. millions or billions of parameters. However, increasing the number of parameters of a machine-learning model can increase the likelihood of the machine-learning model "overfilling" during training, in particular, by capturing some of the residual variation (noise) in the training data as if that variation represented underlying structure in the training data. The risk of overfitting can be exacerbated when the training data is limited in quantity and subject to noise, as is the case with capsid fitness training data, as described above.

[0039] Furthermore, as described above, the sequence space of possible viral capsid mutants is vast (e.g., 20735possible sequences for AAV VP1 monomer). Attempting an exhaustive sampling of this space would demand impractical amounts of computation resources, even with the aid of computationally efficient prediction models.

[0040] The capsid optimizing system described in this specification implements various innovations to address these issues.

[0041] The system implements a high-quality discriminative machine-learning model to predict, for each candidate capsid monomer, a respective fitness of the capsid including the candidate capsid. In some implementations, the discriminative machine-learning model includes (i) an embedding neural netw ork configured to generate embeddings of input amino acid sequences and (ii) a fitness prediction model configured to process the embeddings to predict fitness scores for the sequences. The embedding neural network can be pre-trained based on large quantities of protein data for which labels (such as capsid fitness scores) are unavailable using self-supervised learning techniques for generating effective embeddings for capsid monomer sequences. Training the embedding neural network to perform selfsupervised tasks causes the embedding neural network to leam to generate capsid monomer embeddings that compactly encode biological / chemical / mechanical properties of capsid monomers which can be leveraged by the fitness prediction model to generate accurate fitness scores. Moreover, training the embedding neural network on self-supervised tasks does not rely on capsid fitness training data, and therefore is not affected by the limited quantity and quality of the available capsid fitness training data.

[0042] Furthermore, instead of using an exhaustive sampling approach for exploring the sequence space, the capsid optimizing system uses model-guided search, where a generative or inferential machine-learning model is used to generate or guide generating candidate capsid monomer sequences in each of multiple iterations. The discriminative machinelearning model is used in each iteration to evaluate the generated candidate capsid monomer sequences, and the evaluation data is used to train the generative or inferential machinelearning model in each iteration. In particular, in some implementations, the system can use generative or inferential machine learning to perform proximal exploration of the sequence space, where both the inferred fitness of a candidate capsid monomer sequence and a mutation distance between the candidate capsid monomer and the corresponding wild-type capsid monomer are taken into consideration. By using proximal exploration, the system can effectively explore the sequence space with a reduced number of queries for identifying high- quality capsid monomer sequences.

[0043] The details of one or more embodiments of the subject matter described in this disclosure are set forth in the accompanying drawings and the description below. Other features, aspects, and advantages of the subject matter will become apparent from the description, the drawings, and the claims.

[0044] BRIEF DESCRIPTION OF THE DRAWINGS

[0045] FIG. 1 shows an example environment for optimizing capsid monomer sequences.

[0046] FIG. 2 shows an example capsid optimization system.

[0047] FIG. 3 shows an example discriminative model for benchmarking capsid fitness.

[0048] FIG. 4 is a flow diagram of an example process for optimizing capsids for a predefined property.

[0049] FIG. 5 is a flow diagram of an example process for generating query capsid monomer sequences.

[0050] FIG. 6 illustrates the capsid quality achieved by the capsid optimization system described in this specification compared to capsid quality achieved by randomly mutating candidate capsid monomer sequences.

[0051] FIG. 7 is a block diagram of an example computer system. Like reference numbers and designations in the various drawings indicate like elements.

[0052] DETAILED DESCRIPTION

[0053] FIG. 1 shows an example environment 100 for designing a target capsid monomer that is optimized for a predefined property using a capsid optimization system 200.

[0054] The initial capsid monomer data 102 defines a set of initial capsid monomers, where each capsid monomer is represented by a respective sequence of amino acids. The initial capsid monomer data 102 further includes data specifying respective fitness scores for the set of initial capsid monomers. A fitness score for a capsid monomer can characterize a predefined property of the corresponding capsid.

[0055] The initial capsid monomer data 102 can be generated in any of a variety of possible ways. For instance, some or all of the initial capsid monomers defined by the initial capsid monomer data 102 can be mutations of an ‘'original’' capsid monomer, e.g., a VP1, VP2, or VP3 capsid monomer of an AAV. More specifically, each initial capsid monomer can be generated by modifying the identity of a respective amino acid at one or more positions in the amino acid sequence of the original capsid monomer. Positions in the amino acid sequence of the original capsid monomer can be selected for mutation in any appropriate way, e.g., through random selection or selection in accordance with a predefined rule. The identities of new amino acids substituted into positions in the amino acid sequence of the original capsid monomer can be selected in any appropriate way, e.g., randomly selected from a probability distribution over a set of possible amino acids. The respective amino acid sequence of each capsid monomer in the initial capsid monomer data can differ from the amino acid sequence of the original capsid monomer in any appropriate number of positions, e.g., 1 position, 3 positions, or 10 positions. The fitness scores of the initial capsid monomers can be experimentally measured or predicted by a prediction system. In some implementations, the initial capsid monomers can be selected from capsid monomers based on measured or predicted fitness scores of the capsid monomers.

[0056] The capsid optimization system 200 is configured to generate target capsid monomer sequences 106 that are optimized for the predefined property. In some implementations, the system 200 generates target capsid monomer sequences 106 that are optimized for the manufacturability of the corresponding capsid. The manufacturability of a capsid characterizes a rate at which the capsid can be generated when plasmids coding the capsid are transfected into one or more cells. (More specifically, the manufacturability' of a capsid can be based at least in part on a number of instances of the capsid that are generated when plasmids coding for the capsid are transfected into one or more cells). That is, the manufacturability of a capsid characterizes how efficiently the capsid can be generated. The manufacturability of a capsid indirectly characterizes various structural and biochemical properties of the capsid that affect its manufacture, e.g., how readily the capsid monomers of the capsid can assemble into the three-dimensional structure of the capsid, the stability’ of the capsid, etc.

[0057] In some implementations, the system 200 can generate target capsid monomer sequences 106 that are optimized for a capacity' of a virus that includes the corresponding capsid to avoid neutralization in an organism. Virus neutralization refers to a process by which the capacity of a virus to infect a cell is reduced, e.g., through the binding of antibodies to epitopes on the surface of the virus.

[0058] In some implementations, the system 200 can generate target capsid monomer sequences 106 that are optimized for an immunoreactivity of the corresponding capsid. The immunoreactivity of a capsid refers to a measure of immune reaction caused by the capsid.

[0059] In some implementations, the system 200 can generate target capsid monomer sequences 106 that are optimized for the capacity of viruses that include the capsid to penetrate a target tissue in an organism. The target tissue can be, e.g., tissue corresponding to a particular organ, e.g., liver tissue, brain tissue, eye tissue, etc.

[0060] In some implementations, the system 200 can generate target capsid monomer sequences 106 that are optimized for a packing capacity of the corresponding capsid. The packing capacity of a capsid refers to a measure of the amount of genetic material (e.g., deoxyribonucleic acid (DNA) or ribonucleic acid (RNA)) that can be contained by the capsid.

[0061] In some implementations, the system 200 can generate target capsid monomer sequences 106 that are optimized for a capacity of viruses that include the corresponding capsid to integrate into host genomes in a site-specific manner. For each target capsid monomer 106. viruses that include the capsid corresponding to the target capsid monomer 106 can then be manufactured 108, i.e., physically generated, using appropriate manufacturing techniques. In some implementations, the capsids are expressed in a cell. For example, transfer plasmids carry ing the gene of interest (GOI) can be co-transfected with Rep-cap plasmids (including the sequences for capsids) and helper plasmids encoding adenovirus genes (e.g., E4, E2A and VA) that mediate AAV replication into packaging cells (e.g., HEK293T). After a short incubation period, viral particles can be harvested from cell lysate or supernatant and concentrated e.g., by PEG precipitation. Viral particles are further purified and concentrated e.g., by cesium chloride (CsCl) gradient ultracentrifugation. The generated viruses can be used in any of a variety' of applications, e.g., the generated viruses can be applied as a therapeutic 110 to a subject 112 to achieve a therapeutic effect in the subject. For instance, the generated viruses can be used as viral vectors to implement gene therapies, as described above. The generated viruses can be applied to any appropriate subject, e.g., a mouse, a cat, a dog, a pig, or a human, to achieve any appropriate therapeutic effect, e.g., treatment of a disease, e.g., hemophilia, leukemia, melanoma, etc.

[0062] FIG. 2 shows operations of the capsid optimization system 200. The system 200 is an example of a system implemented as computer programs on one or more computers in one or more locations, in which the systems, components, and techniques described below can be implemented.

[0063] The system 200 maintains candidate monomer data 210 characterizing a population of one or more candidate capsid monomers. In particular, the candidate monomer data 210 includes, for each candidate capsid monomer in the population, (i) data defining a respective amino acid sequence of the respective candidate capsid monomer and (ii) data specifying a respective fitness score that characterizes a predefined property of a corresponding capsid including the respective candidate capsid monomer.

[0064] The system 200 initializes the population of candidate capsid monomers using the initial capsid monomer data 102 which defines a set of initial capsid monomers and specifies the respective fitness scores for the set of initial capsid monomers.

[0065] The system 200 uses a first machine-learning model 220 to generate data representing a set of query capsid monomers 230. The system 200 uses a second machine-learning model 250 to process data representing each of the query capsid monomers to generate a respective output specifying the respective benchmark fitness score 260 for the respective query capsid monomer. The training engine 270 is configured to update the current values of the model parameters of the first machine-learning model using the benchmark fitness scores for the query capsid monomers. The system 200 can update the candidate capsid monomer data 210 by adding one or more of the query capsid monomers 230 into the population of candidate capsid monomers. The above process can be repeated for multiple iterations until a stop condition is met, for example, until a predefined number of iterations have been performed, or until one or more candidate capsid monomers have been identified to have fitness scores that reach a threshold fitness score. In some implementations, the system 200 can perform the explorations using two different manners: (1) a broader search, and (2) a deeper search. For the broader search, the system 200 can perform an extensive mutational exploration in an outer search loop from the initial sequence. For the deeper search, the system 200 can perform a higher number of mutation iterations in an inner search loop, but starting from a more limited set of seed sequences. This dual approach enhances the system’s adaptability, enabling it to either diversify its search range or focus deeper into specific sequence optimizations.

[0066] After performing the multiple iterations, the system 200 can select one or more target capsid monomers (e.g.. based on the respective fitness scores) from the population of candidate capsid monomers. The system 200 can output data specifying the selected capsid monomers.

[0067] In some implementations, the first machine-learning model 220 can be a generative model, such as a variational autoencoder (VAE), configured to generate the query capsid monomer sequences from the candidate capsid monomer sequences.

[0068] In some other implementations, the first machine-learning model 220 can be an inference model configured to infer a property of an input capsid monomer sequence. The system 200 can use the inferred capsid monomer properties to guide the generation of the query capsid monomer sequences. In particular, the inference model 220 can infer an adjusted fitness score of the input capsid monomer. In a particular example, the adjusted fitness score measures a combination of (i) a predicted fitness that characterizes the predefined property of input capsid monomer, and (ii) a mutation distance between the input capsid monomer and a corresponding wild-ty pe capsid monomer from which the input capsid monomer is mutated from.

[0069] The inference model 220 can adopt any appropriate architecture. In a particular model, the inference model 220 includes (i) an encoder network configured to process data specifying a respective input capsid monomer fragment that neighbors each mutation of the input capsid monomer from the corresponding wild-type capsid monomer to generate a respective feature vector, (ii) a pooling network layer configured to combine the respective feature vectors generated from the input capsid monomer fragments into a combined feature vector, and (iii) a decoder network configured to process the combined feature vector to generate the predictive score of the input capsid monomer. Examples of the inference model 220 can be found in Ren, et al., '‘Proximal Exploration for Model-guided Protein Sequence Design,” Proceedings of the 39th International Conference on Machine Learning, in Proceedings of Machine Learning Research 162: 18520-18536 , 2022, and Sinai, et al., “Generative AAV capsid diversification by latent interpolation,” bioRxiv, doi: 10.1101 / 2021.04.16.440236, April, 2021.

[0070] An example process for generating the query' capsid monomer sequences will be described with reference to FIG. 5.

[0071] FIG. 3 shows an example of the second machine-learning model 250, which is a fitness prediction model. The fitness prediction model 250 is configured to process data defining an amino acid sequence of a query capsid monomer 230 to generate a benchmark fitness score 260 for the query capsid monomer 230. The benchmark fitness score 260 characterizes a predicted qualify of the capsid corresponding to the query capsid monomer 230. For instance, the benchmark fitness score 260 can characterize a predicted manufacturability' of the capsid, a predicted capacity of viruses that include the capsid to avoid neutralization, a predicted immunoreactivify of the capsid, a predicted capacity of viruses that include the capsid to penetrate target tissues, a predicted packing capacity of the capsid, or a predicted capacity' of viruses that include the capsid to integrate into host genomes, as described above with reference to FIG. 1.

[0072] The fitness prediction model 250 includes an embedding neural network 251, a fitness prediction machine-learning model 253, and an error prediction machine-learning model 254, which are each described in more detail next. The embedding neural network 251 is configured to process data defining the amino acid sequence of the query capsid monomer 230 to generate an embedding 252 of the query capsid monomer 230.

[0073] The embedding neural network 251 can be configured to process any appropriate representation of the amino acid sequence of the query capsid monomer 230. For instance, the embedding neural network 251 can process a representation of the amino acid sequence of the query capsid monomer 230 as a sequence of amino acid embeddings. The amino acid embedding at each position in the sequence of amino acid embeddings can represent the identity of the amino at the corresponding position in the amino acid sequence of the capsid monomer. An amino acid embedding representing an amino acid can be, e.g., a one-hot embedding, or any other appropriate predefined embedding.

[0074] The embedding neural network 251 can have any appropriate neural network architecture that enables the embedding neural network 251 to perform its described functions. In particular, the embedding neural network 251 can include any appropriate types of neural network layers (e.g., fully connected layers, convolutional layers, recurrent layers, attention layers, etc.) in any appropriate number (e.g., 5 layers, 10 layers, 50 layers, etc.) and connected in any appropriate configuration (e.g., as a linear sequence of layers).

[0075] In some implementations, the embedding neural network 251 can be a recurrent neural network that is configured to sequentially process each amino acid embedding in a sequence of amino acid embeddings representing the amino acid sequence of the query’ capsid monomer 230.

[0076] A system can train the embedding neural network 251 on a set of training examples, where each training example corresponds to a respective protein and defines an amino acid sequence of the protein. In particular, the embedding neural network 251 can be trained, on the set of training examples, using self-supervised learning techniques, i.e., training techniques that do not rely on the training examples being labeled with fitness scores. For example, the self-supervised learning can include (i) training the embedding neural network 251 on the set of training capsid monomers to perform a token unmasking task, and / or (ii) training the embedding neural network on the set of training capsid monomers to perform a next token prediction task. Training the embedding neural network 251 using self-supervised learning techniques, e.g., to perform tasks such as token unmasking, next token prediction, or both, can cause the embedding neural network 251 to learn to generate embeddings densely encode rich information characterizing proteins, e.g., capsid monomers.

[0077] The fitness prediction machine-learning model 253 is configured to process the embedding 252 of the query capsid monomer 230 to generate the benchmark fitness score 260 characterizing a predicted quality of the capsid corresponding to the query capsid monomer 230.

[0078] The fitness prediction machine-learning model 253 can have any appropriate machine-learning model architectures that enable the fitness prediction machine-learning model 253 to perform its described functions. For instance, the fitness prediction machinelearning model can be a parametric machine-learning model, such as a neural network model, a random forest model, or a support vector regression model. As another example, the fitness prediction machine-learning model can be a non-parametric model, such as a k-nearest neighbor model.

[0079] A system can train the fitness prediction machine-learning model 253 on a set of training examples using supervised learning. Each training example corresponds to a respective capsid monomer and defines (i) an embedding of the amino acid sequence of the capsid monomer, generated using the embedding neural network; and (ii) a target fitness score associated with the capsid monomer. The target fitness score for a capsid monomer represents the prediction target for the fitness prediction system, i.e., the "ground truth” fitness that should be generated by the fitness prediction system for the capsid monomer.

[0080] The fitness prediction machine-learning model 253 can be trained using target fitness scores that are generated in any appropriate manner, such as next-generation sequencing reads from a physical experiment. For example, the fitness score for a capsid monomer can describe the manufacturability of the given capsid. To generate a target fitness score, plasmids corresponding to the given capsid can be transfected into one or more cells. Experimental measurements from a next-generation sequencing assay can determine the number of instances that a given capsid or wild-type capsid can be generated from one or more cells. The fitness score can be defined as the ratio of the generated instances of a given capsid to the wild-type capsid. The error prediction machine-learning model 254 is configured to process a representation of the amino acid sequence of the query capsid monomer 230 to generate a fitness error 255. The fitness error 255 is an estimate of an error in the fitness score 256 generated by the fitness prediction machine-learning model 253.

[0081] The fitness prediction model 250 updates the fitness score 256 generated by the fitness prediction machine-learning model 208 by combining: (i) the fitness score 256, and (ii) the fitness error 255. The fitness prediction model 250 can combine the fitness score 256 and the fitness error 255, e.g., by summing the fitness score 256 and the fitness error 255. The fitness prediction model 250 can then output the updated fitness score 256 as the benchmark fitness score for the query capsid monomer 230.

[0082] FIG. 4 is a flow diagram illustrating an example process 400 for optimizing capsids for a predefined property. For convenience, the process 400 will be described as being performed by a system of one or more computers located in one or more locations. For example, a capsid optimization system, e.g., the capsid optimization system 200 of FIG. 2, appropriately programmed in accordance with this disclosure, can perform the process 400.

[0083] The system maintains candidate monomer data characterizing a population of one or more candidate capsid monomers. The candidate monomer data includes, for each candidate capsid monomer in the population, (i) data defining a respective amino acid sequence of the respective candidate capsid monomer and (ii) data specifying a respective fitness score that characterizes a predefined property of a capsid comprising the respective candidate capsid monomer.

[0084] At 410, the system initializes the population of candidate capsid monomers using (i) one or more initial capsid monomer sequences and (ii) a corresponding fitness score for each respective initial capsid monomer sequence. At 420, the system initializes the current values of model parameters of a first machine-learning model.

[0085] The system performs operations 430-460 for multiple iterations to update (i) the candidate monomer data and (ii) the current values of the model parameters of the first machine-learning model.

[0086] At 430, the system generates, using the first machine-learning model according to the current values of the model parameters and based on the respective amino acid sequences of one or more of the candidate capsid monomers in the population, data representing a set of query capsid monomers.

[0087] At 440, the system uses a second machine-learning model to process respective data representing each of the query capsid monomers to generate a respective output specifying a respective benchmark fitness score for the respective query capsid monomer.

[0088] At 450, the system updates the current values of the model parameters of the first machine-learning model using the benchmark fitness scores for the query capsid monomers.

[0089] At 460, the system adds one or more of the query capsid monomers into the population of candidate capsid monomers, and updates the candidate monomer data by including, for each of the set of query capsid monomers, (i) data defining the respective amino acid sequence of the respective query capsid monomer and (ii) data specifying the respective benchmark fitness score of the respective query capsid monomer.

[0090] After performing the multiple iterations, at 470, the system selects one or more target capsid monomers from the population of candidate capsid monomers.

[0091] At 480, the system outputs data specifying the selected target capsid monomers.

[0092] FIG. 5 is a flow diagram illustrating an example of the process 430 for generating query capsid monomer sequences.

[0093] At 431, the system generates data representing a set of mutant capsid monomers that are mutated from one or more of the candidate capsid monomers.

[0094] At 432, the system initializes the set of query capsid monomers by selecting from the set of mutant capsids. The system can use any appropriate strategy7to select mutant capsids, e.g., to select the mutant capsids with properties satisfying a predefined condition.

[0095] The system updates the set of query7capsid monomers using the first machinelearning model, for example, by performing operations 433-436 for multiple iterations.

[0096] At 433, the system predicts, using the first machine-learning model, the adjusted fitness scores of a subset of query7capsid monomers. The adjusted fitness score of a query capsid can measure a combination of (i) a predicted fitness that characterizes the predefined property of the input capsid monomer, and (ii) a mutation distance between the input capsid monomer and the corresponding wild-type capsid monomer.

[0097] At 434, the system selects, based on the predicted adjusted fitness scores, one or more seed capsid monomers from the subset of query capsid monomers. For example, the system can select the query capsid monomers that have the highest adjusted fitness score as the seed monomers.

[0098] At 435, the system generates data representing an additional set of mutant capsid monomers that are mutated from the selected seed capsid monomers.

[0099] At 436, the system updates the set of query capsid monomers by including one or more of the mutant capsid monomers from the additional set.

[0100] The processes 433-436 can be repeated for multiple iterations until a stop condition is met, for example, until a predefined number of iterations have been performed, or until a predefined number of query7capsid monomer sequences have been generated. The multiple iterations of 433-436 represent a proximal exploration process, where the query capsid monomers are explored in the direction of higher adjusted fitness scores for low-order mutants of the capsid monomer sequences. The adjusted fitness score takes into account both the fitness and the mutation distance of a query7capsid monomer, which has been show n to be an effective and efficient strategy to explore the capsid monomer space.

[0101] FIG. 6 provides an illustration of the capsid quality achieved by the capsid optimization system 200 compared to the capsid quality achieved by randomly mutating candidate capsid monomer sequences. In particular, FIG. 6 compares the fitness scores for capsid manufacturability of the randomly generated capsid monomers and the capsid monomers optimized using the system 200, and demonstrates that the system 200 improves the optimized capsid monomer quality7with statistical significance.

[0102] FIG. 7 is a block diagram of an example computer system 700 that can be used to perform operations described above. The system 700 includes a processor 710, a memory 720, a storage device 730, and an input / output device 740. Each of the components 710, 720, 730, and 740 can be interconnected, for example, using a system bus 750. The processor 710 is capable of processing instructions for execution within the system 700. In one implementation, the processor 710 is a single-threaded processor. In another implementation, the processor 710 is a multi-threaded processor. The processor 710 is capable of processing instructions stored in the memory 720 or on the storage device 730.

[0103] The memory 720 stores information within the system 700. In one implementation, the memory' 720 is a computer-readable medium. In one implementation, the memory' 720 is a volatile memory’ unit. In another implementation, the memory’ 720 is a non-volatile memory unit.

[0104] The storage device 730 is capable of providing mass storage for the system 700. In one implementation, the storage device 730 is a computer-readable medium. In various different implementations, the storage device 730 can include, for example, a hard disk device, an optical disk device, a storage device that is shared over a network by' multiple computing devices (for example, a cloud storage device), or some other large capacity’ storage device.

[0105] The input / output device 740 provides input / output operations for the system 700. In one implementation, the input / output device 740 can include one or more network interface devices, for example, an Ethernet card, a serial communication device, for example, a RS- 232 port, and / or a wireless interface device, for example, a 502.11 card. In another implementation, the input / output device can include driver devices configured to receive data and send output data to other input / output devices, for example, keyboard, printer and display devices 760. Other implementations, however, can also be used, such as mobile computing devices, mobile communication devices, set-top box television client devices, etc.

[0106] Although an example processing system has been described in FIG. 7, implementations of the subject matter and the functional operations described in this disclosure can be implemented in other types of digital electronic circuitry’, or in computer software, firmware, or hardware, including the structures disclosed in this disclosure and their structural equivalents, or in combinations of one or more of them.

[0107] This disclosure uses the term “configured” in connection with systems and computer program components. For a system of one or more computers to be configured to perform particular operations or actions means that the system has installed on it software, firmware, hardware, or a combination of them that in operation cause the system to perform the operations or actions. For one or more computer programs to be configured to perform particular operations or actions means that the one or more programs include instructions that, when executed by data processing apparatus, cause the apparatus to perform the operations or actions. Embodiments of the subject matter and the functional operations described in this disclosure can be implemented in digital electronic circuitry, in tangibly- embodied computer software or firmware, in computer hardware, including the structures disclosed in this disclosure and their structural equivalents, or in combinations of one or more of them. Embodiments of the subject matter described in this disclosure can be implemented as one or more computer programs, i.e.. one or more modules of computer program instructions encoded on a tangible non transitory storage medium for execution by, or to control the operation of, data processing apparatus. The computer storage medium can be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or a combination of one or more of them. Alternatively or in addition, the program instructions can be encoded on an artificially generated propagated signal, e.g., a machine-generated electrical, optical, or electromagnetic signal, that is generated to encode information for transmission to suitable receiver apparatus for execution by a data processing apparatus.

[0108] The term ’‘data processing apparatus” refers to data processing hardware and encompasses all kinds of apparatus, devices, and machines for processing data, including byway of example a programmable processor, a computer, or multiple processors or computers. The apparatus can also be. or further include, special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit). The apparatus can optionally include, in addition to hardware, code that creates an execution environment for computer programs, e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them.

[0109] A computer program, which may also be referred to or described as a program, software, a software application, an app, a module, a software module, a script, or code, can be written in any form of programming language, including compiled or interpreted languages, or declarative or procedural languages; and it can be deployed in any form, including as a stand alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A program may, but need not, correspond to a file in a file system. A program can be stored in a portion of a file that holds other programs or data, e.g., one or more scripts stored in a markup language document, in a single file dedicated to the program in question, or in multiple coordinated files, e.g., files that store one or more modules, sub programs, or portions of code. A computer program can be deployed to be executed on one computer or on multiple computers that are located at one site or distributed across multiple sites and interconnected by a data communication network.

[0110] In this disclosure, the term "‘database” is used broadly to refer to any collection of data: the data does not need to be structured in any particular way, or structured at all, and it can be stored on storage devices in one or more locations. Thus, for example, the index database can include multiple collections of data, each of which may be organized and accessed differently.

[0111] Similarly, in this disclosure the term “engine” is used broadly to refer to a softwarebased system, subsystem, or process that is programmed to perform one or more specific functions. Generally, an engine will be implemented as one or more software modules or components, installed on one or more computers in one or more locations. In some cases, one or more computers will be dedicated to a particular engine; in other cases, multiple engines can be installed and running on the same computer or computers.

[0112] The processes and logic flows described in this disclosure can be performed by one or more programmable computers executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows can also be performed by special purpose logic circuitry, e.g.. an FPGA or an ASIC, or by a combination of special purpose logic circuitry and one or more programmed computers.

[0113] Computers suitable for the execution of a computer program can be based on general or special purpose microprocessors or both, or any other kind of central processing unit. Generally, a central processing unit will receive instructions and data from a read only memory or a random access memory or both. The essential elements of a computer are a central processing unit for performing or executing instructions and one or more memory devices for storing instructions and data. The central processing unit and the memory’ can be supplemented by, or incorporated in, special purpose logic circuitry'. Generally, a computer will also include, or be operatively coupled to receive data from or transfer data to, or both, one or more mass storage devices for storing data, e g., magnetic, magneto optical disks, or optical disks. However, a computer need not have such devices. Moreover, a computer can be embedded in another device, e.g., a mobile telephone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a Global Positioning System (GPS) receiver, or a portable storage device, e.g., a universal serial bus (USB) flash drive, to name just a few.

[0114] Computer readable media suitable for storing computer program instructions and data include all forms of non volatile memory, media and memory' devices, including by way of example semiconductor memory devices, e g., EPROM, EEPROM, and flash memory devices; magnetic disks, e.g., internal hard disks or removable disks; magneto optical disks; and CD ROM and DVD-ROM disks.

[0115] To provide for interaction with a user, embodiments of the subject matter described in this disclosure can be implemented on a computer having a display device, e.g.. a CRT (cathode ray tube) or LCD (liquid crystal display) monitor, for displaying information to the user and a keyboard and a pointing device, e.g., a mouse or a trackball, by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback, e.g., visual feedback, auditory feedback, or tactile feedback; and input from the user can be received in any form, including acoustic, speech, or tactile input. In addition, a computer can interact with a user by sending documents to and receiving documents from a device that is used by the user; for example, by sending web pages to a web browser on a user’s device in response to requests received from the web browser. Also, a computer can interact with a user by sending text messages or other forms of message to a personal device, e.g., a smartphone that is running a messaging application, and receiving responsive messages from the user in return.

[0116] Data processing apparatus for implementing machine-learning models can also include, for example, special-purpose hardware accelerator units for processing common and compute-intensive parts of machine-learning training or production, i.e., inference, workloads.

[0117] Machine learning models can be implemented and deployed using a machine-learning framework, e.g., a PyTorch or a TensorFlow framework.

[0118] Embodiments of the subject matter described in this disclosure can be implemented in a computing system that includes a back end component, e.g., as a data server, or that includes a middleware component, e.g., an application server, or that includes a front end component, e.g.. a client computer having a graphical user interface, a web browser, or an app through which a user can interact with an implementation of the subject matter described in this disclosure, or any combination of one or more such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication, e.g., a communication network. Examples of communication networks include a local area network (LAN) and a wide area network (WAN), e.g.. the Internet.

[0119] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. In some embodiments, a server transmits data, e g., an HTML page, to a user device, e.g., for purposes of displaying data to and receiving user input from a user interacting with the device, which acts as a client. Data generated at the user device, e.g., a result of the user interaction, can be received at the server from the device.

[0120] While this disclosure contains many specific implementation details, these should not be construed as limitations on the scope of any invention or on the scope of what may be claimed, but rather as descriptions of features that may be specific to particular embodiments of particular inventions. Certain features that are described in this disclosure in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination. Moreover, although features may be described above as acting in certain combinations and even initially be claimed as such, one or more features from a claimed combination can in some cases be excised from the combination, and the claimed combination may be directed to a subcombination or variation of a subcombination.

[0121] Similarly, while operations are depicted in the drawings and recited in the claims in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. In certain circumstances, multitasking and parallel processing may be advantageous. Moreover, the separation of various system modules and components in the embodiments described above should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

Claims

CLAIMS1 . A computer-implemented method comprising: maintaining candidate monomer data characterizing a population of one or more candidate capsid monomers, wherein the candidate monomer data comprises, for each candidate capsid monomer in the population, (i) data defining a respective amino acid sequence of the respective candidate capsid monomer and (ii) data specifying a respective fitness score that characterizes a predefined property of a capsid comprising the respective candidate capsid monomer; initializing the population of candidate capsid monomers using (i) one or more initial capsid monomer sequences and (ii) a corresponding fitness score for each respective initial capsid monomer sequence; initializing current values of model parameters of a first machine-learning model; at each of a plurality of iterations, updating (i) the candidate monomer data and (ii) the current values of the model parameters of the first machine-learning model, the updating comprising: generating, using the first machine-learning model according to the current values of the model parameters and based on the respective amino acid sequences of one or more of the candidate capsid monomers in the population, data representing a set of query capsid monomers; using a second machine-learning model to process respective data representing each of the query capsid monomers to generate a respective output specifying a respective benchmark fitness score for the respective query capsid monomer; updating the current values of the model parameters of the first machinelearning model using the benchmark fitness scores for the query capsid monomers; adding one or more of the query capsid monomers into the population of candidate capsid monomers, and updating the candidate monomer data by including, for each of the set of query capsid monomers, (i) data defining the respective amino acid sequence of the respective query capsid monomer and (ii) data specifying the respective benchmark fitness score of the respective query capsid monomer; after performing the plurality of iterations, selecting one or more target capsid monomers from the population of candidate capsid monomers; and outputting data specifying the selected target capsid monomers.

2. The method of claim 1, wherein selecting one or more target capsid monomers comprises selecting a target capsid monomer having a highest fitness score from the population of candidate capsid monomers.

3. The method of claim 1 or claim 2, wherein the predefined property characterizes a manufacturability’ of capsids.

4. The method of claim 3, wherein the manufacturability of the capsid is defined based on a number of instances of the capsid that are generated when a plasmid coding for the capsid is transfected into one or more cells.

5. The method of claim 4, wherein the number of instances of the capsid that are generated when a plasmid coding the capsid is transfected into one or more cells is normalized by a number of generated wild-type sequences.

6. The method of any proceeding claim, wherein the predefined property characterizes a capacity of viruses that include the capsid to avoid neutralization.

7. The method of any proceeding claim, wherein the predefined property characterizes an immunoreactivity of the capsid.

8. The method of any proceeding claim, wherein the predefined property of the capsid characterizes a predicted capacity of viruses that include the capsid to penetrate a target tissue.

9. The method of any proceeding claim, wherein the predicted fitness score characterizes a packing capacity of the capsid.

10. The method of any proceeding claim, wherein the predicted fitness score characterizes a predicted capacity of viruses that include the capsid to integrate into host genomes.

11. The method of any proceeding claim, wherein the capsid corresponds to a capsid of an adeno-associated virus (AAV).

12. The method of any proceeding claim, further comprising: physically generating one or more viruses that include a protein having one of the selected target capsid monomer.

13. The method of claim 12, further comprising: applying the generated viruses to a subject to achieve a therapeutic effect in the subject.

14. The method of any proceeding claim, wherein the second machine-learning model comprises: an embedding neural network configured to process an input defining the amino acid sequence of the respective query capsid monomer to generate a respective embedding of the amino acid sequence; and a fitness prediction machine-learning model configured to process the respective embedding of the amino acid sequence to generate an output characterizing a respective predicted fitness score for the respective query capsid monomer.

15. The method of claim 14. wherein the second machine-learning model further comprises an error prediction machine-learning model configured to process an input representing the amino acid sequence of the respective query' capsid monomer to generate a respective fitness error, wherein the respective fitness error is an estimate of an error in the respective predicted fitness score generated by the fitness prediction machine-learning model.

16. The method of claim 15, wherein the respective benchmark fitness score is computed by combining: (i) the respective predicted fitness score generated by the fitness prediction machine-learning model, and (ii) the respective fitness error generated by the error prediction machine-learning model.

17. The method of claim 15 or claim 16, wherein the fitness prediction machine-learning model has a different model architecture than the error prediction machine-learning model.

18. The method of any of claims 14-17, wherein the embedding neural network has been trained using self-supervised learning on a set of training capsid monomers.

19. The method of claim 18, wherein the self-superv ised learning comprises training the embedding neural network on the set of training capsid monomers to perform a token unmasking task.

20. The method of claim 18 or claim 19, wherein the self-supervised learning comprises training the embedding neural network on the set of training capsid monomers to perform a next token prediction task.

21. The method of any of claims 14-20, wherein the fitness prediction machine-learning model has been trained on a set of training examples, wherein each training example corresponds to a respective training capsid monomer and includes: (i) an embedding of the training capsid monomer, and (ii) a target fitness score for the training capsid monomer.

22. The method of any proceeding claim, wherein generating the data representing the set of query capsid monomers comprises: generating data representing a set of mutant capsid monomers that are mutated from one or more of the candidate capsid monomers; initializing the set of query' capsid monomers by selecting from the set of mutant capsid; and updating the set of query capsid monomers using the first machine-learning model.

23. The method of claim 22, wherein the first machine-learning model is configured to process a first input characterizing an input capsid monomer to predict an adjusted fitness score of the input capsid monomer.

24. The method of claim 23, wherein the first input further characterizes one or more mutated mutations of the input capsid monomer from a corresponding wild-ty pe capsid monomer that the input capsid monomer is mutated from.

25. The method of claim 24, wherein the adjusted fitness score measures a combination of (i) a predicted fitness that characterizes the predefined property7of input capsid monomer, and (ii) a mutation distance between the input capsid monomer and the corresponding wildtype capsid monomer.

26. The method of claim 25, wherein the first machine-learning model comprises: an encoder network configured to process data specifying a respective input capsid monomer fragment that neighbors each mutation of the input capsid monomer from the corresponding wild-type capsid monomer to generate a respective feature vector; a pooling network layer configured to combine the respective feature vectors generated from the input capsid monomer fragments into a combined feature vector; and a decoder network configured to process the combined feature vector to generate the predictive score of the input capsid monomer.

27. The method of claim 25 or claim 26. wherein updating the set of query capsid monomers using the first machine-learning model comprises: at each of a plurality of iterations, performing: predicting, using the first machine-learning model, the adjusted fitness scores of a subset of query7capsid monomers; selecting, based on the predicted adjusted fitness scores, one or more seed capsid monomers from the subset of query capsid monomers; generating data representing an additional set of mutant capsid monomers that are mutated from the selected seed capsid monomers; and updating the set of query capsid monomers by including one or more of the additional set of mutant capsid monomers.

28. A system comprising: one or more computers; and one or more storage devices communicatively coupled to the one or more computers, wherein the one or more storage devices store instructions that, when executed by the one or more computers, cause the one or more computers to perform operations of any of claims 1-27.

29. One or more non-transitory computer storage media storing instructions that when executed by one or more computers cause the one or more computers to perform operations of any of claims 1-27.

Images