Training mRNA property prediction machine learning models using active learning

Abstract

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training a machine learning model. In one aspect, a method comprises: determining a respective importance score for each mRNA codon in a set of possible mRNA codons; at each of a plurality of training iterations in a sequence of training iterations, selecting a current batch of mRNA sequences for training the machine learning model at the training iteration using the importance scores for the mRNA codons, and training the machine learning model on at least the current batch of mRNA sequences; and outputting the trained machine learning model.

Description

TRAINING mRNA PROPERTY PREDICTION MACHINE LEARNING MODELSUSING ACTIVE LEARNINGCROSS-REFERENCE TO RELATED APPLICATION

[0001] This application claims priority to EP Patent Application No. 24315294.9, filed on June 17, 2024, the disclosure of which is hereby incorporated by reference in its entirety.TECHNICAL FIELD10002 ] This specification relates to using active learning for training machine learning models.BACKGROUND10003] The messenger ribonucleic acid (RNA), or mRNA, is a type of RNA molecule that plays a crucial role in gene expression and protein synthesis. The primary function of mRNA is to carry' the genetic instructions from DNAto the ribosomes, where proteins are synthesized. mRNA is typically single-stranded and can be several hundred to several thousand nucleotides in length. It consists of a sequence of codons, where each codon consists of three nucleotides that specify a particular amino acid or a start or a stop signal during protein synthesis. The sequence of codons determines the order in which amino acids are assembled during translation.|0004] Machine learning models receive an input and generate an output, e.g., a predicted output, based on the received input. Some machine learning models are parametric models and generate the output based on the received input and on values of the parameters of the model.

[0005] Some machine learning models are deep neural network models that employ multiple layers of models to generate an output for a received input. For example, a deep neural network is a deep machine learning model that includes an output layer and one or more hidden layers that each apply7a non-linear transformation to a received input to generate an output.SUMMARY

[0006] This specification generally describes a system implemented as computer programs on one or more computers in one or more locations that can train a machine learning model over a sequence of training iterations using active learning.|0007] Throughout this specification, an “embedding” of an entity' (e.g., a model input) can refer to a representation of the entity as an ordered collection of numerical values, e.g.. a vector, matrix, or other tensor of numerical values.

[0008] Throughout this specification, a “subject” can refer to an animal or a human.

[0009] Throughout this specification, a “batch” of data elements (e.g., model inputs to a machine learning model) can refer to a set of data elements, e g., a set of 5, or 10, or 100. or 1000 data elements.1 0101 Throughout this specification, the “uncertainty” of a machine learning model in generating a predicted label for a model input can refer to a confidence of the machine learning model in the predicted label for the model input.

[0011] Throughout this specification, the “diversity” of model inputs in a batch of model inputs can characterize a level of correlation between predicted labels generated by a machine learning model for model inputs included in the batch of model inputs. More specifically, lower levels of correlation between the predicted labels of the model inputs in the batch can be indicative of higher batch diversity, and conversely, higher levels of correlation between the predicted labels of the model inputs in the batch can be indicative of a lower batch diversity.

[0012] In one aspect, this disclosure provides a training method for training a machine learning model. The method can be implemented by a system including one or more computers. The system determines a respective importance score for each mRNA codon in a set of possible mRNA codons, where the importance score for an mRNA codon characterizes an impact of a presence of the mRNA codon in an mRNA sequence on a predicted property of the mRNA sequence. The system trains a machine learning model that is configured to process a model input that characterizes an input messenger ribonucleic acid (mRNA) sequence to generate a model output that includes a predicted score characterizing the predicted property of the input mRNA sequence. The training includes, at each of a plurality of training iterations in a sequence of training iterations: selecting a cunent batch of mRNA sequences for training the machine learning model at the training iteration using the importance scores for the mRNA codons; and training the machine learning model on at least the current batch of mRNA sequences. The system outs the trained machine learning model.|0013] In some implementations of the training method, to determine the respective importance score for each mRNA codon in the set of possible mRNA codons, the system trains a regression model to process a plurality of features that includes, for each mRNA codon in the set of mRNA codons, a respective feature based on a number of occurrences of the mRNA codon in an mRNA sequence to generate a predicted score characterizing the predicted property of the mRNA sequence, and determines the respective importance score for each mRNA codon in the set of possible mRNA codons using the regression model.

[0014] In some cases, the regression model includes a respective regression coefficient corresponding to each mRNA codon in the set of mRNA codons. To determine the importance score for each mRNA codon in the set of possible mRNA codons using the regression model, the system determines the importance score for each mRNA codon based on a value of the regression coefficient corresponding to the mRNA codon.

[0015] In some cases, the regression model is configured to generate the predicted score characterizing the predicted property' of the mRNA sequence as an affine function of the plurality' of features.

[0016] In some cases, the regression model is trained to optimize an objective function.

[0017] In some implementations of the training method, to selecting the current batch of mRNA sequences for training the machine learning model at the training iteration, for one or more mRNA sequences in the current batch of mRNA sequences, the system determines, for each position in the mRNA sequence, a set of eligible mRNA codons for the position, where an mRNA codon is designated as being eligible at a position if the mRNA codon codes for an amino acid in a corresponding position of an amino acid sequence of a target protein to be coded for by the mRNA sequence. The system identifies a plurality of positions in the mRNA sequence as being optimal positions in the mRNA sequence, and selects, for each position designated as an optimal position in the mRNA sequence, an mRNA codon for the position as a codon having a highest importance score from among the set of eligible mRNA codons for the position. The term “optimal,” when used in relation to a position in the mRNA sequence, as in “optimal position,” is used merely as a convenient identifier for such positions (e.g., positions occupied by a codon having a highest importance score from among the set of eligible codons for the position) and is not intended to convey any special meaning in this specification.

[0018] In some cases, the system further selects, for each position that is not designated as an optimal position in the mRNA sequence, an mRNA codon for the position other than a codon having a highest importance score from among the set of eligible mRNA codons for the position.

[0019] In some cases, for each position that is not designated as an optimal position in the mRNA sequence, to select the mRNA codon for the position other than the codon having the highest importance score from among the set of eligible mRNA codons for the position, the system stochastically selects the mRNA codon from among the set of eligible codons for the position other than the codon having the highest importance score from among the set of eligible mRNA codons for the position.

[0020] In some cases, identifying a plurality' of positions in the mRNA sequence as being optimal positions includes: stochastically selecting a proper subset of the positions in the mRNA sequence as being optimal positions in the mRNA sequence.1 0211 In some cases, stochastically selecting a proper subset of the positions in the mRNA sequence as being optimal positions in the mRNA sequence includes, for each position in the mRNA sequence: selecting the position in the mRNA sequence as being an optimal position with a predefined probability.

[0022] In some implementations of the training method, selecting a current batch of mRNA sequences for training the machine learning model at the training iteration using the importance scores for the mRNA codons includes: generating a pool of mRNA sequences using the importance scores for the mRNA codons; and selecting a plurality of mRNA sequences from the pool of mRNA sequences for inclusion in the current batch of mRNA sequences for training the machine learning model at the training iteration.

[0023] In some cases, to select a plurality of mRNA sequences from the pool of mRNA sequences for inclusion in the current batch of mRNA sequences for training the machine learning model at the training iteration, the system generates a set of candidate batches of mRNA sequences, where each candidate batch of mRNA sequences is a proper subset of the pool of mRNA sequences. The system generates, for each candidate batch mRNA sequences, a respective score for the candidate batch of mRNA sequences that characterizes: (i) an uncertainty of the machine learning model in generating predicted scores for the mRNA sequences in the candidate batch of mRNA sequences, and (li) a diversity of the mRNA sequences in the candidate batch of mRNA sequences. The system selects the current batch of mRNA sequences from the set of candidate batches of mRNA sequences based on the scores.

[0024] In some implementations of the training method, the training includes, at each of the plurality of training iterations in the sequence of training iterations: obtaining a respective target score for each mRNA sequence in the current batch of mRNA sequences, where a target score for an mRNA sequence defines a score that should be generated by the machine learning model by processing a model input characterizing the mRNA sequence. Training the machine learning model on at least the current batch of mRNA sequences includes: training the machine learning model on at least the current batch of model inputs using the target scores for the current batch of mRNA sequences.

[0025] In some cases, at each of a plurality’ of training iterations, the target score for each mRNA sequence in the current batch of mRNA sequences is generated by operations including:physically generating one or more instances of the mRNA sequence; determining one or more properties of the instances of the mRNA sequence; and determining the target score for the mRNA sequence based on the properties of the instances of the mRNA sequence.100261 In some implementations of the training method, the predicted property of the mRNA sequence characterizes a predicted level of protein expression from the mRNA sequence.|0027] In some implementations of the training method, the predicted property of the mRNA sequence characterizes a predicted stability of the mRNA sequence.

[0028] In some implementations of the training method, the predicted property of the mRNA sequence characterizes a predicted translation efficiency of the mRNA sequence.[0029| In some implementations of the training method, the machine learning model includes one or more of: a regression model, or a neural network model, or a random forest model, or a decision tree model, or a support vector machine model.10030] In some implementations of the training method, outputting the trained machine learning model includes: selecting an mRNA sequence using the machine learning model; and determining that the selected mRNA sequence should be included in an mRNA therapeutic agent (e.g., an mRNA vaccine).[00311 In some cases, the system further determines that a subject should receive the mRNA therapeutic agent (e.g., the mRNA vaccine) that includes the selected mRNA sequence.

[0032] In some cases, the method further includes physically synthesizing the vaccine that includes the selected mRNA sequence.

[0033] In some cases, the method further includes administering the vaccine that includes the selected mRNA sequence to a subject.10034] In some implementations of the training method, outputting the trained machine learning model includes: selecting an mRNA sequence using the machine learning model; and physically synthesizing the selected mRNA sequence.

[0035] In some cases, the method further including: performing experiments to determine one or more properties of the physically synthesized mRNA sequence.[0036| This disclosure also provides a prediction method for predicting a property of an mRNA sequence. The method can be implemented by a system including one or more computers. The system obtains a model input that characterizes an input messenger ribonucleic acid (mRNA) sequence, and processes the model input using a machine learning model togenerate a model output that comprises a predicted score characterizing the predicted property7of the input mRNA sequence. The machine learning model has been trained using the training methods described herein.100371 This disclosure also provides a vaccine comprising one or more mRNA sequences that have been selected using a machine learning model that is configured to process a model input that characterizes an input messenger ribonucleic acid (mRNA) sequence to generate a model output that comprises a predicted score characterizing a predicted property7of the input mRNA sequence, wherein the machine learning model has been trained using the training methods described herein.[00381 This disclosure also provides a design method for generating target mRNA sequences that are optimized for a property measure. The method can be implemented by a system including one or more computers. The system determines a set of importance scores for a set of possible mRNA codons, where each of the set of importance scores characterizes an impact of a presence of one or more mRNA codons in the set of possible mRNA codons in an mRNA sequence on a predicted property of the mRNA sequence. The system generates a plurality of candidate mRNA sequences based on the set of importance scores. In particular, for each position in a first candidate mRNA sequence, the system determines a respective set of eligible mRNA codons for the respective position, where an mRNA codon is designated as being eligible at a position if the mRNA codon codes for an amino acid in a corresponding position of an amino acid sequence of a target protein to be coded for by the first candidate mRNA sequence, for each eligible mRNA codon in the respective set of eligible mRNA codons, determines a respective selection probability7for the eligible mRNA codon based at least on one or more importance scores in the set of importance scores, and stochastically samples, from the respective set of eligible mRNA codons, a respective mRNA codon for the respective position according to the selection probabilities determined for the respective set of eligible mRNA codons. For each of the candidate mRNA sequences, the system processes a respective input characterizing the respective candidate mRNA sequence using a machine learning model to generate a respective model output comprising a respective predicted score characterizing the predicted property. The system then selects one or more target mRNA sequences from the plurality7of candidate mRNA sequences based on the predicted scores.

[0039] In some implementations, the set of importance scores include, for each mRNA codon in the set of possible mRNA codons, a respective single-codon importance score that characterizes an impact of the presence of the respective mRNA codon in the mRNA sequenceon the predicted property' of the mRNA sequence. In some cases, to determine the respective selection probability for the eligible mRNA codon, the system can compute the selection probability for the eligible mRNA codon as the single-codon importance score for the eligible mRNA codon normalized by a sum of the single-codon importance scores of the respective set of eligible mRNA codons for the respective position.

[0040] In some implementations, the set of importance scores include, for each two-codon subsequence formed by two mRNA codons selected from the set of possible mRNA codons, a respective two-codon importance score that characterizes an impact of the presence of the respective two-codon subsequence in the mRNA sequence on the predicted property of the mRNA sequence. In some cases, to determine the respective selection probability7for the eligible mRNA codon, the system can compute the selection probability7for the eligible mRNA codon as (i) the two-codon importance score for a two-codon subsequence that includes the mRNA codon that has been selected for the preceding position followed by the eligible mRNA codon normalized by (ii) a sum of the two-codon importance scores of a set of two-codon subsequences, wherein each of the set of two-codon subsequences includes the mRNA codon that has been selected for the preceding position followed by one of the respective set of eligible mRNA codons determined for the current position.

[0041] This disclosure also provides a system including one or more computers and one or more storage devices storing instructions that, when executed by the one or more computers, cause the one or more computers to perform the methods described above.

[0042] This disclosure also provides one or more computer storage media storing instructions that when executed by one or more computers, cause the one or more computers to perform the methods described above.

[0043] The subject matter described in this disclosure can be implemented in particular embodiments so as to realize one or more advantages.

[0044] mRNA-based therapeutics have dominated various domains of drug design, such as vaccine design, cancer immunotherapy, and gene therapy. This is in part due to the high transduction efficiency of mRNAs and the affordable cost associated with mRNA production. Like other therapeutics, mRNA drugs undergo extensive engineering, such as untranslated region and codon usage bias where the mRNA sequence is optimized to maximize cell transduction and protein expression level. The optimization method varies depending on the administration route and mRNA vehicle.

[0045] The search space for an mRNA therapeutic agent is enormous. The existence of synonymous codons that maintain the protein sequence but impact mRNA properties, or codon usage bias, adds to the complexity of the codon optimization in the mRNA sequence. For a protein with roughly 500 amino acids, the number of possible mRNA sequences is approximately 3500. Finding the optimal (or high-performance) sequence is thus very challenging.

[0046] Machine learning models have been developed to predict mRNA properties. However, obtaining labeled data for training the machine learning model is time-consuming and costly. The techniques described herein use active learning, a computational method to select the most informative training data, to overcome the challenges of lack of labeled data.

[0047] In particular, according to the described techniques, a computer system determines a respective importance score for each mRNA codon in a set of possible mRNA codons, where the importance score for an mRNA codon characterizes an impact of a presence of the mRNA codon in an mRNA sequence on the predicted property of the mRNA sequence. At each training iteration, the system uses the codon importance scores to select a new batch of model inputs and obtains a respective target label for each model input in the current batch of model inputs. A label for a model input defines a model output that should be generated by the machine learning model by processing the model input. The system can then train the machine learning model with the inputs from at least the current batch that has been labeled using the target labels for the model inputs.

[0048] The described techniques overcome the challenges of conventional techniques by allowing for sampling candidate training mRNA sequences characterized by high uncertainty. In particular, certain implementations of the described techniques determine importance scores for the list of mRNA codons, and use a stochastic process to select mRNA sequences for the mRNA pool based on importance scores. The codon importance scores can be learned (e.g., using a regression model) based on measurement data of mRNA property metrics in a particular host, and thus represent a correlation between each mRNA codon with the desired mRNA property. Therefore, the codon importance scores can be used as an effective guidance for selecting mRNA codon sequences. Combining the codon importance scores with a stochastic sampling process can extrapolate to optimal or suboptimal mRNA sequences and thus provides an effective means for sampling the vast mRNA codon sequence space, resulting in mRNA sequence pool with higher uncertainty distribution.

[0049] The mRNA sequence pool can be evaluated by an active learning method to select the subset of sequences based on uncertainty and diversity measures, resulting in training batches that increase the prediction accuracy of the machine learning model and reduce the number of training iterations required for training the machine learning model.|0050] In particular, the described techniques can increase the prediction accuracy of the machine learning model, reduce the number of training iterations required for training the machine learning model, and reduce the overall number of inputs for which a label is needed. Reducing the number of training iterations required for training the machine learning model can enable more efficient use of resources. For instance, training the machine learning model at each training iteration requires computational resources (e g., memory and computing power), and therefore reducing the number of training iterations can reduce the consumption of computational resources during training. Further, obtaining target labels for model inputs at each training iteration can be time-consuming and expensive, e.g., in cases where generating a target label for a model input can require performing a physical experiment, e.g., involving physically synthesizing and testing an mRNA sequence. Therefore, reducing the number of training iterations required for training the machine learning model can reduce the consumption of resources required for obtaining target labels for model inputs.

[0051] The details of one or more embodiments of the subject matter of this specification are set forth in the accompanying drawings and the description below. Other features, aspects, and advantages of the subject matter will become apparent from the description, the drawings, and the claims.BRIEF DESCRIPTION OF THE DRAWINGS

[0052] FIG. 1 shows an example workflow in an application scenario using a training system to train an mRNA property prediction machine learning model.

[0053] FIG. 2 shows an example of a training system.

[0054] FIG. 3 is a flow diagram illustrating an example process for training a machine learning model.

[0055] FIG. 4 is a flow diagram illustrating an example process for determining codon importance scores.(0056] FIG. 5 is a flow diagram illustrating an example process for generating a current training batch.

[0057] FIG. 6 is a flow diagram illustrating another example process for generating a current training batch.

[0058] FIG. 7 is a flow diagram of an example process 700 for generating one or more target mRNA sequences.|00591 Like reference numbers and designations in the various drawings indicate like elements.DETAILED DESCRIPTION|0060] Predicting the properties of mRNA sequences is an important aspect of the development of mRNA-based therapeutics. For example, in mRNA, predicting mRNA stability, expression levels, and potential immunogenicity are crucial in designing effective mRNA agents, such as mRNA vaccines.

[0061] Next-generation sequencing techniques have enabled researchers to generate thousands of mRNA sequences and measure their corresponding properties. However, the redundancy of the genetic code (64 codons for 21 amino acids) results in a vast search space that grows exponentially with sequence length. This makes it unfeasible to test all possible mRNA candidates experimentally. Machine learning models have been developed to predict mRNA properties and guide the selection and design of optimal sequences. However, obtaining labeled data for training the machine learning model is time-consuming and costly. Therefore, advanced techniques are needed to guide the selection of training data.

[0062] FIG. 1 shows an example workflow in an application scenario using a training system 200 to train an mRNA property prediction machine learning model 255.1 0631 The training system 200 is configured to generate a plurality- of training batches 110 and use the training batches 110 to train the machine learning model 255. The machine learning model 255 is configured to process a model input that characterizes an input mRNA sequence to generate a model output that characterizes a predicted property of the input mRNA sequence. The property to be predicted can include, for example, expression levels of one or more proteins (e.g., antigenic proteins of a target pathogen) of the mRNA in a specific type of cell or tissue, a stability' under one or more environmental conditions, a switching factor of the mRNA molecule in a specific type of cell or tissue, or a degradation rate of the mRNA molecule under one or more environmental conditions.

[0064] Each training batch 110 includes a respective set of training examples, with each training example including a respective training input characterizing an input mRNA sequence and a corresponding training label characterizing a respective target property measure for the respective input mRNA sequence.|0065] As described below, the training system 200 is configured to generate training batches 110 that are optimized to increase the prediction accuracy of the machine learning model 255 and reduce the number of training iterations required for training the machine learning model 255.

[0066] Once the machine learning model 255 has been trained, an mRNA design / selection system 250 can use the trained machine learning model 255 to design or select target mRNA sequences that are optimized for a property measure.1 067 In some cases, the system 250 can screen the mRNA library that defines a set of candidate mRNAs to identify mRNAs having desirable property measures. More specifically, the system 250 can use the machine learning model 255 to predict a respective property7measure for each mRNA in the mRNA library'. The system 250 can designate a proper subset of the mRNAs in the mRNA library as being ’’target" mRNAs 260 based at least in part on the predicted property measures.[00681 The system 250 can select a proper subset of the mRNAs in the mRNA library to be designated as target mRNAs in any variety of ways. For instance, the system 250 can designate any mRNA having a property7measure that satisfies a predefined threshold as being a target mRNA. As another example, the system 250 can designate a predefined number of mRNAs having the highest property7measures as being target mRNAs.

[0069] In some cases, the system 250 can be used with reinforcement learning or another technique to determine the optimal codon sequence of an mRNA. For example, a reinforcement learning system can use a reinforcement learning (RL) model to process one or more of the candidate sequences to generate one or more new sequences. The RL model can be trained using a reward signal including mRNA molecule properties predicted using the ML model 255. An optimal sequence can be selected from the new sequences.

[0070] The designed or selected target mRNAs 260 can be manufactured, i.e., physically synthesized, using any suitable techniques 270 and used in any variety of applications. For example, the manufactured mRNAs can be applied as a therapeutic 280 to a subject 290 toachieve a therapeutic effect in the subjects, such as an mRNA vaccine for an infectious disease or a cancer, or as a gene delivery agent to treat genetic disorders.

[0071] FIG. 2 shows an example training system 200. The training system 200 is an example of a system implemented as computer programs on one or more computers in one or more locations in which the systems, components, and techniques described below are implemented.10072] The training system 200 trains a machine learning model 255 over a sequence of training iterations to perform a machine learning task, in particular, to process a model input that characterizes an input mRNA sequence to generate a model output that includes a predicted score characterizing the predicted property of the input mRNA sequence. The training system 200 can train the machine learning model 255 to predict any appropriate properties of the input mRNA sequence. Examples of the predicted properties can include a predicted level of protein expression from the mRNA sequence, a predicted stability of the mRNA sequence, or a predicted translation efficiency of the mRNA sequence. Examples of the machine learning model 255 and the training process are described in U. S. Provisional Application No. 63 / 516,226, “Predicting mRNA properties using large language transformer models / ’ filed on July 28, 2023.[0073 j The model input to the machine learning model 255 can be represented in any appropriate way. In one example, the model input can include a codon sequence vector that numerically encodes the codon sequence of the input mRNA, e.g., by mapping each codon of a list of codons (e.g., a list of all 64 codons) to a respective numerical value and concatenating the mapped numerical values. In another example, the model input can include an embedding generated by processing the codon sequence vector using an embedding machine-learning model that has been trained on mRNA sequences. In some cases, the model input can include data characterizing certain properties of the input mRNA sequence, such as the length of the sequence, the secondary folding structure, the GC content — the percentage of guanine (G) and cytosine (C) nucleotides in the sequence, presence of certain known motifs, calculation of A-mer frequencies, likelihood of the sequence binding to RNA binding proteins or microRNAs, and so on.10074] The machine learning model 255 can be any appropriate type of machine learning model and can have any appropriate machine learning model architecture. For instance, the machine learning model 255 can include one or more of: a neural network, or a decision tree, or a random forest, or a support vector machine. In implementations where the machine learning model includes one or more neural networks, each of the neural networks can include any appropriate types of neural network layers (e.g., fully connected layers, message passing layers,convolutional layers, atention layers, and so forth) in any appropriate number (e.g., 5 layers, or 10 layers, or 100 layers) and connected in any appropriate configuration (e.g., as a directed graph of layers). A particular example of the machine learning model 255 includes a Bayesian ridge regression model. In this case, the model 255 ’s output characterizes a probability distribution of a predicted metric for the predicted property of the input mRNA sequence. In another example, the machine learning model 255 includes a gradient boosting on decision trees (CatBoost) model.

[0075] The training system 200 includes a codon scoring engine 210, a batch generation engine 220, a labeling engine 230, and ataining engine 230, which are each described in more detail next.

[0076] The codon scoring engine 210 is configured to determine a respective importance score 215 for each mRNA codon in a set of possible mRNA codons (e.g., each of the complete list of 64 codons). The importance score for an mRNA codon characterizes the impact of the presence of the mRNA codon in an mRNA sequence on a predicted property of the mRNA sequence. For example, for mRNA sequences corresponding to a particular protein, the respective importance score for each particular codon in the mRNA sequences can characterize an importance of the particular codon on the protein expression level of the particular protein. Example techniques of determining the importance scores will be described with reference to FIG. 4.100771 The batch generation engine 220 is configured to select, at each training iteration in a sequence of training iterations, a respective current batch 225 of model inputs for training the machine learning model 255 at the training iteration. In general, the batch generation engine 220 selects training input batches 225 based on the importance scores 215 for the list of codons. Some example techniques of generating mRNA sequences for the training input batch 225 are described in more detail below with reference to FIG. 5.[00781 In some cases, the batch generation engine 220 implements a policy for selecting the current batch of model inputs that accounts for both the prediction uncertainty associated with individual model inputs and for the diversity of the current batch of model inputs as a whole. More specifically, as part of selecting the current batch of model inputs, the batch generation engine 220 evaluates a set of “candidate” batches of model inputs. For each candidate batch of model inputs, the system determines a score for the candidate batch of model inputs that characterizes both: (i) an uncertainty of the machine learning model 255 in generating predicted labels for the model inputs in the candidate batch of model inputs, and (ii) a diversity of the model inputs in the candidate batch of model inputs. The batch generation engine 220 can thenselect the current batch of model inputs from the set of candidate batches of model inputs based on the scores, e.g., by selecting a candidate batch of model inputs associated with the highest score as the current batch of model inputs.100791 The system can generate a score for a candidate batch of model inputs that jointly characterizes the prediction uncertainty and the diversity of the model inputs in the batch in any of a variety7of possible ways. Example techniques for selecting the current batch are further described in more detail below with reference to FIG. 6.

[0080] The labeling engine 230 is configured to obtain, at each training iteration in the sequence of training iterations, a respective target label 235 for each model input in the current batch of model inputs. A target label 235 for a model input defines a model output that should be generated by the machine learning model 255 by processing the model input.100811 The labeling engine 230 can obtain the target labels 235 for the model inputs in the current batch of model inputs 225 in any of a variety of possible ways. A few example techniques for obtaining target labels 235 for model inputs are described next.

[0082] For instance, the labeling engine 230 can provide instructions, e.g., by way of a user interface or an application programming interface (API) made available by the system 200, that one or more physical experiments should be performed to obtain the target labels 235. In some cases, determining a target label for a model input 235 specifying an mRNA sequence can require physically synthesizing and testing the properties of one or more instances of the mRNA. The labeling engine 230 can receive the results of the experiments, e.g., by way of the user interface or the API, and can associate each model input in the current batch 225 with a respective target label based on the results of the experiments.

[0083] As another example, the labeling engine 230 can perform numerical simulations in order to obtain the target labels 235 for the current batch of model inputs 225. The numerical simulations can include, e.g., molecular dynamics (MD) simulations, or quantum mechanics / molecular mechanics (QM / MM) simulations, or density functional theory (DFT) simulations, coarse-grained (CG) and continuum models simulation, and so forth.1 084] The training engine 240 is configured to, at each training iteration, train the machine learning model 255 on at least the current batch of model inputs 225 using the target labels 235 for the current batch of model inputs 225. Optionally, the training engine 240 can also train the machine learning model 255 on model inputs and target labels obtained at any previous training iterations.

[0085] Training the machine learning model 255 on a model input can include training the machine learning model 255 to reduce a discrepancy between: (i) a predicted label generated by the machine learning model 255 for the model input, and (ii) the target label 235 for the model input. More specifically, the training engine 240 can train the machine learning model 255 to optimize (e.g., minimize) an objective function that measures an error between: (i) the predicted label for the model input, and (ii) the target label for the model input. The objective function can measure the error between a predicted label and a target label, e.g., by a cross-entropy loss, or a squared-error loss, or a hinge loss, or a Huber loss, or in any other appropriate way. In an example, the objective function J can be defined as / (y, y) = 1 1 L(.yi>yd, where y represents a vector of actual values or labels, y represents a vector of predicted values outputted by the machine learning model 255, L represents a loss function that quantifies the error between y and y, N is the number of data points, ytis the predicted value for the i -th data point, and ytis the actual label or value for the i-th data point.

[0086] The training engine 240 can train the machine learning model 255 using any appropriate machine learning training technique. For instance, for a machine learning model 255 implemented as a neural network, the training engine 240 can train the neural network on a model input by processing by the model input to generate a predicted label, evaluating the objective function on the predicted label, determining gradients of the objective function with respect to a set of neural network parameters of the neural network, and adjusting the current values of the set of neural network parameters using the gradients. The training engine 240 can determine the gradients of the objective function, e.g., using backpropagation, and can adjust the current values of the neural network parameters based on the gradients using the update rule of an appropriate gradient descent optimization technique, e.g., RMSprop or Adam.|0087] After training the machine learning model 255, the training system 200 can output the trained machine learning model 255, e.g., by storing data defining the trained machine learning model 255 in a memory, or by transmitting data defining the trained machine learning model 255 over a data communications network. The trained machine learning model 255 can be defined by data specifying the architecture of the machine learning model 255 and the trained values of the set of model parameters of the machine learning model.

[0088] A downstream system, e.g., the mRNA design / selection system 250 of FIG. 1, can use the trained machine learning model 255 to design or select target mRNA sequences with desirable properties. In some cases, the mRNA sequences can be physically synthesized and applied in a clinical application, as described with reference to FIG. 1.

[0089] FIG. 3 is a flow diagram of an example process 300 for training a machine learning model. For convenience, the process 300 will be described as being performed by a system of one or more computers located in one or more locations. For example, a training system, e.g., the training system 200 of FIG. 2, appropriately programmed in accordance with this specification, can perform the process 300.

[0090] At 310, for each mRNA codon in a set of possible mRNA codons (e.g., each of the complete list of 64 codons), the system determines a respective importance score for the respective mRNA codon. The importance score for an mRNA codon characterizes the impact of the presence of the mRNA codon in an mRNA sequence on a predicted property of the mRNA sequence. Example techniques of determining the importance scores will be described with reference to FIG. 4.|0091] Steps 320 - 350 of the process 300 are performed at each training iteration in a sequence of one or more training iterations. For convenience, the description which follows will reference a “current” training iteration in the sequence of training iterations.

[0092] At 320, the system generates a current batch of model inputs to the machine learning model. The current batch can include any appropriate number of model inputs, e.g., 10 model inputs, or 50 model inputs, or 100 model inputs. Example techniques of generating the current batch will be described with reference to FIG. 5 and 6.|0093] At 330, the system obtains a respective target label for each model input in the current batch of model inputs. The target label for a model input defines a model output that should be generated by the machine learning model by processing the model input. A target label for a model input can be generated, e.g., by physically synthesizing one or more instances of the mRNA corresponding to the model input, determining one or more properties of the physically synthesized instances of the mRNA, and determining the target label for the model input based on the properties. As another example, the system can determine a target label for a model input by performing one or more numerical simulations (as described above with reference to FIG. 2).|0094] At 340, the system trains the machine learning model on at least the current batch of model inputs using the target labels for the current batch of model inputs. Optionally, the system can train the machine learning model on: (i) the current batch of model inputs, and (ii) batches of model inputs selected at any of the preceding training iterations.

[0095] At 350, the system determines whether a termination criterion for terminating the training of the machine learning model has been satisfied. The system can determine that atermination criterion has been satisfied, e.g., if a predefined number of training iterations have been performed, or if a performance (e.g., prediction accuracy) of the machine learning model exceeds a threshold when evaluated on a set of validation data.100961 In response to determining that the termination criterion has been satisfied, the system outputs the trained machine learning model (at 360). In response to determining that the termination criterion has not been satisfied, the system returns to step (320) and proceeds to the next training iteration.

[0097] FIG. 4 is a flow diagram of an example process 400 for determining the respective importance score for the set of mRNA codons. For convenience, the process 400 will be described as being performed by a system of one or more computers located in one or more locations. For example, a training system, e.g., the training system 200 of FIG. 2, appropriately programmed in accordance with this specification, can perform the process 400.|0098] At 410, the system trains a regression model to process an input that characterizes, for each mRNA codon in the set of mRNA codons, a respective feature based on a number of occurrences (count) of the mRNA codon in an mRNA sequence to generate a predicted score characterizing the predicted property of the mRNA sequence.

[0099] For example, the input to the regression model can include a vector that characterizes a respective count of each of the set of codons in the coding sequence of the input mRNA. The coding sequence (CDS) of an mRNA refers to the portion of the mRNA molecule that directly specifies the sequence of amino acids in the protein it codes for. The coding sequence is located between the 5’ untranslated region (UTR) at the beginning of the mRNA and the 3‘ UTR at the end of the mRNA. In another example, the model input can include a vector that characterizes a respective codon frequency of each of the list of codons in the coding sequence of the input mRNA. In some cases, the codon frequency is defined as the count of a particular codon in the coding sequence normalized by the total codon count of the coding sequence. In some cases, the codon frequency can be defined as another function of the count of a particular codon, such as using the TF-IDF algorithm. The TF-1DF representation of codon frequency balances the frequency of a codon within a specific mRNA sequence (term frequency, TF) with its rarity across a larger set of mRNAs (inverse document frequency, IDF). In some cases, this can provide an improved representation over raw codon frequency counts because common codons that appear ubiquitously in many mRNA sequences may not be as informative as those that are more unique to specific sequences. Thus, TF-IDF effectively diminishes the impact of common codons and amplifies the significance of rarer ones.

[0100] The regression model can be any appropriate regression model. In a particular example, the regression model can be a linear regression model, i.e., the regression model is configured to generate the predicted score characterizing the predicted property of the mRNA sequence as an affine function of the plurality of features.|0101] The system can use any appropriate technique to train the regression model on a set of training examples. For example, the system can train the regression model to optimize a ridge regression objective function on the training examples. Each training example includes (i) a training input that characterizes the respective counts of the set of codons in the coding sequence of the input mRNA, such as the codon frequency vector of the coding sequence of the input mRNA and (ii) a corresponding training label that characterizes the target score characterizing the predicted property7of the mRNA in the training input.|0102] At 420, the system determines the respective importance score for each mRNA codon in the set of possible mRNA codons using the regression model.

[0103] In some cases, when the regression model includes a respective regression coefficient corresponding to each mRNA codon in the set of mRNA codons (e.g., when the regression model is a linear regression model or a polynomial regression model), the importance score can be determined based on the value of the regression coefficient corresponding to the mRNA codon.|0104] FIG. 5 is a flow diagram of an example process 500 for generating a current pool of model inputs at the current training iteration. For convenience, the process 500 will be described as being performed by a system of one or more computers located in one or more locations. For example, a training system, e.g., the training system 200 of FIG. 2, appropriately programmed in accordance with this specification, can perform the process 500.[01051 In general, the process 500 is performed for each mRNA sequence in the current batch of mRNA. For a particular mRNA sequence in the current batch of mRNA, the steps 510 and 520 can be performed for each codon position in the coding sequence of the particular mRNA sequence to determine the codon at the codon position.10106] At step 510, the system determines a set of eligible mRNA codons for the codon position. In general, an mRNA codon is designated as being eligible at a position if the mRNA codon codes for an amino acid in a corresponding position of an amino acid sequence of a target protein to be coded for by the mRNA sequence. As there are 64 possible codons but only 20 standard amino acids, most amino acids can be coded for by each of a set of multiple codons,also know n as synonymous codons. At step 510, the system determines the set of synonymous mRNA codons that code for the same amino acid at the codon position.

[0107] At step 520, the system selects one of the eligible codons for the codon position based on the codon importance scores. The system can select the codon from the set of eligible codons for the codon position in any of a variety of ways.10108] In some implementations, the system can identify a plurality of positions in the mRNA sequence as being optimal positions, and select, for each position designated as an optimal position in the mRNA sequence, an mRNA codon for the position as a codon having a highest importance score from among the set of eligible mRNA codons for the position. For each position that is not designated as an optimal position in the mRNA sequence, the system can stochastically select an mRNA codon for the position other than a codon having a highest importance score from among the set of eligible mRNA codons for the position. To identify the plurality of positions in the mRNA sequence as being optimal positions comprises, the system can stochastically select the proper subset of the positions in the mRNA sequence as being optimal positions in the mRNA sequence, e.g., with a predefined probability. That is, for each codon position, with a predefined probability p, the system can stochastically determine whether the codon position is an optimal position, and based on the determination, either assign the codon having the highest importance score from among the set of eligible mRNA codons to the codon position, or assign a codon other than a codon having a highest importance score from among the set of eligible mRNA codons for the position.

[0109] In some implementations, the system can select the codon for the codon position from the set of eligible codons stochastically with probabilities that are determined by the importance scores of the set of eligible codons. For example, for a particular codon in the set of eligible codons having a higher importance score, the system can select the particular codon with a higher probability. In a particular example, the system can determine the respective probabilities of selection of each codon in the set of eligible codons as the importance score of the respective codon normalized by the sum of the importance scores of all the codons in the set of eligible codons. In an illustrative example, the set of eligible codons for encoding a particular amino acid consists of 3 codons Cl, C2, and C3, with respective importance scores of 14, 4, and 2. The system can determine the respective selection probabilities for Cl, C2, and C3 as 0.7, 0.2, and 0.1, respectively.

[0110] To generate a candidate mRNA codon sequence for a particular protein, for each amino acid position, the system can stochastically sample the eligible codons at that position using theprobabilities described above. That is, the stochastic sampling process is independently performed at each position of the codon sequence.

[0111] In a more generalized process, the system can generate a candidate mRNA codon sequence using a Markov process based on the amino acid sequence. That is, the sampling of eligible codons at a current amino acid position depends on the codon selected at the preceding amino acid position. The stochastic sampling for the codon at a current amino acid position is performed based on probabilities found in a corresponding row in the transition matrix, which are probabilities of transitioning from a particular codon (that has been selected for the preceding position) to each of the eligible codons at the current position. The transition probabilities of double codon combinations can be determined in a similar process as determining the codon importance scores, e.g., using a regression model.10112] Using the Markov process to generate candidate mRNA codon sequences can capture dependencies between adjacent codons that influence the desired property, potentially leading to a more effective exploration of the mRNA sequence space.

[0113] FIG. 6 is a flow diagram of another example process 600 for generating a current batch of model inputs at the current training iteration. For convenience, the process 600 will be described as being performed by a system of one or more computers located in one or more locations. For example, a training system, e.g., the training system 200 of FIG. 2, appropriately programmed in accordance with this specification, can perform the process 600.10114] At 610, the system generates a pool of mRNA sequences using the importance scores for the mRNA codons. The system can generate the pool of mRNA sequences using any of a variety of ways. For example, the system can use the techniques described with reference to FIG. 5 to generate mRNA sequences to be included in the pool of model inputs.[0H5| At 620. the system selects a plurality of mRNA sequences from the pool of mRNA sequences for inclusion in the current batch of mRNA sequences.[0116| In some implementations, to select the plurality of mRNA sequences from the pool of mRNA sequences, the system generates a set of candidate batches of mRNA sequences, and determines, for each candidate batch mRNA sequences, a respective score for the candidate batch of mRNA sequences. The system then selects the current batch of mRNA sequences from the set of candidate batches of mRNA sequences based on the scores. For example, the system can select the candidate batch having highest score as the current batch of mRNA sequences. In general, the score for a candidate batch characterizes: (i) an uncertainty of the machine learningmodel in generating predicted scores for the mRNA sequences in the candidate batch of mRNA sequences, and (ii) a diversity of the mRNA sequences in the candidate batch of mRNA sequences. The system can determine the candidate batch score in any of a variety of ways.101171 In some cases, the system determines, for each pair of model inputs included in the batch of model inputs, a respective covariance between: (i) a predicted label for a first model input in the pair of model inputs, and (ii) a predicted label for a second model input in the pair of model inputs. In particular, the predicted label of a model input is associated with a distribution over a space of possible values of the predicted label, e.g., as a result of uncertainty in the parameter values of the machine learning model. (The uncertainty in the parameter values of the machine learning model can arise, e.g., because the current parameter values of the machine learning model define an uncertain estimate for “target” parameter values of the machine learning model that would globally optimize an objective function). Thus the predicted label of a model input defines a random variable over the space of possible values of the predicted label. The “covariance” between the predicted labels of a pair of model inputs thus refers to a covariance between a pair of random variables corresponding to the pair of model inputs. The covariances for the pairs of model inputs in the batch can be represented as a covariance matrix, e.g., where the entry (i,y) of the covariance matrix defines the covariance for model input i and model input j in the candidate batch of model inputs.

[0118] The covariance matrix characterizes both the predictive uncertainty and the diversity of the model inputs in the batch of model inputs. In particular, the covariance between the predicted labels of a pair of identical model inputs defines the variance in the predicted label of the model input. The variance in the predicted label of a model input characterizes the spread or dispersion of the distribution of the values of the predicted label of the model input, and thus defines an uncertainty (e.g., an uncertainty score) of the machine learning model in generating the predicted label for the model input. The covariance between the predicted labels of a pair of different model inputs measures an amount of correlation between the predicted labels of the pair of model inputs, and thus the collection of covariances between pairs of different model inputs collectively characterize the diversity of the model inputs in the batch.[0119| The system can generate the covariance matrix in any of a variety of possible ways. The particular choice of process used for generating the covariance matrix may depend on the machine learning model and the prediction task under consideration, and may be driven, e.g., based on empirical comparisons of perfonnance, available computing power and memory resources, and so forth.

[0120] The system determines the score for the candidate batch of model inputs based on the determinant of the covariance matrix. For instance, the system can determine the score for the batch of model inputs as a logarithm of the determinant of the covariance matrix. The determinant of the covariance matrix defines a single numerical value that summarizes the covariance matrix, and in particular, that characterizes the predictive uncertainty and the diversity of the model inputs in the batch of model inputs.

[0121] FIG. 7 is a flow diagram of an example process 700 for generating one or more target mRNA sequences for encoding a protein sequence. The goal for the process 700 is to generate target mRNA sequences that are optimized for a property measure. For convenience, the process 700 will be described as being performed by a system of one or more computers located in one or more locations. For example, an mRNA design / selection system, e.g., the mRNA design / selection system 250 of FIG. 1, appropriately programmed in accordance with this specification, can perform the process 700.

[0122] At 710, the system determines a set of importance scores for a set of possible mRNA codons. Each importance score characterizes the impact of a presence of one or more mRNA codons in the set of possible mRNA codons in an mRNA sequence on a predicted property of the mRNA sequence.[01231 In some cases, the set of importance scores include, for each mRNA codon in the set of possible mRNA codons, a respective single-codon importance score that characterizes the impact of the presence of the respective mRNA codon in the mRNA sequence on the predicted property of the mRNA sequence. The single-codon importance scores can be determined using techniques similar to those described with reference to FIG. 4.

[0124] In some cases, the set of importance scores includes, for each two-codon subsequence formed by two mRNA codons selected from the set of possible mRNA codons, a respective two- codon importance score that characterizes an impact of the presence of the respective two-codon subsequence in the mRNA sequence on the predicted property’ of the mRNA sequence.|0125] The two-codon importance scores can be determined using techniques similar to those used for determining the single-codon importance scores. The system can train a regression model to process an input that characterizes, for each two-codon subsequence in a set of possible two-codon subsequences, a respective feature based on the number of occurrences (count) of the two-codon subsequence in an mRNA sequence to generate a predicted score characterizing the predicted property of the mRNA sequence. As there are 64 possible mRNA codons, the set of possible two-codon subsequences include 4069 two-codon subsequences.

[0126] For example, the input to the regression model can include a vector that characterizes a respective count of each of the set of two-codon subsequences in the coding sequence of the input mRNA. In another example, the model input can include a vector that characterizes a respective two-codon subsequence frequency of each of the set of possible two-codon subsequences in the coding sequence of the input mRNA. In some cases, the two-codon subsequence frequency is defined as the count of a particular two-codon subsequence in the coding sequence normalized by the total two-codon sequence count of the coding sequence. In some cases, the two-codon subsequence frequency can be defined as another function of the count of a particular two-codon subsequence, such as using the tf-idf algorithm.

[0127] The system can use any appropriate technique to train the regression model on a set of training examples. After the regression model has been trained, the system can determine the importance scores using the trained regression model.

[0128] Steps 720 and 740 can be perfonned for each codon position in the coding sequence for each of one or more candidate mRNA sequences to determine the codon at the codon position.[0129) At 720, the system determines a set of eligible mRNA codons for the codon position. As described in more detail with reference to FIG. 5, an mRNA codon is designated as being eligible at a position if the mRNA codon codes for an amino acid in a corresponding position of an amino acid sequence of a target protein to be coded for by the first candidate mRNA sequence.

[0130] At 730, the system determines, for each mRNA codon in the set of eligible mRNA codons, a respective selection probability for the eligible mRNA codon based at least on one or more importance scores in the set of importance scores.[0131 [ For example, in some cases, the set of importance scores include single-codon importance scores described above. To determine the selection probabilities for an eligible mRNA codon, the system can compute the selection probability for the eligible mRNA codon as the single-codon importance score for the eligible mRNA codon normalized by a sum of the single-codon importance scores of the respective set of eligible mRNA codons for the respective position.

[0132] In some cases, the set of importance scores include the two-codon importance scores described above. To determine the selection probability for an eligible mRNA codon at a current position after the first position, the system computes the selection probability7for the eligiblemRNA codon as (i) the two-codon importance score for a two-codon subsequence that includes the mRNA codon that has been selected for the preceding position followed by the eligible mRNA codon normalized by (ii) a sum of the two-codon importance scores of a set of two-codon subsequences, where each of the set of two-codon subsequences includes the mRNA codon that has been selected for the preceding position followed by one of the respective set of eligible mRNA codons determined for the current position.

[0133] At 740, the system stochastically samples, from the respective set of eligible mRNA codons, an mRNA codon for the respective position based on the selection probabilities determined for the respective set of eligible mRNA codons.

[0134] In some cases, the system can perform the stochastic sampling process independently at each position of the candidate mRNA codon sequence. That is, the system can stochastically sample the mRNA codon at each position according to the selection probabilities computed using the single-codon importance scores.

[0135] In some cases, the system can perform the stochastic sampling process using a Markov process based on the amino acid sequence. In these cases, the sampling of eligible codons at a current position depends on the codon selected at the position. The system can sample the eligible codons for the current position using the selection probabilities computed using the two- codon importance scores described above. That is, the selection probabilities computed using the two-codon importance scores are used as the transition probabilities in the Markov process.10136] At 750, for each candidate mRNA sequence, the system processes a respective input characterizing the respective candidate mRNA sequence using a machine learning model to generate a respective model output comprising a respective predicted score characterizing the predicted property. In some cases, the machine learning model can be a model that has been trained using the techniques described above with references to FIGs. 2-6.

[0137] At 760, the system selects one or more target mRNA sequences from the of candidate mRNA sequences based on the predicted scores. As described above with reference to FIG. 1, the system can select a proper subset from the candidate mRNA sequences as the target mRNA sequences in any variety of ways. For instance, the system can designate any candidate mRNA sequence having the property score that satisfies a predefined threshold as being a target mRNA sequence. As another example, the system can designate a predefined number of candidate mRNA sequences having the highest scores as being target mRNA sequences.

[0138] This specification uses the term “configured'’ in connection with systems and computer program components. For a system of one or more computers to be configured to perform particular operations or actions means that the system has installed on it software, firmware, hardware, or a combination of them that in operation cause the system to perform the operations or actions. For one or more computer programs to be configured to perform particular operations or actions means that the one or more programs include instructions that, when executed by data processing apparatus, cause the apparatus to perform the operations or actions.

[0139] Embodiments of the subject matter and the functional operations described in this specification can be implemented in digital electronic circuitry, in tangibly-embodied computer software or firmware, in computer hardware, including the structures disclosed in this specification and their structural equivalents, or in combinations of one or more of them. Embodiments of the subject matter described in this specification can be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a tangible non-transitory storage medium for execution by, or to control the operation of, data processing apparatus. The computer storage medium can be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or a combination of one or more of them. Alternatively or in addition, the program instructions can be encoded on an artificially-generated propagated signal, e.g., a machine-generated electrical, optical, or electromagnetic signal, that is generated to encode information for transmission to suitable receiver apparatus for execution by a data processing apparatus.

[0140] The term “data processing apparatus” refers to data processing hardware and encompasses all kinds of apparatus, devices, and machines for processing data, including by way of example a programmable processor, a computer, or multiple processors or computers. The apparatus can also be, or further include, special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application-specific integrated circuit). The apparatus can optionally include, in addition to hardware, code that creates an execution environment for computer programs, e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them.[oi4i] A computer program, which may also be referred to or described as a program, software, a software application, an app, a module, a software module, a script, or code, can be written in any form of programming language, including compiled or interpreted languages, or declarative or procedural languages; and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in acomputing environment. A program may, but need not, correspond to a file in a file system. A program can be stored in a portion of a file that holds other programs or data, e.g., one or more scripts stored in a markup language document, in a single file dedicated to the program in question, or in multiple coordinated files, e.g., files that store one or more modules, sub-programs, or portions of code. A computer program can be deployed to be executed on one computer or on multiple computers that are located at one site or distributed across multiple sites and interconnected by a data communication network.

[0142] In this specification the term “engine” is used broadly to refer to a software-based system, subsystem, or process that is programmed to perform one or more specific functions. Generally, an engine will be implemented as one or more software modules or components, installed on one or more computers in one or more locations. In some cases, one or more computers will be dedicated to a particular engine; in other cases, multiple engines can be installed and running on the same computer or computers.

[0143] The processes and logic flows described in this specification can be performed by one or more programmable computers executing one or more computer programs to perfonn functions by operating on input data and generating output. The processes and logic flows can also be performed by special purpose logic circuitry7, e.g., an FPGA or an ASIC, or by a combination of special purpose logic circuitry and one or more programmed computers.10144] Computers suitable for the execution of a computer program can be based on general or special purpose microprocessors or both, or any other kind of central processing unit. Generally, a central processing unit will receive instructions and data from a read-only memory or a random access memory or both. The essential elements of a computer are a central processing unit for performing or executing instructions and one or more memory' devices for storing instructions and data. The central processing unit and the memory can be supplemented by, or incorporated in, special purpose logic circuitry. Generally, a computer will also include, or be operatively coupled to receive data from or transfer data to, or both, one or more mass storage devices for storing data, e.g., magnetic, magneto-optical disks, or optical disks. However, a computer need not have such devices. Moreover, a computer can be embedded in another device, e.g., a mobile telephone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a Global Positioning System (GPS) receiver, or a portable storage device, e.g., a universal serial bus (USB) flash drive, to name just a few.[01451 Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media and memory7devices, including by way ofexample semiconductor memory' devices, e.g., EPROM, EEPROM, and flash memory' devices; magnetic disks, e.g., internal hard disks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks.101461 To provide for interaction with a user, embodiments of the subject matter described in this specification can be implemented on a computer having a display device, e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor, for displaying information to the user and a keyboard and a pointing device, e.g., a mouse or a trackball, by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback, e.g., visual feedback, auditory' feedback, or tactile feedback; and input from the user can be received in any form, including acoustic, speech, or tactile input. In addition, a computer can interact with a user by sending documents to and receiving documents from a device that is used by the user; for example, by sending web pages to a web browser on a user’s device in response to requests received from the web browser. Also, a computer can interact with a user by sending text messages or other forms of message to a personal device, e.g., a smartphone that is running a messaging application, and receiving responsive messages from the user in return.

[0017] Data processing apparatus for implementing machine learning models can also include, for example, special-purpose hardware accelerator units for processing common and computeintensive parts of machine learning training or production, i.e., inference, workloads.|0148] Machine learning models can be implemented and deployed using a machine learning framework, e.g., a TensorFlow framework, or a Jax framework.

[0149] Embodiments of the subject matter described in this specification can be implemented in a computing system that includes a back-end component, e.g., as a data server, or that includes a middleware component, e.g., an application sen- er, or that includes a front-end component, e.g., a client computer having a graphical user interface, a web browser, or an app through which a user can interact with an implementation of the subject matter described in this specification, or any combination of one or more such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication, e.g., a communication network. Examples of communication networks include a local area network (LAN) and a wide area network (WAN), e.g., the Internet.

[0150] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respectivecomputers and having a client-server relationship to each other. In some embodiments, a server transmits data, e.g., an HTML page, to a user device, e.g., for purposes of displaying data to and receiving user input from a user interacting with the device, which acts as a client. Data generated at the user device, e.g., a result of the user interaction, can be received at the server from the device.

[0151] While this specification contains many specific implementation details, these should not be construed as limitations on the scope of any invention or on the scope of what may be claimed, but rather as descriptions of features that may be specific to particular embodiments of particular inventions. Certain features that are described in this specification in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination. Moreover, although features may be described above as acting in certain combinations and even initially be claimed as such, one or more features from a claimed combination can in some cases be excised from the combination, and the claimed combination may be directed to a subcombination or variation of a subcombination.

[0152] Similarly, while operations are depicted in the drawings and recited in the claims in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. In certain circumstances, multitasking and parallel processing may be advantageous. Moreover, the separation of various system modules and components in the embodiments described above should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0153] Particular embodiments of the subject matter have been described. Other embodiments are within the scope of the following claims. For example, the actions recited in the claims can be performed in a different order and still achieve desirable results. As one example, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some cases, multitasking and parallel processing may be advantageous.[01541

Claims

CLAIMS1. A method performed by one or more computers, the method comprising: determining a respective importance score for each mRNA codon in a set of possible mRNA codons, wherein the importance score for an mRNA codon characterizes an impact of a presence of the mRNA codon in an mRNA sequence on a predicted property of the mRNA sequence; training a machine learning model that is configured to process a model input that characterizes an input messenger ribonucleic acid (mRNA) sequence to generate a model output that comprises a predicted score characterizing the predicted property of the input mRNA sequence; wherein the training comprises, at each of a plurality of training iterations in a sequence of training iterations: selecting a current batch of mRNA sequences for training the machine learning model at the training iteration using the importance scores for the mRNA codons; and training the machine learning model on at least the current batch of mRNA sequences; and outputting the trained machine learning model.

2. The method of claim 1. wherein determining the respective importance score for each mRNA codon in the set of possible mRNA codons comprises: training a regression model to process a plurality of features that includes, for each mRNA codon in the set of mRNA codons, a respective feature based on a number of occurrences of the mRNA codon in an mRNA sequence to generate a predicted score characterizing the predicted property of the mRNA sequence; and determining the respective importance score for each mRNA codon in the set of possible mRNA codons using the regression model.

3. The method of claim 2, wherein the regression model comprises a respective regression coefficient corresponding to each mRNA codon in the set of mRNA codons; and wherein determining the importance score for each mRNA codon in the set of possible mRNA codons using the regression model comprises: determining the importance score for each mRNA codon based on a value of the regression coefficient corresponding to the mRNA codon.

4. The method of any one of claims 2-3, wherein the regression model is configured to generate the predicted score characterizing the predicted property of the mRNA sequence as an affine function of the plurality of features.

5. The method of any one of claims 2-4, wherein the regression model is trained to optimize an objective function.

6. The method of any preceding claim, wherein at each of the pl urality of training iterations in the sequence of training iterations, selecting the current batch of mRNA sequences for training the machine learning model at the training iteration comprises, for one or more mRNA sequences in the current batch of mRNA sequences: determining, for each position in the mRNA sequence, a set of eligible mRNA codons for the position, wherein an mRNA codon is designated as being eligible at a position if the mRNA codon codes for an amino acid in a corresponding position of an amino acid sequence of a target protein to be coded for by the mRNA sequence; identifying a plurality of positions in the mRNA sequence as being optimal positions in the mRNA sequence; and selecting, for each position designated as an optimal position in the mRNA sequence, an mRNA codon for the position as a codon having a highest importance score from among the set of eligible mRNA codons for the position.

7. The method of claim 6, further comprising selecting, for each position that is not designated as an optimal position in the mRNA sequence, an mRNA codon for the position other than a codon having a highest importance score from among the set of eligible mRNA codons for the position.

8. The method of claim 7, wherein for each position that is not designated as an optimal position in the mRNA sequence, selecting the mRNA codon for the position other than the codon having the highest importance score from among the set of eligible mRNA codons for the position comprises: stochastically selecting the mRNA codon from among the set of eligible codons for the position other than the codon having the highest importance score from among the set of eligible mRNA codons for the position.

9. The method of any one of claims 6-8, wherein identifying a plurality' of positions in the mRNA sequence as being optimal positions comprises:stochastically selecting a proper subset of the positions in the mRNA sequence as being optimal positions in the mRNA sequence.

10. The method of claim 9, wherein randomly selecting a proper subset of the positions in the mRNA sequence as being optimal positions in the mRNA sequence comprises, for each position in the mRNA sequence: selecting the position in the mRNA sequence as being an optimal position with a predefined probability.

11. The method of any preceding claim, wherein selecting a current batch of mRNA sequences for training the machine learning model at the training iteration using the importance scores for the mRNA codons comprises: generating a pool of mRNA sequences using the importance scores for the mRNA codons; and selecting a plurality of mRNA sequences from the pool of mRNA sequences for inclusion in the current batch of mRNA sequences for training the machine learning model at the training iteration.

12. The method of claim 11, wherein selecting a plurality of mRNA sequences from the pool of mRNA sequences for inclusion in the current batch of mRNA sequences for training the machine learning model at the training iteration comprises: generating a set of candidate batches of mRNA sequences, wherein each candidate batch of mRNA sequences is a proper subset of the pool of mRNA sequences; generating, for each candidate batch mRNA sequences, a respective score for the candidate batch of mRNA sequences that characterizes:(i) an uncertainty of the machine learning model in generating predicted scores for the mRNA sequences in the candidate batch of mRNA sequences, and(ii) a diversity of the mRNA sequences in the candidate batch of mRNA sequences; and selecting the current batch of mRNA sequences from the set of candidate batches of mRNA sequences based on the scores.

13. The method of any preceding claim, wherein the training comprises, at each of the plurality of training iterations in the sequence of training iterations: obtaining a respective target score for each mRNA sequence in the current batch of mRNA sequences, wherein a target score for an mRNA sequence defines a score that should begenerated by the machine learning model by processing a model input characterizing the mRNA sequence; and wherein training the machine learning model on at least the current batch of mRNA sequences comprises: training the machine learning model on at least the current batch of model inputs using the target scores for the current batch of mRNA sequences.

14. The method of claim 13, wherein at each of a plurality of training iterations, the target score for each mRNA sequence in the current batch of mRNA sequences is generated by operations comprising: physically generating one or more instances of the mRNA sequence; and determining one or more properties of the instances of the mRNA sequence; and determining the target score for the mRNA sequence based on the properties of the instances of the mRNA sequence.

15. The method of any preceding claim, wherein the predicted property of the mRNA sequence characterizes a predicted level of protein expression from the mRNA sequence.

16. The method of any preceding claim, wherein the predicted property of the mRNA sequence characterizes a predicted stability of the mRNA sequence.

17. The method of any preceding claim, wherein the predicted property' of the mRNA sequence characterizes a predicted translation efficiency of the mRNA sequence.

18. The method of any preceding claim, wherein the machine learning model comprises one or more of: a regression model, or a neural network model, or a random forest model, or a decision tree model, or a support vector machine model.

19. The method of any preceding claim, further comprising: selecting an mRNA sequence using the machine learning model; and determining that the selected mRNA sequence should be included in an mRNA therapeutic agent.

20. The method of claim 19, further comprising: determining that a subject should receive the mRNA therapeutic agent that includes the selected mRNA sequence.

21. The method of any one of claims 19-20, further comprising: physically synthesizing the mRNA therapeutic agent that includes the selected mRNA sequence.

22. The method of claim 21, further comprising: administering the mRNA therapeutic agent that includes the selected mRNA sequence to a subject.

23. The method of any preceding claim, wherein outputting the trained machine learning model comprises: selecting an mRNA sequence using the machine learning model; and physically synthesizing the selected mRNA sequence.

24. The method of claim 23, further comprising: performing experiments to determine one or more properties of the physically synthesized mRNA sequence.

25. A method performed by one or more computers, the method comprising: obtaining a model input that characterizes an input messenger ribonucleic acid (mRNA) sequence; processing the model input using a machine learning model to generate a model output that comprises a predicted score characterizing a predicted property' of the input mRNA sequence, wherein the machine learning model has been trained by the method of any one of claims 1-18.

26. A vaccine comprising one or more mRNA sequences that have been selected using a machine learning model that is configured to process a model input that characterizes an input messenger ribonucleic acid (mRNA) sequence to generate a model output that comprises a predicted score characterizing a predicted property of the input mRNA sequence, wherein the machine learning model has been trained by the method of any one of claims 1-18.

27. A method performed by one or more computers, the method comprising: determining a set of importance scores for a set of possible mRNA codons, wherein each of the set of importance scores characterizes an impact of a presence of one or more mRNA codons in the set of possible mRNA codons in an mRNA sequence on a predicted property of the mRNA sequence;generating a plurality of candidate mRNA sequences based on the set of importance scores, the generating comprising, for each position in a first candidate mRNA sequence: determining a respective set of eligible mRNA codons for the respective position, wherein an mRNA codon is designated as being eligible at a position if the mRNA codon codes for an amino acid in a corresponding position of an amino acid sequence of a target protein to be coded for by the first candidate mRNA sequence; for each eligible mRNA codon in the respective set of eligible mRNA codons, determining a respective selection probability for the eligible mRNA codon based at least on one or more importance scores in the set of importance scores; and stochastically sampling, from the respective set of eligible mRNA codons, a respective mRNA codon for the respective position according to the selection probabilities determined for the respective set of eligible mRNA codons; for each of the candidate mRNA sequences, processing a respective input characterizing the respective candidate mRNA sequence using a machine learning model to generate a respective model output comprising a respective predicted score characterizing the predicted property; and selecting one or more target mRNA sequences from the plurality of candidate mRNA sequences based on the predicted scores.

28. The method of 27, wherein the set of importance scores comprise, for each mRNA codon in the set of possible mRNA codons, a respective single-codon importance score that characterizes an impact of the presence of the respective mRNA codon in the mRNA sequence on the predicted property7of the mRNA sequence.

29. The method of 28, wherein determining the respective selection probability for the eligible mRNA codon comprises: computing the selection probability for the eligible mRNA codon as the single-codon importance score for the eligible mRNA codon normalized a sum of the single-codon importance scores of the respective set of eligible mRNA codons for the respective position.

30. The method of claim 27, wherein the set of importance scores comprise, for each two-codon subsequence formed by two mRNA codons selected from the set of possible mRNA codons, a respective two-codon importance score that characterizes an impact of the presence of therespective two-codon subsequence in the mRNA sequence on the predicted property of the mRNA sequence.

31. The method of claim 30, wherein determining the respective selection probability for the eligible mRNA codon comprises, for a current position after the first position in the first candidate mRNA sequence: computing the selection probability’ for the eligible mRNA codon as (i) the two-codon importance score for a two-codon subsequence that includes the mRNA codon that has been selected for the preceding position followed by the eligible mRNA codon normalized by (ii) a sum of the two-codon importance scores of a set of two-codon subsequences, wherein each of the set of two-codon subsequences includes the mRNA codon that has been selected for the preceding position followed by one of the respective set of eligible mRNA codons determined for the current position.

32. A system comprising: one or more computers; and one or more storage devices communicatively coupled to the one or more computers, wherein the one or more storage devices store instructions that, when executed by the one or more computers, cause the one or more computers to perform operations of the respective method of any one of claims 1-20, 25, and 27-31.

33. One or more non-transitory computer storage media storing instructions that when executed by one or more computers cause the one or more computers to perform operations of the respective method of any one of claims 1-20, 25, and 27-31.

Images