Protein sequence generation using machine learning methods

Abstract

The amino acid sequence of an antibody can be generated using a generative allele network including a first generator that generates the amino acid sequence of an antibody light chain and a second generator that generates the amino acid sequence of an antibody heavy chain. The amino acid sequence of an antibody can be generated by combining the amino acid sequences generated by the first generator and the second generator. The training of the first generator and the second generator can proceed at different speeds. Furthermore, the antibody amino acid sequences generated by combining the amino acid sequences from the first generator and the second generator can be evaluated according to the integrity-determining regions of the antibody amino acid sequence. A training dataset can be generated using amino acid sequences corresponding to antibodies with specific binding affinities for molecules, such as binding affinities for major histocompatibility complex (MHC) molecules.

Description

[Background technology]

[0001] A protein is a biological molecule that contains one or more chains of amino acids. Proteins can have various functions in an organism. For example, some proteins are involved in biological reactions. In other instances, proteins can transport molecules within an organism. In yet other instances, proteins can be involved in gene replication. Additionally, some proteins have therapeutic properties and can be used to treat various biological conditions. The structure and function of a protein are based on the arrangement of amino acids that make up the protein. The arrangement of amino acids in a protein can be represented by a string of letters, where each letter corresponds to an amino acid at a particular position. The arrangement of amino acids in a protein can also be represented by the tertiary structure of the protein, such as alpha helix or beta sheet, in addition to showing the amino acids at a particular position in the protein. Summary of the Invention

[0002] The present disclosure is by way of example and not limitation in the figures of the accompanying drawings in which like reference numbers refer to similar elements and in which: [Brief description of the drawings]

[0003] [Figure 1] FIG. 1 illustrates an exemplary framework for generating a protein sequence according to an embodiment.

[0004] [Diagram 2] FIG. 1 illustrates an exemplary framework including encoding and decoding elements for generating a protein sequence, according to an embodiment.

[0005] [Diagram 3] FIG. 1 illustrates an exemplary framework including encoding and decoding elements for generating a protein sequence, according to an embodiment.

[0006] [Figure 4]FIG. 1 illustrates an exemplary framework for generating protein sequences using a first set of training data comprising a first set of features and a second set of training data comprising a second, different set of features, according to an embodiment.

[0007] [Diagram 5] FIG. 1 shows an exemplary framework for generating antibody sequences that are variants of a parent antibody, according to certain embodiments.

[0008] [Figure 6] FIG. 1 shows an exemplary framework for generating amino acid sequences of antibodies that bind to specific antigens, according to certain embodiments.

[0009] [Figure 7] FIG. 1 illustrates an exemplary framework for generating multiple libraries of proteins and combining the libraries of proteins to generate additional proteins, according to an embodiment.

[0010] [Figure 8] FIG. 13 shows a further exemplary framework for generating the amino acid sequence of an antibody using paired amino acid sequences of antibody heavy and light chains, according to certain embodiments.

[0011] [Figure 9] 1 is a diagram illustrating a framework for generating antibody paired amino acid sequences from antibody heavy and light chain amino acid sequences using transfer learning techniques, according to an embodiment.

[0012] [Figure 10] FIG. 1 shows a framework for linking the amino acid sequences of heavy and light chains of an antibody according to certain embodiments.

[0013] [Figure 11] FIG. 1 is a flow diagram illustrating an exemplary method for generating a protein sequence, according to an embodiment.

[0014] [Figure 12] FIG. 1 is a flow diagram illustrating another exemplary method for generating a protein sequence, according to an embodiment.

[0015] [Figure 13] 1 is a flow chart illustrating an exemplary method of generating an amino acid sequence for a protein that binds to a specific target molecule, according to an embodiment.

[0016] [Figure 14] FIG. 1 is a flow diagram showing an exemplary method of combining antibody heavy and light chain amino acid sequences to generate an antibody amino acid sequence, according to certain embodiments.

[0017] [Figure 15] 1 is an example of a scheme for structurally aligning an antibody's amino acid sequence prior to encoding it for input into a generative allele network, according to an embodiment.

[0018] [Figure 16] FIG. 1 is a schematic diagram of a machine, in the form of a computer system, that, when executed, causes the machine to perform any one or more of the methods described herein. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0019] Proteins can have many beneficial uses within an organism. In certain circumstances, proteins can be used to treat diseases and other biological conditions that can adversely affect the health of humans and other mammals. In various scenarios, proteins can be involved in reactions that can be beneficial to a subject and counteract one or more biological conditions that the subject is experiencing. In some instances, proteins can also bind to target molecules within an organism that may be harmful to the health of the subject. For this reason, many individuals and organizations have pursued the development of proteins with therapeutic effects.

[0020] Protein development is a time-consuming and resource-intensive process. Often, candidate proteins for development can be identified as potentially having desired biophysical properties, three-dimensional (3D) structure, and / or behavior in an organism. To determine whether a candidate protein actually has the desired properties, the protein can be synthesized and then tested to determine whether the actual properties of the synthesized protein correspond to the desired properties. Due to the amount of resources required to synthesize a protein and analyze its specific biophysical properties, tertiary structure, and / or behavior, the number of candidate proteins that can be synthesized for therapeutic purposes is limited. If a candidate protein that does not have the desired properties is synthesized, the loss of resources may limit the number of proteins that can be synthesized for therapeutic purposes.

[0021] Computer-implemented techniques are increasingly being used to identify candidate proteins with specific properties. However, these conventional techniques may be limited in scope and accuracy. In various situations, conventional computer-implemented techniques for generating protein sequences may be limited by the amount of available data and / or the type of available data required for the conventional techniques to accurately generate protein sequences with specific properties. Furthermore, the techniques used to generate models that can generate protein sequences with specific properties may be complex, and the know-how required to generate accurate and efficient models may be complex. In some scenarios, the length of protein sequences generated by conventional models may also be limited, because the accuracy of conventional techniques may decrease as the protein length increases. Thus, there is a limit to the number of proteins that can be generated by conventional techniques.

[0022] The techniques and systems described herein can be used to accurately and efficiently generate amino acid sequences for proteins. In some embodiments, a generative allele network can be implemented to determine a model capable of generating amino acid sequences for proteins. The generative allele network can be trained with multiple different training data sets to generate amino acid sequences for proteins with specific properties. For example, the generative allele network described herein can generate amino acid sequences for proteins with specific biophysical properties. In other examples, the generative allele network described herein can generate amino acid sequences with specific structures. Additionally, the techniques and systems described herein can utilize a computer-implemented process to analyze the amino acid sequences generated by the generative allele network. Analysis of the amino acid sequences can determine whether the properties of the amino acid sequences generated by the generative allele network correspond to a set of desired properties. In some embodiments, the computer-implemented process can filter the amino acid sequences generated by the generative allele network to identify amino acid sequences that correspond to a specific set of properties.

[0023] In a further example, one or more embodiments described herein may include an autoencoder architecture capable of generating protein sequences. In one or more examples, a variational autoencoder may be used to generate protein sequences. In various examples, a variational autoencoder may be implemented to generate amino acid sequences of an antibody. In one or more embodiments, the generative machine learning architecture may include at least one encoder and a decoder that optimize a loss function to generate a model that generates amino acid sequences corresponding to sequences of proteins. After initial training of the model, the model may be further modified by training the model with data corresponding to amino acid sequences of proteins with a particular set of properties, such as one or more particular biophysical properties.

[0024] Additionally, the techniques and systems described herein can be used to generate amino acid sequences for antibodies that have at least a threshold probability of binding to a particular antigen. In these scenarios, the amino acid sequences can be generated based on antibody-antigen interaction data that indicates the interaction between the antibody and the antigen. For example, the antibody-antigen interaction data can indicate the antigen-binding region of the antibody and the corresponding epitope of the antigen that is bound to the antigen-binding region of the antibody.

[0025] Additionally, the techniques and systems described herein can be used to generate antibody amino acid sequences using antibody heavy chain amino acid sequences and antibody light chain amino acid sequences that have been generated separately and combined. In various embodiments, a generator allele network using two generators (one for the heavy chain amino acid sequence and one for the light chain amino acid sequence) can be used to generate the heavy and light chain amino acid sequences separately, which can then be combined to generate an antibody sequence comprising both a heavy and light chain. The implementation of separate generators for generating the light and heavy chain amino acid sequences improves the efficiency of the generator allele network and minimizes the computational resources utilized to generate the antibody amino acid sequences associated with a generator allele network implementing a single generator for generating both the heavy and light chain amino acid sequences of the antibody. That is, fewer computational resources are utilized to generate a large number of antibody sequences from a combination of light and heavy chains generated separately than the same number of antibody sequences generated as the amino acid sequences of both the light and heavy chains are first generated together. Furthermore, as described herein, the overall resources utilized to chemically synthesize libraries of light chains and libraries of heavy chains that can be combined to generate large numbers of antibodies based on mechanically generated light and heavy chain sequences are less than techniques that simply chemically synthesize antibodies that already comprise both heavy and light chains.

[0026] FIG. 1 illustrates an exemplary framework 100 for generating protein sequences, according to some embodiments. Framework 100 may include a generative machine learning architecture 102. Generative machine learning architecture 102 may include a sequence generation component 104. Sequence generation component 104 may implement a model to generate amino acid sequences based on input provided to sequence generation component 104. For example, sequence generation component 104 may generate generated sequences 106. Generated sequences 106 may include amino acid sequences of a protein. In one or more examples, generated sequences 106 may include amino acid sequences of an antibody. In various embodiments, the model implemented by sequence generation component 104 may include one or more functions.

[0027] In various embodiments, the generative machine learning architecture 102 may implement one or more neural network techniques. For example, the machine learning architecture 102 may implement one or more recurrent neural networks. Additionally, the machine learning architecture 102 may implement one or more convolutional neural networks. In certain embodiments, the machine learning architecture 102 may implement a combination of recurrent and convolutional neural networks. In an example, the machine learning architecture 102 may include a generative adversarial network. In such a situation, the sequence generation element 104 may include a generator, and the generative machine learning architecture 102 may also include a challenging element. In a further example, the generative machine learning architecture 102 may include an autoencoder. In one or more exemplary embodiments, the generative machine learning architecture 102 may include a variational autoencoder. In these scenarios, the sequence generation element 104 may include at least one of an encoder or a decoder of the variational autoencoder.

[0028] The generated sequences 106 can be generated by the sequence generation component 104 based on input data 108. In one or more examples, the input data 108 can include one or more amino acid sequences, such as a template protein sequence. The input data 108 can also include an input vector that includes computer-generated noise generated by a random or pseudorandom noise generator.

[0029] The generated sequences 106 may be evaluated against training sequences 110. The training sequences 110 may correspond to amino acid sequences obtained from protein sequence data 112. The protein sequence data 112 may include sequences of proteins obtained from one or more data sources that store protein amino acid sequences. The protein sequence data 112 may include amino acid sequences of one or more proteins, such as fibronectin type III (FNIII) proteins, avimers, antibodies, VHH domains, kinases, zinc fingers, etc.

[0030] Protein sequences included in the protein sequence data 112 may undergo data preprocessing 114 before being provided to the generative machine learning architecture 102. In an embodiment, the protein sequence data 112 may be arranged through a classification system by data preprocessing 114 before being provided to the generative machine learning architecture 102. Data preprocessing 114 may include pairing amino acids included in the proteins of the protein sequence data 112 with numerical values ​​that may represent a structure-based position within the protein. The numerical values ​​may include a series of numbers with a start and an end. In an illustrative example, a T may be paired with the number 43, indicating that a threonine molecule is located at structure-based position 43 in a particular protein domain type.

[0031] In various embodiments, the classification system implemented by data preprocessing 114 may designate a certain number of positions for a particular region of a protein. For example, the classification system may indicate that there are a certain number of positions in a portion of a protein that may have a certain function and / or property. In various circumstances, not all of the positions included in the classification system may be associated with an amino acid, since the number of amino acids in a particular region of a protein may vary between proteins. For example, the number of amino acids in a region of a protein may vary between different types of proteins. In other examples, it may reflect the structure of the protein. In examples, the positions in the classification system that are not associated with a particular amino acid may indicate various structural characteristics of the protein, such as turns or loops. In an illustrative example, the classification system for an antibody may indicate a heavy chain region. The light chain region and hinge region have a certain number of positions assigned to them, and the amino acids of the antibody may be assigned to positions according to the classification system.

[0032] In an embodiment, data included in the protein sequence data 112 used to train the generative machine learning architecture 102 may affect the amino acid sequences generated by the sequence generation component 104. For example, the properties, biophysical properties, manufacturing properties (e.g., potency, yield, etc.) of the protein sequence data 112 may affect the properties, biophysical properties, and / or manufacturing properties of the sequences 106 generated by the sequence generation component 104. By way of illustration, if the protein sequence data 112 provided to the generative machine learning architecture 102 includes an antibody, the amino acid sequence generated by the sequence generation component 104 may correspond to the amino acid sequence of the antibody. In another example, if the protein sequence data 112 provided to the generative machine learning architecture 102 includes a T cell receptor, the amino acid sequence generated by the sequence generation component 104 may correspond to the amino acid sequence of the T cell receptor. In a further example, if the protein sequence data 112 provided to the generative machine learning architecture 102 includes a kinase, the amino acid sequence generated by the sequence generation component 104 may correspond to the amino acid sequence of the kinase. In embodiments in which the protein sequence data 112 provided to the generative machine learning architecture 102 includes amino acid sequences for a variety of different types of proteins, the sequence generation component 104 may generate amino acid sequences that are generally characteristic of proteins, but may not correspond to a particular type of protein.

[0033] The output generated by the data preprocessing 114 may include a structured sequence 116. The structured sequence 116 may include a matrix that indicates the amino acids associated with various positions of the protein. In an example, the structured sequence 116 may include a matrix with columns corresponding to different amino acids and rows corresponding to the positions of the structural bases of the protein. For each element in the matrix, a 0 may be used to indicate that the amino acid is not present at the corresponding position, and a 1 may be used to indicate that the amino acid is present at the corresponding position. If a position represents a gap in the amino acid sequence, the row associated with that position may include zeros for each column. The generated sequence 106 may also be represented using vectors with the same or similar number scheme as used for the structured sequence 116. In some illustrative examples, the structured sequence 116 and the generated sequence 106 may be encoded using what is also known as one-hot encoding.

[0034] The generative machine learning architecture 102 can analyze the generated sequences 106 with respect to the training sequences 110 to evaluate a loss function 118 of the generative machine learning architecture 102. In one or more examples, the output of the loss function 118 can be used to modify the sequences generated by the sequence generation element 104. For example, an output related to the loss function 118 can be used to modify one or more elements of the generative machine learning architecture 102, such as an encoder, a decoder, and / or a generator of the GAN, to generate a generated sequence 106 that is closer to the training sequence 110. In one or more examples, the elements of the generative machine learning architecture 102 can be modified to optimize the loss function 118. In various examples, the elements of the generative machine learning architecture 102 can be modified to minimize the loss function 118.

[0035] After the generative machine learning architecture 102 has undergone a training process, a trained model 120 may be generated to generate sequences of proteins. The trained model 120 may include one or more elements of the generative machine learning architecture 102 after the training process using the protein sequence data 112. In one or more embodiments, the trained model 120 may include a generator of a GAN trained using the protein sequence data 112. Additionally, the trained model 120 may include at least one of an encoder or a decoder of an autoencoder trained using the protein sequence data 112. In an example, the training process for the generative machine learning architecture 102 may be completed after a function implemented by one or more elements of the generative machine learning architecture 102 has converged. As protein sequences are generated by the sequence generation elements 104, the function may converge based on moving values ​​of model parameters to specific values ​​such that feedback is obtained for the loss function 118 based on the dissimilarity between the training sequences 110 and the generated sequences 106.

[0036] In various embodiments, training of the generative machine learning architecture 102 can be completed when the protein sequences generated by the sequence generation component 104 have certain properties. Illustratively, the amino acid sequences generated by the sequence generation component 104 can be analyzed by a software tool that can analyze the amino acid sequences to determine at least one of biophysical properties of the amino acid sequences, structural properties of the amino acid sequences, or adhesion to amino acid sequences corresponding to one or more protein germlines. As used herein, germline can correspond to an amino acid sequence of a protein that is conserved when a cell of the protein replicates. An amino acid sequence can be conserved from a parent cell to a progeny cell if the amino acid sequence of the progeny cell has at least a threshold amount of identity with respect to the corresponding amino acid sequence in the parent cell. In an illustrative example, the portion of the amino acid sequence of a human antibody that is a portion of the kappa light chain that is conserved from a parent cell to a progeny cell can be the germline portion of the antibody.

[0037] A sequence input 122 can be provided to the training model 120, which can generate a sequence 124. The sequence input 122 can correspond to a random or pseudo-random sequence of numbers that can be used to generate the sequence 124, which includes the amino acid sequence of a protein. The sequence 124 generated by the trained model 120 can be represented as a matrix structure identical or similar to the matrix structure used to represent the structured sequence 116 and the generated sequence 106. In various embodiments, the matrix generated by the trained model 120, which includes the sequence 124, can be decoded to generate a set of amino acids that correspond to the sequence of the protein. In operation 126, the sequence 124 can be evaluated to determine whether the sequence 124 has a particular set of characteristics. The sequence evaluation performed in operation 126 can generate an index 128 indicative of the characteristics of the sequence 124. Furthermore, the index 128 can indicate the amount of correspondence between the characteristics of the sequence 124 and a particular set of characteristics. In some instances, index 128 may indicate multiple positions in amino acid sequence 124 that differ from the amino acid sequence of a protein produced from a germline gene.

[0038] The sequence 124 generated by the model 120 may correspond to various types of proteins. For example, the sequence 124 may correspond to a protein that functions as a T cell receptor. In a further example, the sequence 124 may correspond to a protein that functions as a catalyst for biochemical reactions in vivo. The sequence 124 may also correspond to one or more types of antibodies. By way of example, the sequence 124 may correspond to one or more antibody subtypes, such as immunoglobin A (IgA), immunoglobin D (IgD), immunoglobin E (IgE), immunoglobin G (IgG), or immunoglobin M (IgM). Furthermore, the sequence 124 may correspond to additional proteins that bind to antigens. In examples, the sequence 124 may correspond to an affibody, an affilin, an affimer, an affitin, an alphabody, an anchorin, an avimer, a monobody, a designed ankyrin repeat protein (DARPins), a nanoCLAMP (clostridal antibody mimicking protein), an antibody fragment, or a combination thereof. In yet another example, sequence 124 may correspond to an amino acid sequence involved in protein-protein interactions, such as a protein that has a region that binds to an antigen or a region that binds to another molecule.

[0039] In some embodiments, the sequences 124 may undergo sequence filtering in operation 130 to generate one or more filtered sequences 132. The sequence filtering 130 may analyze the sequences 124 for one or more sequences 124 that correspond to one or more properties. For example, the sequence filtering in operation 130 may analyze the sequences 124 to identify sequences 124 with particular amino acids at particular positions. The sequence filtering 130 may also identify one or more of the sequences 124 with one or more particular strings of amino acids. In various embodiments, the sequence filtering in operation 130 may identify one or more of the sequences 124 with a set of biophysical properties based on a similarity between at least one of the sequences 124 and an amino acid sequence of a protein with the set of biophysical properties.

[0040] FIG. 2 illustrates an exemplary framework 200 including an encoding element and a decoding element for generating protein sequences, according to an embodiment. The framework 200 may include a generative machine learning architecture 202. The generative machine learning architecture 202 may correspond to an implementation of an autoencoder and includes an encoding element 204 and a decoding element 206. The encoding element 204 may determine an encoding of an input amino acid sequence, which may be decoded by the decoding element 206 to generate one or more additional amino acid sequences. In various examples, an input sample 208 may be provided to the decoding element 206, which may use the input sample 208 and the encoding to generate a generated sequence 210. The generated sequence 210 may be analyzed with respect to a training sequence 212, and a loss function 214 may be optimized based on the dissimilarity between the generated sequence 210 and the training sequence 212. In one or more examples, the output of the loss function 214 may be used to modify the sequence generated by the decoding element 206. In one or more examples, at least one of the encoding element 204 or the decryption element 206 may be modified to optimize the loss function 214. In various examples, at least one of the encoding element 204 or the decryption element 206 may be modified to minimize the loss function 214.

[0041] The generated sequence 210 may include an amino acid sequence of a protein. In one or more examples, the generated sequence 210 may include an amino acid sequence of an antibody. In various embodiments, the deciphering component 206 may implement a model to generate the generated sequence 210. In various examples, the model implemented by the deciphering component 206 may include one or more functions.

[0042] The training sequences 212 may correspond to amino acid sequences obtained from protein sequence data 214. The protein sequence data 214 may include the amino acid sequences of one or more proteins, such as fibronectin type III (FNIII) proteins, avimers, antibodies, VHH domains, kinases, zinc fingers, and the like.

[0043] The protein sequences included in the protein sequence data 214 may undergo data preprocessing 216 before being provided to the generative machine learning architecture 202. In an embodiment, the data preprocessing 216 may arrange the protein sequence data 214 according to a classification system before being provided to the generative machine learning architecture 202. The data preprocessing 216 may include pairing the amino acids included in the proteins of the protein sequence data 214 with a numerical value that may represent a structure-based position within the protein. The numerical value may include a series of numbers with a start and an end. In an illustrative example, a T may be paired with the number 43, indicating that a threonine molecule is located at structure-based position 43 in a particular protein domain type.

[0044] In various embodiments, the classification system implemented by data preprocessing 216 may designate a certain number of positions for a particular region of a protein. For example, the classification system may indicate that there are a certain number of positions in a portion of a protein that may have a certain function and / or property. In various circumstances, not all of the positions included in the classification system may be associated with an amino acid, since the number of amino acids in a particular region of a protein may vary between proteins. For example, the number of amino acids in a region of a protein may vary between different types of proteins. In other examples, the positions in the classification system that are not associated with a particular amino acid may indicate various structural characteristics of the protein, such as turns or loops. In an illustrative example, the classification system for an antibody may indicate a heavy chain region. The light chain region and hinge region have a certain number of positions assigned to them, and the amino acids of the antibody may be assigned to positions according to the classification system.

[0045] In embodiments, data included in the protein sequence data 216 used to train the generative machine learning architecture 202 can affect the amino acid sequence generated by the decode element 206. For example, the properties, biophysical properties, manufacturing properties (e.g., potency, yield, etc.) of the protein sequence data 216 can affect the properties, biophysical properties, and / or manufacturing properties of the generated sequence 210 generated by the decode element 206. By way of example, in a situation where an antibody is included in the protein sequence data 216 provided to the generative machine learning architecture 202, the amino acid sequence generated by the decode element 206 can correspond to an antibody amino acid sequence. In another example, in a scenario where a T cell receptor is included in the protein sequence data 216 provided to the generative machine learning architecture 202, the amino acid sequence generated by the decode element 206 can correspond to a T cell receptor amino acid sequence. In a further example, if a kinase is included in the protein sequence data 216 provided to the generative machine learning architecture 202, the amino acid sequence generated by the decode element 206 can correspond to the amino acid sequence of the kinase. In embodiments in which the protein sequence data 216 provided to the generative machine learning architecture 202 includes amino acid sequences for a variety of different types of proteins, the decoding component 206 may generate amino acid sequences that are generally characteristic of proteins, but may not correspond to a particular type of protein.

[0046] The output generated by data preprocessing 218 may include a structured array 220. The structured array 220 may include a matrix indicating the amino acids associated with various positions of a protein. In an example, the structured array 220 may include a matrix with columns corresponding to different amino acids and rows corresponding to the positions of structural bases of the protein. For each element in the matrix, a zero may be used to indicate the absence of an amino acid at the corresponding position, and a one may be used to indicate the presence of an amino acid at the corresponding position. If a position represents a gap in the amino acid sequence, the row associated with that position may include zeros for each column. Additionally, the generated array 210 may also be represented using vectors with the same or similar number scheme as used for the structured array 220. In some illustrative examples, the structured array 220 and the generated array 210 may be encoded using what is also known as one-hot encoding.

[0047] After the generative machine learning architecture 202 undergoes a training process, a trained model 222 is generated and can generate sequences of proteins. The trained model 222 can include one or more elements of the generative machine learning architecture 202 after a training process using protein sequence data 216. In one or more embodiments, the trained model 222 can include at least one of the encoding element 204 or the decoding element 206 trained using the protein sequence data 216. In an example, the training process for the generative machine learning architecture 202 can be completed after a function implemented by one or more elements of the generative machine learning architecture 202 has converged. Protein sequences are generated by the decoding element 206, and the function can converge based on moving values ​​of model parameters to specific values ​​such that feedback is obtained for the loss function 214 based on the dissimilarity between the training sequences 212 and the generated sequences 210.

[0048] In various embodiments, training of the generative machine learning architecture 202 can be completed if the sequence of the protein generated by the decoding element 206 comprises certain properties. Illustratively, the amino acid sequence generated by the decoding element 206 can be analyzed by a software tool that can analyze the amino acid sequence to determine at least one of biophysical properties of the amino acid sequence, structural properties of the amino acid sequence, or attachment to amino acid sequences corresponding to one or more protein germlines. As used herein, germline may correspond to an amino acid sequence of a protein that is preserved when a cell of the protein replicates. An amino acid sequence may be preserved from a parent cell to a progeny cell if the amino acid sequence of the progeny cell has at least a threshold amount of identity with respect to the corresponding amino acid sequence in the parent cell. In an illustrative example, the portion of the amino acid sequence of a human antibody that is a portion of the kappa light chain that is preserved from a parent cell to a progeny cell may be the germline portion of the antibody.

[0049] A sequence input 224 may be provided to the training model 222, which may generate a sequence 226. The sequence input 224 may correspond to a random or pseudo-random sequence of numbers that may be used to generate the sequence 226, which comprises an amino acid sequence of a protein. The sequence 226 generated by the trained model 222 may be represented as a matrix structure identical or similar to the matrix structure used to represent the structured sequence 220 and the generated sequence 210. In various embodiments, the matrix generated by the trained model 222 that constructs the sequence 226 may be decoded to generate a set of amino acids that correspond to the sequence of the protein. In operation 228, the sequence 226 may be evaluated to determine whether the sequence 226 has a particular set of characteristics. The sequence evaluation performed in operation 228 may generate an index 230 indicative of the characteristics of the sequence 226. Additionally, the index 230 may indicate an amount of correspondence between the characteristics of the sequence 226 and a particular set of characteristics. In some instances, the index value 230 may indicate a number of positions in the amino acid sequence 226 that differ from the amino acid sequence of the protein produced from a germline gene.

[0050] The sequence 226 generated by the model 222 may correspond to various types of proteins. For example, the sequence 226 may correspond to a protein that functions as a T cell receptor. In a further example, the sequence 226 may correspond to a protein that functions as a catalyst for biochemical reactions in vivo. The sequence 226 may also correspond to one or more types of antibodies. By way of example, the sequence 226 may correspond to one or more antibody subtypes, such as immunoglobin A (IgA), immunoglobin D (IgD), immunoglobin E (IgE), immunoglobin G (IgG), or immunoglobin M (IgM). Furthermore, the sequence 226 may correspond to additional proteins that bind to antigens. In examples, the sequence 226 may correspond to an affibody, an affilin, an affimer, an affitin, an alphabody, an anchorin, an avimer, a monobody, a designed ankyrin repeat protein (DARPins), a nanoCLAMP (clostridal antibody mimicking protein), an antibody fragment, or a combination thereof. In yet another example, sequence 226 may correspond to an amino acid sequence involved in protein-protein interactions, such as a protein that has a region that binds to an antigen or a region that binds to another molecule.

[0051] In some embodiments, the sequences 226 may undergo sequence filtering in operation 232 to generate one or more filtered sequences 234. The sequence filtering 232 may analyze the sequences 226 for one or more sequences 226 that correspond to one or more properties. For example, the sequence filtering in operation 232 may analyze the sequences 226 to identify sequences 226 with particular amino acids at particular positions. The sequence filtering 232 may also identify one or more of the sequences 226 with one or more particular strings of amino acids. In various embodiments, the sequence filtering in operation 232 may identify one or more of the sequences 226 with a set of biophysical properties based on a similarity between at least one of the sequences 226 and an amino acid sequence of a protein with the set of biophysical properties.

[0052] 3 is a diagram illustrating an exemplary framework 300 for generating protein sequences, including a generating element and a challenging element, according to an embodiment. The framework 300 may include a generative machine learning architecture 302. The generative machine learning architecture 302 may include a generating element 304 and a challenging element 306. The generating element 304 may implement a model to generate amino acid sequences based on inputs provided to the generating element 304. In various embodiments, the model implemented by the generating element 304 may include one or more functions. The challenging element 306 may generate an output indicating whether the amino acid sequence generated by the generating element 304 satisfies various properties. The output generated by the challenging element 306 may be provided to the generating element 304, and the model implemented by the generating element 304 may be modified based on feedback provided by the challenging element 306. In various embodiments, the challenging element 306 can compare the amino acid sequence generated by the generating element 304 with the amino acid sequence of the protein and can generate an output indicative of the amount of correspondence between the amino acid sequence generated by the generating element 304 and the amino acid sequence of the protein provided to the challenging element 306.

[0053] In various embodiments, the machine learning architecture 302 may implement one or more neural network techniques. For example, the machine learning architecture 302 may implement one or more recurrent neural networks. Additionally, the machine learning architecture 302 may implement one or more convolutional neural networks. In certain embodiments, the machine learning architecture 302 may implement a combination of recurrent and convolutional neural networks. In an example, the machine learning architecture 302 may include a generative adversarial network (GAN). In such situations, the generating element 304 may include a generator and the challenging element 306 may include a discriminator. In further embodiments, the machine learning architecture 302 may include a Wasserstein generative adversarial network (wGAN). In these scenarios, the generating element 304 may include a generating element and the classifying element 306 may include a critique.

[0054] 3, an input vector 308 may be provided to a generating element 304, which may use the model to generate one or more generated sequences 310 from the input vector 308. In particular embodiments, the input vector 308 may include noise generated by a random or pseudo-random generator. The generated sequences 310 may be compared by a challenging element 306 to sequences of proteins contained in protein sequence data 312 that are coded according to a particular schema. The protein sequence data 312 may include sequences of proteins obtained from one or more data sources that store protein sequences.

[0055] Based on the similarities and differences between the generated sequence 310 and the sequence obtained from the protein sequence data 312, the classification component 306 may generate a classification output 314 indicating the amount of similarity or the amount of difference between the generated sequence 310 and the sequence contained in the protein sequence data 312. In an example, the challenging component 306 may label the generated sequence 310 as zero and the structured sequence obtained from the protein sequence data 312 as one. In such a situation, the classification output 314 may correspond to a number between 0 and 1. In a further example, the challenging component 306 may implement a distance function that generates an output indicating an amount of distance between the generated sequence 310 and the protein contained in the protein sequence data 312. In these scenarios, the challenging component 306 may label the generated sequence 310 as -1 and the encoded amino acid sequence obtained from the protein sequence data 312 as one. In embodiments in which the challenging component 306 implements a distance function, the classification output 314 may be a number between -∞ and ∞. In some examples, the amino acid sequences obtained from the protein sequence data 312 are referred to as ground truth data.

[0056] Protein sequences included in protein sequence data 312 may be subjected to data pre-processing 316 before being provided to challenging element 306. In an embodiment, protein sequence data 312 may be arranged according to a classification system by data pre-processing 316 before being provided to challenging element 306. Data pre-processing 316 may include pairing amino acids included in proteins in protein sequence data 312 with a numerical value that may represent a structure-based position within the protein. The numerical value may include a series of numbers with a start and an end. In an illustrative example, a T may be paired with the number 43, indicating that a threonine molecule is located at structure-based position 43 in a particular protein domain type. In an illustrative example, structure-based numbering may be applied to amino acid sequences of one or more proteins, such as fibronectin type III (FNIII) proteins, avimers, antibodies, VHH domains, kinases, zinc fingers, etc.

[0057] In various embodiments, the classification system implemented by data preprocessing 316 may specify a certain number of positions for a particular region of a protein. For example, the classification system may specify that a portion of a protein with a particular function and / or property may have a certain number of positions. In various circumstances, not all of the positions included in the classification system may be associated with an amino acid, since the number of amino acids in a particular region of a protein may vary between proteins. For example, the number of amino acids in a region of a protein may vary between different types of proteins. In other examples, the positions in the classification system that are not associated with a particular amino acid may indicate various structural characteristics of the protein, such as turns or loops. In an illustrative example, a classification system for an antibody may indicate a heavy chain region. The light chain region and hinge region have a certain number of positions assigned to them, and the amino acids of the antibody may be assigned to positions according to the classification system.

[0058] In embodiments, the data used to train the machine learning architecture 302 may influence the amino acid sequences generated by the generation element 304. For example, if an antibody is included in the protein sequence data 312 provided to the challenge element 306, the amino acid sequences generated by the generation element 304 may correspond to an antibody amino acid sequence. In another example, if a T cell receptor is included in the protein sequence data 312 provided to the challenge element 306, the amino acid sequences generated by the generation element 304 may correspond to an amino acid sequence of a T cell receptor. In a further example, if the protein sequence data 312 provided to the classification element 306 includes amino acid sequences of different types of proteins, the amino acid sequences generated by the classification element 306 may correspond to an amino acid sequence of a kinase. In embodiments where the protein sequence data 312 provided to the generation machine learning architecture 102 includes amino acid sequences of a variety of different types of proteins, the generation element 304 may generate amino acid sequences with characteristics of proteins in general, but may not correspond to a particular type of protein. Additionally, in various examples, the amino acid sequences generated by the generation element 304 may correspond to the types of amino acid sequences included in the protein sequence data 312 provided to the challenge element 306.

[0059] The output generated by data preprocessing 316 may include a structured array 318. The structured array 318 may include a matrix indicating the amino acids associated with various positions of the protein. In an example, the structured array 318 may include a matrix with columns corresponding to different amino acids and rows corresponding to the positions of the structural bases of the protein. For each element in the matrix, a zero may be used to indicate the absence of an amino acid at the corresponding position, and a one may be used to indicate the presence of an amino acid at the corresponding position. If a position represents a gap in the amino acid sequence, the row associated with that position may include zeros for each column. Additionally, the generated array 310 may also be represented using vectors with the same or similar number scheme as used for the structured array 318. In some illustrative examples, the structured array 318 and the generated array 310 may be encoded using what is also known as one-hot encoding.

[0060] After the machine learning architecture 302 undergoes a training process, a trained model 320 can be generated to generate sequences of proteins. The trained model 320 can include the generator 304 after a training process using protein sequence data 312. In an example, the training process for the machine learning architecture 302 can be completed after the functions performed by the generator 304 and the functions performed by the challenger 306 converge. As protein sequences are generated by the sequence generator 304 and feedback is obtained from the challenger 306, the values ​​of the model parameters can be based on moving towards certain values. In various embodiments, the training of the machine learning architecture 302 can be completed when the protein sequences generated by the generator 304 possess certain properties. By way of example, the amino acid sequences generated by the generator 304 can be analyzed by a software tool that can analyze the amino acid sequences to determine at least one of biophysical properties of the amino acid sequences, structural properties of the amino acid sequences, or adhesion to amino acid sequences corresponding to one or more protein germlines. As used herein, germline can correspond to amino acid sequences of a protein that are conserved when cells replicate the protein. An amino acid sequence may be conserved from a parent cell to a progeny cell if the amino acid sequence of the progeny cell has at least a threshold amount of identity with respect to the corresponding amino acid sequence in the parent cell. In an illustrative example, the portion of the amino acid sequence of a human antibody that is the portion of the kappa light chain that is conserved from a parent cell to a progeny cell may be the germline portion of the antibody.

[0061] A sequence input 322 may be provided to the training model 320, which may generate a sequence 324. The sequence input 322 may correspond to a random or pseudo-random sequence of numbers that may be used to generate the sequence 324, which includes the amino acid sequence of a protein. The sequence 324 generated by the trained model 320 may be represented as a matrix structure identical or similar to the matrix structure used to represent the structured sequence 318 and the generated sequence 310. In various embodiments, the matrix generated by the trained model 320, which includes the sequence 324, may be decoded to generate a set of amino acids that correspond to the sequence of the protein. In operation 326, the sequence 324 may be evaluated to determine whether the sequence 324 has a particular set of characteristics. The sequence evaluation performed in operation 326 may generate an index 328 indicative of the characteristics of the sequence 324. Additionally, the index 328 may indicate the amount of correspondence between the characteristics of the sequence 324 and a particular set of characteristics. In some instances, index 328 may indicate multiple positions in amino acid sequence 324 that differ from the amino acid sequence of a protein made from a germline gene.

[0062] The sequence 324 generated by the model 320 may correspond to various types of proteins. For example, the sequence 324 may correspond to a protein that functions as a T cell receptor. In a further example, the sequence 324 may correspond to a protein that functions as a catalyst for biochemical reactions in vivo. The sequence 324 may also correspond to one or more types of antibodies. By way of example, the sequence 324 may correspond to one or more antibody subtypes, such as immunoglobin A (IgA), immunoglobin D (IgD), immunoglobin E (IgE), immunoglobin G (IgG), or immunoglobin M (IgM). Furthermore, the sequence 324 may correspond to additional proteins that bind to antigens. In examples, the sequence 324 may correspond to an affibody, an affilin, an affimer, an affitin, an alphabody, an anchorin, an avimer, a monobody, a designed ankyrin repeat protein (DARPins), a nanoCLAMP (clostridal antibody mimicking protein), an antibody fragment, or a combination thereof. In yet another example, sequence 324 may correspond to an amino acid sequence involved in protein-protein interactions, such as a protein that has a region that binds to an antigen or a region that binds to another molecule.

[0063] In some embodiments, the sequences 324 may be subjected to a sequence filter process 330 to generate one or more filtered sequences 332. The sequence filter process 330 may analyze the sequences 324 for one or more sequences 124 that correspond to one or more properties. For example, the sequence filter process 330 may analyze the sequences 324 to identify sequences 324 with particular amino acids at particular positions. The sequence filter 330 may also identify one or more of the sequences 324 with one or more particular strings of amino acids. In various embodiments, the sequence filter process 330 may identify one or more of the sequences 324 with a set of biophysical properties based on a similarity between at least one of the sequences 324 and an amino acid sequence of a protein with the set of biophysical properties.

[0064] FIG. 4 is a diagram illustrating an exemplary framework 400 for generating protein sequences using a first set of training data with a first set of characteristics and a second set of training data with a second set of characteristics different from the first set of characteristics, according to an embodiment. The framework 400 may include a first generating adversarial network 402. The first generating adversarial network 402 may include a first generating element 404 and a first challenging element 406. The first challenging element 406 may be a classifier. In further situations, for example, when the first generating adversarial network 402 is a Wasserstein GAN, the first challenging element 406 may include a critique. The first generating element 404 may implement a model that generates amino acid sequences based on inputs provided to the first generating element 404. The first challenging element 406 may generate an output indicating whether the amino acid sequences generated by the generating element 404 satisfy various characteristics. The output generated by the first challenging element 406 may be provided to the generation element 404, which may modify the model implemented by the first challenging element 404 based on feedback provided by the first challenging element 406. In various embodiments, the first challenging element 406 may compare the amino acid sequence generated by the first challenging element 404 to the amino acid sequence of the protein, and may generate an output indicative of the amount of correspondence between the amino acid sequence generated by the first challenging element 404 and the amino acid sequence of the protein provided to the first challenging element 406.

[0065] A first input vector 408 may be provided to a first generating element 404, which may use the model to generate one or more first generating sequences 410 using the first input vector 408. In particular embodiments, the first input vector 408 may be generated using a random or pseudo-random generator. In an exemplary embodiment, the first input vector 408 may include a noise signal that includes a series of numbers.

[0066] The first generated sequence 410 can be compared by the first challenging element 406 against sequences of proteins contained in the protein sequence data 412. The first protein sequence data 412 can include sequences of proteins obtained from one or more data sources storing protein sequences. Based on the similarities and differences between the first generated sequence 410 and sequences obtained from the protein sequence data 412, the first challenging element 406 can generate a first classification output 414 indicating an amount of similarity or an amount of difference between the first generated sequence 410 and sequences contained in the first protein sequence data 412. The first challenging element 406 can label the first protein sequence data 412 as zero and the structured sequence obtained from the first generated sequence 410 as one. In this situation, the first classification output 414 can include a number from 0 and 1 when the first generated adversarial network 402 is a Wasserstein GAN. In further examples, the first challenging element 406 may implement a distance function that generates an output indicating an amount of distance between the first generated sequence 410 and the protein contained in the first protein sequence data 412. In these scenarios, the first challenging element 406 may label the first generated sequence 410 as -1 and the encoded amino acid sequence obtained from the first protein sequence data 412 as 1. In embodiments in which the first challenging element 406 implements a distance function, the first classification output 414 may be a number between -∞ and ∞. In some examples, the amino acid sequence obtained from the first protein sequence data 412 is referred to as ground truth data.

[0067] The protein sequences included in the protein sequence data 412 may undergo a first data pre-processing 416 before being provided to the first challenging element 406. In an embodiment, the protein sequence data 412 may be arranged according to a classification system provided to the first challenging element 406. The first data pre-processing 416 may include matching amino acids included in the proteins of the protein sequence data 412 with a numerical value that may represent a structure-based position within the protein. The numerical value may include a series of numbers with a start point and an end point. The first data pre-processing 416 may generate a first structured array 418 that is provided to the first challenging element 406. The first structured array 418 may include a matrix that indicates amino acids associated with various positions of the protein. In an example, the first structured array 418 may include a matrix with columns corresponding to different amino acids and rows corresponding to structural base positions of the protein. For each element in the matrix, a 0 may be used to indicate the absence of an amino acid at the corresponding position and a 1 may be used to indicate the presence of an amino acid at the corresponding position. If a position represents a gap in the amino acid sequence, the row associated with that position may contain zeros for each column. First generated sequence 410 may also be represented using a vector with the same or similar number scheme as used for first structured sequence 418. In some illustrative examples, first structured sequence 418 and first generated sequence 410 may be encoded using what is also known as one-hot encoding.

[0068] After the first generating adversarial network 402 undergoes a training process, a trained model 420 is generated and can generate sequences of proteins. In an example, the training process for the first generating adversarial network 402 can be completed after the function implemented by the first generating element 404 has converged. In various embodiments, the training of the first generating adversarial network 402 can be completed if the protein sequences generated by the first generating element 404 have certain characteristics. By way of example, the amino acid sequences generated using the trained model 420 can be analyzed by a software tool that can analyze the amino acid sequences to determine at least one of biophysical properties of the amino acid sequences, structural properties of the amino acid sequences, or attachment to amino acid sequences corresponding to one or more amino acid sequences derived from germline genes of the protein.

[0069] A first sequence input 422 can be provided to the trained model 420, which can generate a first sequence 424. The first sequence input 422 can correspond to a random or pseudorandom sequence of numbers that can be used to generate the first sequence 424, which includes an amino acid sequence of a protein. The first sequence 424 generated by the trained model 420 can be represented as a matrix structure that is the same as or similar to the matrix structure used to represent the first structured sequence 418 and the first generated sequence 410. In various embodiments, the matrix generated by the trained model 420, which includes the first sequence 424, can be decoded to generate a set of amino acids corresponding to the sequence of the protein. In particular embodiments, the model evaluation criteria 428 can be the same as or similar to the criteria used to determine whether training of the first generated adversarial network 402 is complete. In further embodiments, the model evaluation criteria 428 can be different from the criteria used to determine whether training of the first generated adversarial network 402 is complete. In illustrative examples, the model evaluation criteria 428 may correspond to characteristics of sequences generated using the trained model 420. In these scenarios, the model evaluation 426 may include comparing the first sequence 424 to the sequence characteristics included in the model evaluation criteria 428. By way of example, the model evaluation 426 may include determining whether the first sequence 424 includes a particular amino acid sequence at a particular position. In further embodiments, the model evaluation 426 may include determining whether the first sequence 424 corresponds to an amino acid with a particular biophysical and / or tertiary structure. Additionally, the model evaluation 426 may include determining whether there is convergence with respect to the model 420. In some examples, the model evaluation 426 may include a human expert review of the first sequence 424 based on the model evaluation criteria 428.

[0070] In illustrative examples, the protein sequence data 412 may include amino acid sequences of antibodies. In these scenarios, the amino acid sequences provided to the first challenge element 406 may correspond to antibodies with a number of different characteristics. For example, the first challenge element 406 may provide amino acid sequences from antibodies of different isotypes, IgA, IgD, IgE, IgD, and / or IgM. In further examples, the amino acid sequences provided to the first challenge element 406 may relate to proteins derived from different germline genes. In further examples, the antibody amino acid sequences provided to the first challenge element 406 may be of varying lengths and / or sequences for the light chain variable region and / or the heavy chain variable region. In yet other examples, the amino acid sequences provided to the first challenge element 406 may be at least a portion of an antibody light chain region, at least a portion of an antibody heavy chain region, or a combination thereof. In yet a further example, the amino acid sequences provided in the first challenge element 406 can comprise a number of different biophysical properties of antibodies, such as amino acid sequences corresponding to hydrophobic regions, amino acid sequences corresponding to negatively charged regions, amino acid sequences corresponding to positively charged regions, or combinations thereof. Additionally, the amino acid sequences provided in the first challenge element 406 can correspond to antibodies and / or antibody regions with different solubility characteristics and / or different thermal degradation characteristics.

[0071] As a result of training the first generative adversarial network 402 on a large number of amino acid sequences of proteins with a first set of properties, in an example, the first generative adversarial network 402 may be trained with amino acid sequences with relatively common properties. In an embodiment, the first generative adversarial network 402 may be trained with a data set on the order of several thousand or more amino acid sequences. In certain illustrative examples, the first generative adversarial network 402 may be trained to generate amino acid sequences that exhibit properties corresponding to common antibodies. In these embodiments, the model evaluation 426 may determine whether the first sequence 424 corresponds to the common properties of antibodies. For example, the model evaluation criteria 428 may identify the structure of an antibody in the first sequence 424. By way of example, the model evaluation 426 may determine whether the first sequence 424 has a variable region and a constant region. The model evaluation 426 may also determine whether the first sequence 424 has a particular number or range of amino acids corresponding to a large number of amino acids in the heavy chain region and / or a particular number or range of amino acids in the light chain region. Additionally, model evaluation 426 can determine whether first sequence 424 has a hinge region that connects constant regions of the heavy chains. Additionally, model evaluation 426 can determine whether first sequence 424 has amino acids at specific positions that can form disulfide bonds.

[0072] After the model evaluation 426 determines that the trained model 420 meets one or more of the model evaluation criteria, the trained model 420 may undergo further training on other data sets. In an embodiment, the trained model 420 may be represented as being included in a second generative adversarial network 430 that includes a second generator element 432 and a second challenging element 434. In a particular embodiment, the trained model 420 may be represented by the second generator element 432. In various embodiments, the second generator element 432 may include the training model 420 after making one or more modifications to the training model 420. For example, the modifications to the training model 420 may be made in relation to the architecture of the training model 420, such as adding one or more hidden layers or making changes to one or more network filters. The second generator element 432 may obtain a second input vector 436 to generate a second generated sequence 438. In various embodiments, the second challenging element 434 may be a classifier. In further situations, for example, when the second generative adversarial network 430 is a Wasserstein GAN, the second challenging element 434 may include criticism. The second input vector 436 may include a random or pseudorandom sequence of numbers.

[0073] The second challenging element 434 may generate a second classification output 440 indicating whether the amino acid sequence generated by the second generating element 432 satisfies various characteristics. In exemplary embodiments, based on the similarities and differences between the second generated sequence 438 and the sequence provided to the second challenging element 434, e.g., the amino acid sequence included in the protein sequence data 412, the second challenging element 434 may generate a second classification output 440 indicating the amount of similarity or the amount of difference between the second generated sequence 238 and the comparison sequence provided to the second challenging element 434. The comparison sequence provided to the second challenging element 434 may correspond to an amino acid sequence included in the second protein sequence data 442. The second protein sequence data 442 may include an amino acid sequence that is at least partially different from the amino acid sequence included in the first protein sequence data 412. In some exemplary embodiments, the second protein sequence data 442 may include a subset of the first protein sequence data 412. In embodiments, protein sequence filtering may be applied to the first protein sequence data 412 by analyzing the first protein data 412 according to one or more criteria. For example, the second protein sequence data 442 may correspond to a particular type of protein and / or amino acid sequence that includes a region with particular amino acids at particular positions. Additionally, the second protein sequence data 442 may correspond to an amino acid sequence of a protein with particular biophysical properties. In various embodiments, the second protein sequence data 442 may correspond to an amino acid sequence of a protein with particular structural properties. Examples of particular structural properties include surface charge and post-translational modifications of one or more regions.

[0074] The second challenging element 434 may label the second generated sequences 438 as zero and the structured sequences obtained from the protein sequence data 442 as one. In such a situation, the second classification output 440 may include a number from 0 and 1 if the second generating adversarial network 430 is a Wasserstein GAN. In a further example, the second challenging element 434 may implement a distance function that generates an output indicating an amount of distance between the second generated sequences 238 and the proteins included in the second protein sequence data 442. In an embodiment in which the second challenging element 434 implements a distance function, the second classification output 440 may be a number from -∞ to ∞. In some examples, the amino acid sequences obtained from the second protein sequence data 442 are referred to as ground truth data.

[0075] After the filtered sequences 444 are generated by the protein sequence filter 442, the filtered sequences may be subjected to a second data pre-processing 446 before being provided to the second challenging element 434. In an embodiment, the filtered sequences 444 may be arranged according to a classification system before being provided to the second challenging element 434. The second data pre-processing 444 may include matching amino acids contained in the proteins of the second protein sequence data 442 with numerical values ​​that may represent positions within the protein. The numerical values ​​may include a series of numbers with a start and end point. The second data pre-processing 444 may generate a second structured sequence 446 that is provided to the second challenging element 434. The second structured sequence 446 may include a matrix that indicates amino acids associated with various positions of the protein. In an example, the second structured sequence 446 may include a matrix with columns corresponding to different amino acids and rows corresponding to positions of structural bases of the protein. For each element in the matrix, a 0 may be used to indicate the absence of an amino acid at the corresponding position, and a 1 may be used to indicate the presence of an amino acid at the corresponding position. If a position represents a gap in the amino acid sequence, the row associated with that position may contain zeros for each column. Second generated sequence 438 may also be represented using vectors with the same or similar number scheme as used for second structured sequence 446. In some illustrative examples, second structured sequence 446 and second generated sequence 438 may be encoded using what is known as one-hot encoding.

[0076] After the second generative adversarial network 430 has undergone a training process, a modified trained model 448 may be generated to generate protein sequences. The modified trained model 448 may represent the trained model 420 after being trained using the second protein sequence data 442. In an example, the training process for the second generative adversarial network 430 may be completed after the functions implemented by the second generating element 432 and the second challenging element 434 have converged. The functions may be converged based on the movement of the values ​​of the model parameters to a particular value such that protein sequences are generated by the second generating element 432 and feedback is provided for the second challenging element 434 based on the movement of the values ​​of the model parameters to a particular value. In various embodiments, the training of the second generative adversarial network 430 may be completed when the protein sequences generated by the second generating element 432 have particular characteristics.

[0077] A second sequence input 450 can be provided to the modified trained model 448, which can generate a second sequence 452. The second sequence input 450 can correspond to a random or pseudo-random sequence that can be used to generate the second sequence 452, which includes an amino acid sequence of a protein. In operation 454, the second sequence 452 can be evaluated to determine whether the second sequence 452 has a particular set of characteristics. The sequence evaluation 454 can generate an index 456 indicative of a characteristic of the second sequence 452, such as a biophysical characteristic of a protein or a region of a protein, and / or the presence or absence of an amino acid at a particular position. Additionally, the index 456 can indicate an amount of correspondence between the characteristic of the second sequence 452 and a particular set of characteristics. In some examples, the index 456 can indicate a number of positions of the second sequence 452 that differ from a sequence generated by a germline gene for the protein. Additionally, the sequence evaluation 454 can determine the presence or absence of a structural characteristic of a protein that corresponds to the second sequence 452.

[0078] Continuing training of the training model 420 as part of the second generative adversarial network 430 can generate a modified trained model 448 that generates more specifically tailored amino acid sequences than those generated by the training model 420. For example, the second generative adversarial network 430 can be trained using extracted amino acid sequences corresponding to proteins with specific structural and / or biophysical properties. Thus, once the trained model 420 is generated by the first generative adversarial network 402 to generate amino acid sequences corresponding to proteins, the modified trained model 448 can be generated as part of the second generative adversarial network 430 to generate more specific amino acid sequences of proteins according to the second protein sequence data 442 provided to the second generative adversarial network 430.

[0079] Furthermore, when it is desired to generate amino acid sequences of proteins with many specific properties, the number of sequences available for training the generative allele network is limited. In such a situation, the accuracy, efficiency and / or effectiveness of the generative allele network for generating amino acid sequences of proteins with specific properties may not be perfect. Thus, if there is not a sufficient number of amino acid sequences available for training the generative allele network, the amino acid sequences generated by the generative allele network may not have the desired properties. When implementing the techniques and systems described with respect to FIG. 4, the first generative allele network 402 can use a first data set to perform a part of the process of determining amino acid sequences corresponding to a protein or a broader class of proteins, and the second generative allele network 430 can use a second, different data set to perform a further part of the process of accurately and efficiently generating amino acid sequences of proteins with more specific properties.

[0080] In an illustrative example, the modified model 448 can generate amino acid sequences corresponding to antibodies or portions of antibodies with specific properties. For example, after the first generative allele network 402 generates a trained model 420 to generate amino acid sequences with properties of an antibody, the second generative allele network 430 can generally generate a modified trained model 448 to generate antibodies or portions of antibodies with at least one of a specific biophysical property, an amino acid sequence corresponding to an antibody or antibody region associated with a germline gene, or an amino acid sequence of an antibody with a specific structural property, such as a specific structural property at a specific position. In a particular illustrative example, the trained model 420 can be used to generate amino acid sequences corresponding to an IgG antibody, and the modified trained model 448 can be used to generate amino acid sequences corresponding to an IgG antibody that includes a light chain with a variable region with specific amino acids at specific positions. In a further illustrative example, the trained model 420 can be used to generate amino acid sequences of an antibody heavy chain, and the modified trained model 448 can be used to generate amino acid sequences of an antibody heavy chain that can form a disulfide bond at a specific position.

[0081] The illustrative example of Fig. 4 shows training of a model using multiple training sets in a framework including two generative adversarial networks. In further embodiments, training of a model using multiple training datasets can also be represented using a single generative adversarial network. Furthermore, although the illustrative example of Fig. 4 shows training of a model using a generative adversarial network with two training datasets, in various embodiments, more than two datasets can be used to train a model using a generative adversarial network according to the embodiments described herein.

[0082] FIG. 5 illustrates an exemplary framework 500 for generating antibody sequences that are variants of a parent antibody, according to some embodiments. The framework 500 may include a generative adversarial network 502, which may include a generator element 504 and a challenging element 506. The generator element 504 may implement a model to generate amino acid sequences based on inputs provided to the generator element 504. In various embodiments, the model implemented by the generator element 504 may include one or more functions. In an embodiment, the generator element 504 may utilize a model to generate generated sequences 508. In various examples, the generator element 504 may implement a model to generate amino acid sequences based on input vectors 510 and the parent antibody sequence 512 provided to the generator element 504. The training input 510 may include a random or pseudo-random sequence of a particular length. The parent antibody sequence 512 may be provided to the generator element 504 as a matrix matrix, with columns corresponding to different amino acids and rows corresponding to the position of the structural bases of the protein. For each element in the matrix, a 0 may be used to indicate that the amino acid is absent at the corresponding position, and a 1 may be used to indicate that the amino acid is present at the corresponding position. The matrix may also include additional columns that represent gaps in the amino acid sequence where no amino acid is present at a particular position in the amino acid sequence. Thus, if a position represents a gap in the amino acid sequence, the row associated with that position may include zeros for each column. The parent antibody sequence 512 may include base molecules that the generative allele network 502 may utilize to train a model to generate variant antibody sequences that correspond to the parent antibody sequence 512.

[0083] The challenging element 506 can generate an output indicating whether the amino acid sequence generated by the generating element 504 satisfies various properties. In various embodiments, the challenging element 506 can be a classifier. In further situations, for example, when the generating adversarial network 502 is a Wasserstein GAN, the challenging element 506 can include a critique. In an example, the challenging element 506 can generate a classification output 514 indicating whether the amino acid sequence generated by the generating element 504 satisfies various properties. In an illustrative example, based on the similarities and differences between the generated sequence 508 and further sequences provided to the challenging element 506, such as amino acid sequences included in the antibody sequence data 516, the challenging element 506 can generate a classification output 514 to indicate an amount of similarity or an amount of difference between the generated sequence 508 and the sequences provided to the challenging element 506 included in the antibody sequence data 516. The antibody sequence data 516 can include amino acid sequences of antibodies obtained from one or more databases that store antibody sequences. Additionally, the classification output 514 may indicate the amount of similarity or the amount of dissimilarity between the generated sequences 508 and the parent antibody sequences 512 .

[0084] In an example, the challenging element 506 may label the generated sequence 508 as zero and the structured sequence obtained from the protein sequence data 516 as one. The challenging element 506 may also label the parent antibody sequence 512 as one. In these circumstances, the classification output 514 may include a first number from zero to one for one or more amino acid sequences included in the antibody sequence data 516 and a second number from zero to one for the parent antibody sequence 512. In a further example, the challenging element 506 may implement a distance function that generates an output indicative of an amount of distance between the generated sequence 508 and the protein included in the antibody sequence data 516. Additionally, the challenging element 506 may implement a distance function that generates an output indicative of an amount of distance between the generated sequence 508 and the parent antibody sequence 512. In embodiments in which the challenging element 506 implements a distance function, the classification output 514 may include a first number between -∞ and ∞ indicating the distance between the generated sequence 508 and one or more sequences included in the antibody sequence data 516, and a second number between -∞ and ∞ indicating the distance between the generated sequence 508 and the parent antibody sequence 512.

[0085] In various embodiments, the classification output 514 in terms of the amount of dissimilarity or amount of similarity between the generated sequence 508 and the parent antibody sequence 512 may be determined using a penalty function. In particular embodiments, the challenging element 506 may evaluate the generated sequence(s) 508 in terms of the parent antibody sequence 512 in terms of the amount of similarity between the generated sequence(s) 508 and the parent antibody sequence 512 and / or the amount of dissimilarity between the generated sequence(s) 508 and the parent antibody sequence 512. In an example, a first threshold amount of dissimilarity between the generated sequence 508 and the parent antibody sequence 512 may be specified. Additionally, a second threshold amount of similarity between the generated sequence 508 and the parent antibody sequence 512 may be specified. The challenging element 506 may evaluate the generated sequence 508 in terms of the parent antibody sequence 512 based on at least one of the first threshold amount of dissimilarity or the second threshold amount of similarity. In an embodiment, the challenging element 506 may perform a penalty function based on the amount of similarity and / or dissimilarity between the generated sequence 508 and the parent antibody sequence 512 with respect to the first threshold and / or the second threshold, and may utilize the output of the penalty function in generating the portion of the classification output 514 that corresponds to the parent antibody sequence 512 with respect to the generated sequence 508.

[0086] The antibody sequences included in the antibody sequence data 516 may be subjected to data pre-processing at 518, where the antibody sequences are mapped onto a classification system 520 before being provided to the challenging element 506. The classification system 520 may indicate that a particular region of the antibody should be represented by a particular number of positions. For example, a heavy chain variable region may be represented by 125-165 positions (e.g., 149 positions) in the classification system 520. In another example, a heavy chain constant region may be represented by 110-140 positions (e.g., 123 positions) in the classification system 520. In a further example, a hinge region of the heavy chain may be represented by 110-140 positions (e.g., 123 positions) in the classification system 520. In situations where the amino acid sequence of the antibody does not include a large number of amino acids corresponding to a predetermined number of positions for a region of the antibody, the data pre-processing at 518 may introduce zero values ​​for one or more positions in the classification system 520. In embodiments, the 0 regions may correspond to gaps that may reveal structural information of the antibody. Thus, the classification system 520 can accommodate variable numbers of amino acids contained in different regions of an antibody. By mapping at least a portion of the antibody sequences in the antibody sequence data 516 to the classification system 520, a standardized data set can be generated that can be processed by the generative allele network 502 and that is independent of the number of amino acids contained in individual regions of the antibody.

[0087] In an illustrative example, the mapping of the antibody sequence onto the classification system 520 performed at 518 may include determining the variable and constant regions of the antibody. The variable and constant regions of the antibody may be determined by comparing the amino acid sequence contained in the antibody sequence data 516 to template amino acid sequences corresponding to various regions of the antibody. In a particular example, a position specific scoring matrix (PSSM) may be created for each region type to determine the alignment between portions of the antibody sequence and the template amino acid sequence of the antibody region. In situations where a local alignment between the antibody sequence and the template sequence is determined, the position of individual amino acids of the antibody sequence within the classification system 520 may be further determined using antibody sequences produced from germline genes.

[0088] After identifying portions of the antibody sequence that may correspond to a particular type of antibody region using sequences associated with germline genes, the amount of identity between the particular portion of the antibody sequence with the template sequence can be determined. If a minimum number of amino acids in the given antibody sequence correspond to the template sequence and / or if the amount of identity between the template sequence and the given antibody sequence is at least a threshold amount of identity, the given antibody sequence can be classified as a particular region that corresponds to the template. In one example, the given antibody sequence can be classified as a complementarity determining region (CDR) of an antibody heavy chain. Furthermore, each antibody sequence classified as associated with a particular region can also be assigned a score indicating the likelihood that the given antibody sequence corresponds to the classification. Portions of the antibody sequence with overlapping sequences can be filtered by score such that the highest scoring portions of the antibody sequence are assigned to the classification. Gaps in the antibody sequence can be determined in situations where the number of amino acids for a given portion of the antibody sequence with a particular classification is less than the number of positions assigned to the classification in the classification system 520.

[0089] Although a particular classification system 520 has been described with respect to the exemplary embodiment of Figure 5, additional classification systems may be used in addition to or as an alternative to classification system 520. For example, the Kabat classification scheme, the Chotia classification scheme, the Martin classification scheme, the Gelfand classification scheme, the IMGT classification scheme, the Aho classification scheme, combinations thereof, and the like may be used to classify the amino acid sequences of antibodies included in antibody sequence data 516.

[0090] After the antibody sequence data 516 has been mapped 518 onto the classification system 520, the mapped sequence data 522 may be provided to the challenging element 506. The mapped sequence data 522 may include a matrix indicating the position of amino acids relative to regions of the antibody, and a matrix indicating the amino acid associated with each position. If there are gaps in the amino acid sequence with respect to the classification system, a zero value may be associated with each amino acid included in the matrix.

[0091] After the generative adversarial network 502 undergoes a training process, a trained model 524 is generated and can generate sequences of proteins. In an example, the training process for the generative adversarial network 502 can be completed after the function implemented by the generating component 504 has converged. The function can be converged based on the movement of values ​​of the model parameters to certain values ​​such that antibody sequences are generated by the generating component 504 and feedback is obtained from the challenging component 506. In various embodiments, the training of the generative adversarial network 502 can be completed when the antibody sequences generated by the generating component 504 have certain characteristics. By way of example, the amino acid sequences generated by the generating component 504 can be analyzed by a software tool that can analyze the amino acid sequences to determine at least one of biophysical properties of the amino acid sequences, structural properties of the amino acid sequences, or adhesion to amino acid sequences corresponding to one or more protein germlines.

[0092] A sequence input 526 can be provided to the model 524, which can generate an antibody sequence 528. The sequence input 526 can correspond to a random or pseudo-random sequence of a particular length. In various embodiments, the antibody sequence 528 can include variants of the parent antibody sequence 512. In sequence evaluation 530, the antibody sequence 528 can be evaluated to determine whether the antibody sequence 528 comprises a particular set of properties. The sequence evaluation 530 can generate an index 532 indicative of the properties of the antibody sequence 528. Further, the index 532 can indicate the amount of correspondence between the properties of the antibody sequence 528 and the particular set of properties. In some examples, the index 532 can indicate a number of positions in the amino acid sequence 528 that differ from the amino acid sequence of the protein generated from a germline gene.

[0093] 6 illustrates an example of a framework 400 for generating amino acid sequences of antibodies that bind to a particular antigen, according to an embodiment. The framework 600 may include a generative adversarial network 602, which may include a generating element 604 and a challenging element 606. The generative adversarial network 602 may include a conditional generative adversarial network. The generating element 604 may implement a model to generate an amino acid sequence based on inputs provided to the generating element 604. In various embodiments, the model implemented by the generating element 604 may include one or more functions. In an embodiment, the generating element 604 may utilize the model to generate a generated sequence 608.

[0094] The generator 604 can implement a model that generates an amino acid sequence based on an input vector 610 and an antigen sequence 612 provided to the generator 604. The input vector 610 can include a random or pseudo-random sequence of numbers of a particular length. The antigen sequence 612 can include a matrix with columns corresponding to different amino acids and rows corresponding to positions of structural bases of a protein. For each element in the matrix, a 0 can be used to indicate that the amino acid is absent at the corresponding position, and a 1 can be used to indicate that the amino acid is present at the corresponding position. If a position represents a gap in the amino acid sequence, the row associated with that position can include zeros for each column. Thus, if a position represents a gap in the amino acid sequence, a 1 can be placed in the gap column for the row associated with the position where the amino acid is not present. The antigen sequence 612 can correspond to an antigen to which an antibody with an amino acid sequence produced by the generator 604 can bind. In various examples, the antigen sequence 612 can correspond to an antigen with one or more epitope regions to which the antibody can bind. In one or more examples, the generator adversarial network 602 can generate an amino acid sequence for an antibody that binds to one or more epitope regions.

[0095] The challenging component 606 may generate an output indicating whether the amino acid sequence generated by the generating component 604 has various characteristics. In an embodiment, the challenging component 606 may be a classifier of the generating adversarial network 602. The challenging component 606 may generate a classification output 614 indicating whether the amino acid sequence generated by the generating component 604 satisfies one or more criteria. In an exemplary embodiment, based on the similarities and differences between the generated sequence 608 and further sequences provided to the challenging component 606 as training data, such as amino acid sequences included in the antibody-antigen sequence data 616, the challenging component 606 may generate a classification output 614 indicating the amount of similarity or difference between the generated sequence 608 and the sequences provided to the challenging component 606 from the antibody-antigen interaction data 616.

[0096] The antibody-antigen interaction data 616 may be obtained from one or more databases that store data related to the binding of antibodies to antigens. The antibody-antigen interaction data 616 may store the amino acid sequence of an antibody and the amino acid sequence of an antigen bound by the antibody. The antibody-antigen interaction data 616 may also include information regarding at least one of the secondary or tertiary structures of each antibody and each antigen. In various examples, the antibody-antigen interaction data 616 may include information corresponding to at least one of the secondary or tertiary structures of each antibody and each antigen when bound to each other. Furthermore, the antibody-antigen interaction data 616 may include at least one of the amino acid sequences of epitope regions of the antigen and the amino acid sequences of the corresponding binding regions of the antibody and the binding to each epitope region by various antibody regions. In an exemplary embodiment, the antibody-antigen interaction data 616 may indicate multiple positions of an antigen that binds to one or more positions of an antibody via non-covalent molecular / atomic / chemical interactions, such as van der Waals forces, hydrogen bonds, electrostatic interactions, hydrophobic forces, combinations thereof, etc.

[0097] In an illustrative example, the portion of the antibody sequence data 616 provided to the challenging element 606 with respect to the antigen sequence 612 may include an amino acid sequence of an antibody that has at least a minimum binding affinity with the antigen sequence 612. The binding affinity of the portion of the antibody's amino acid sequence with respect to the antigen sequence 612 can be determined based on the binding affinity of the portion of the antibody's amino acid sequence with respect to the antigen that has at least a threshold amount of similarity with respect to the antigen sequence 612. For example, the antigen sequence 612 can be compared to the amino acid sequences of the antigens stored as part of the antibody-antigen interaction data 616. Antigens with amino acid sequences that have at least a threshold amount of identity with respect to the antigen sequence 612 can be determined. Amino acid sequences of antibodies that bind to the antigen can then be identified, and these amino acid sequences can be sent as input to the challenging element 606. In a further example, the amino acid sequences of epitope regions of the antigens included in the antibody-antigen interaction data 616 can be compared to the antigen sequence 612. In these circumstances, epitope regions of the antigens included in the antibody-antigen interaction data 616 that have at least a threshold amount of identity with one or more portions of the antigen sequence 612 can be determined. Antibodies that bind to these epitope regions can then be identified and sent as input to the challenging element 606.

[0098] The challenging element 606 can generate a classification output 614 that identifies the generated sequence 608 based on an amount of correspondence between the generated sequence 608 and the training data provided to the challenging element 606. The training data can include at least a portion of the amino acid sequences included in the antibody-antigen interaction data 616. The classification output 614 can be based on a type of generative adversarial network associated with the generative adversarial network 602. For example, for a first type of generative adversarial network, the challenging element 606 can generate a classification output 614 of 1 for generated sequences 608 that have at least a threshold amount of correspondence with respect to the training data provided to the challenging element 820. Also, for the first type of generative adversarial network, the challenging element 606 can generate a classification output 614 of 0 for generated sequences 608 that have less than a threshold amount of correspondence with respect to the training data provided to the challenging element 606. In various examples, the classification output 614 can be generated that identifies the generated sequence 608 using a numerical scale from 0 to 1 based on an amount of similarity between the generated sequence 608 and the amino acid sequences included in the training data provided to the challenging element 606.

[0099] Additionally, where the generative adversarial network 602 implements a second generative adversarial network, such as a Wasserstein GAN, the challenging element 606 can implement a distance function to generate a classification output 614 indicative of a measure of distance between the generated sequence 608 and an amino acid sequence included in the training data provided to the challenging element 606. For example, the challenging element 606 can generate a classification output 614 that includes a number between -∞ and ∞ indicative of a distance between the generated sequence 608 and at least a portion of an amino acid sequence included in the antibody-antigen interaction data 616. In various examples, the training data obtained from the antibody-antigen interaction data 616 is referred to as ground truth data.

[0100] The amino acid sequences obtained from the antibody-antigen interaction data 616 may be subjected to data pre-processing at 618. The amino acid sequences obtained from the antibody-antigen interaction data 616 may include at least one of an amino acid sequence of an antibody or an amino acid sequence of an antigen. The amino acid sequences may be mapped onto a classification system as part of the data pre-processing 618 before being provided to the challenging element 606. For example, the classification system may indicate that a particular region of the antibody should be represented by a particular number of positions. In an exemplary embodiment, the classification system may be the same as or similar to the classification system 520 described with respect to FIG. 5. In various examples, the Kabat classification system, the Chotia classification system, the Martin classification system, the Gelfand classification system, the IMGT classification system, the Aho classification system, combinations thereof, and the like may be used to classify the amino acid sequences of the antibodies included in the antibody-antigen interaction data 616. At least a portion of the amino acid sequences included in the antibody-antigen interaction data 616 may be mapped to the classification system by the generated adversarial network 602 to generate a standardized data set that is independent of the number of amino acids included in each region of the antibody. After obtaining the amino acid sequence derived from the antibody-antigen interaction data 616 via pre-processing 618, antibody sequence data 620 corresponding to the amino acid sequence can be sent to a challenging element 608.

[0101] After the generative adversarial network 602 undergoes a training process, a trained model 622 may be generated to generate a set of antibodies. In an example, the training process for the generative adversarial network 602 may be completed after the function implemented by the generating component 604 has converged. Antibody sequences may be generated by the generating component 604 by converging the function based on the movement of the values ​​of the model parameters to specific values, and feedback may be obtained from the challenging component 606. In various embodiments, the training of the generative adversarial network 602 may be completed when the antibody sequences generated by the generating component 604 have specific properties. By way of example, the amino acid sequences generated by the sequence generating component 104 may be analyzed by a software tool that may analyze the amino acid sequences to determine at least one of biophysical properties of the amino acid sequences, structural properties of the amino acid sequences, or adhesion to amino acid sequences corresponding to one or more protein germlines.

[0102] A sequence input 624 can be provided to the training model 622, which can generate an antibody sequence 626. The sequence input 624 can correspond to a random or pseudorandom sequence of a particular length that can be used to generate a sequence 324 that includes an amino acid sequence of a protein. In an illustrative example, the sequence input 624 can include an antigen sequence 612. In a further example, the sequence input 624 can include information indicative of an interaction between at least a portion of one or more regions of one or more antibodies and at least a portion of one or more regions of one or more antigens. In the sequence evaluation 628, the antibody sequence 626 can be evaluated to determine whether the antibody sequence 626 comprises a particular set of characteristics. For example, the sequence evaluation 628 can generate an index 630 indicative of the characteristics of the antibody sequence 626. Furthermore, the index 630 can indicate the amount of correspondence between the characteristics of the antibody sequence 626 and a particular set of characteristics. Indicator 630 may also indicate characteristics such as a number of hydrophobic amino acids in antibody 626 sequence, a number of positively charged amino acids in antibody sequence 626, a number of negatively charged amino acids in antibody sequence 626, a measure of a biophysical property of an antibody having antibody sequence 626, an expression level of an antibody having antibody sequence 626, or a combination of one or more thereof. In some examples, indicator 630 may correspond to an amount of binding by an antibody to an antigen, an interaction between the antibody and an antigen, an interaction between the antigen and an amino acid sequence of an antibody derived from a germline gene.

[0103] 7 illustrates an exemplary framework 700 for generating multiple libraries of proteins and combining the protein libraries to generate additional proteins, according to some embodiments. The framework 700 may include a first generative adversarial network 702 and a second generative adversarial network 704. The first generative adversarial network 702 may be trained to generate a model based on a first protein sequence 706. Additionally, the second generative adversarial network 704 may be trained to generate an additional model based on a second protein sequence 708. In various embodiments, the first protein sequence 706 and the second protein sequence 708 may include the same amino acid sequence. In further embodiments, the first protein sequence 706 may include at least one amino acid sequence that is different from the second protein sequence 708.

[0104] The first generative allele network 702 can generate a number of amino acid sequences for proteins included in a first protein sequence library 710. Additionally, the second generative allele network 704 can generate a number of amino acid sequences for included in a second protein sequence library 712. At least a portion of the amino acid sequences included in the first protein sequence library 710 can differ from the amino acid sequences included in the second protein sequence library 712. At 714, the first protein sequence library 710 and the second protein sequence library 712 can be combined to generate a combined protein sequence 716.

[0105] At 718, the combined protein sequences 716 can be evaluated according to one or more criteria 720. For example, the combined protein sequences can be evaluated to determine whether the combined protein sequences 718 have particular amino acid sequence regions, whether they are associated with amino acid sequences that have particular biophysical properties, and / or whether they are associated with amino acid sequences that have particular tertiary structures. In various embodiments, the combined protein sequences 716 can be evaluated based on the amino acid sequences of proteins derived from germline genes.

[0106] After the binding protein sequences 716 are evaluated at 718, a binding protein sequence library 722 can be generated. The binding protein sequence library 722 can include at least a portion of the binding protein sequences 716. In certain embodiments, the binding protein sequences 716 can be filtered according to criteria 720 to include certain protein sequences included in the binding protein sequences in the binding protein sequence library.

[0107] In an illustrative example, the first protein sequence library 710 may include amino acid sequences corresponding to heavy chain regions of an antibody, and the second protein sequence library 712 may include amino acid sequences corresponding to light chain regions of an antibody. In these circumstances, the heavy and light chain regions may be combined to generate a full antibody amino acid sequence that includes both heavy and light chain regions. In various embodiments, a further generative allele network may be generated in which amino acid sequences corresponding to heavy and light chain regions may be combined.

[0108] 8 illustrates a further exemplary framework 800 for generating antibody amino acid sequences using separately generated heavy and light chain amino acid sequences of a paired antibody, according to an embodiment. Framework 800 may include a generative adversarial network 802. Generative adversarial network 802 may implement one or more neural network techniques. For example, generative adversarial network 802 may implement one or more recurrent neural networks. Additionally, generative adversarial network 802 may implement one or more convolutional neural networks. In an embodiment, generative adversarial network 802 may implement a combination of recurrent neural networks and convolutional neural networks.

[0109] The generative adversarial network 802 may include a light chain generation component 804 and a heavy chain generation component 806. The light chain generation component 804 may implement a first model to generate data corresponding to an amino acid sequence of a light chain of an antibody. Additionally, the heavy chain generation component 806 may implement a second model to generate data corresponding to an amino acid sequence of a heavy chain of an antibody. The light chain generation component 804 may implement a first model to generate an amino acid sequence of a light chain of an antibody based on first input data 808 provided to the light chain generation component 804. The heavy chain generation component 806 may implement a second model to generate an amino acid sequence of a heavy chain of an antibody based on second input data 810. The first input data 808 may include first noise data generated by a random or pseudorandom generator. The second input data 810 may include second noise data generated by a random or pseudorandom generator. In various embodiments, the first model implemented by the light chain generation component 804 may include one or more first functions including one or more first variables, each with a first weight, and the second model implemented by the heavy chain generation component 806 may include one or more second functions including one or more second variables, each with a second weight.

[0110] The light chain generation component 804 can implement a first model to generate a light chain sequence 812 based on the first input data 808. The light chain sequence 812 can include data corresponding to amino acids at positions in a light chain of an antibody. The light chain sequence 812 can include a sequence of amino acids of a light chain of an antibody encoded according to one or more encoding schemes. In various examples, the light chain sequence 812 can include data corresponding to amino acids at individual positions of a light chain of an antibody encoded according to a scheme. In one or more illustrative examples, the light chain sequence 812 can include an amino acid sequence of a light chain of an antibody encoded according to a one-hot encoding scheme.

[0111] The heavy chain generation component 806 can implement a second model to generate a heavy chain sequence 814 based on second input data 810. The heavy chain sequence 814 can include data corresponding to amino acids at positions in a heavy chain of an antibody. The heavy chain sequence 814 can include a sequence of amino acids in a light chain of an antibody encoded according to one or more encoding schemes. In various examples, the heavy chain sequence 814 can include data corresponding to amino acids at individual positions in a heavy chain of an antibody encoded according to a scheme. In one or more illustrative examples, the heavy chain sequence 814 can include an amino acid sequence in a heavy chain of an antibody encoded according to a one-hot encoding scheme.

[0112] The light chain sequences 812 and the heavy chain sequences 814 may be provided to a sequence combination element 816. The sequence combination element 816 may combine at least one light chain sequence 812 with at least one heavy chain sequence 814 to generate a combined antibody sequence 818. In various embodiments, the sequence combination element 816 may combine a single light chain sequence 812 with a single heavy chain sequence 814. The combined antibody sequence 818 may include data corresponding to amino acids at positions in the one or more light chain sequences 812 and one or more heavy chain sequences 814. In one or more examples, the sequence combination element 816 may generate the combined antibody sequence 818 by combining one or more light chain sequences 812 and one or more heavy chain sequences 814. For example, the sequence combination element 816 may add a first string of alphanumeric characters representing a light chain sequence of an antibody to a second string of alphanumeric characters representing a heavy chain sequence of an antibody to generate the combined antibody sequence 818. The combined antibody sequence 818 may include a first portion that corresponds to the light chain sequence 812 and a second portion that corresponds to the heavy chain sequence 814. For example, a first number of positions of the combined antibody sequence 818 may correspond to amino acids of the light chain sequence 812 and a second number of positions of the combined antibody sequence 818, positions after the first number of positions, may correspond to amino acids of the heavy chain sequence 814. In a further example, the first number of positions of the combined antibody sequence 818 may correspond to amino acids of the heavy chain sequence 814 and a second number of positions of the combined antibody sequence 818, positions after the first number of positions, may correspond to amino acids of the light chain sequence 812. In various embodiments, the combined antibody sequence 818 may correspond to amino acids of one or more light chain sequences 812 and one or more heavy chain sequences 814 arranged according to a schema.

[0113] The generated adversary network 802 may include a challenging element 820. The challenging element 820 may generate an output indicating that the combined antibody sequences 818 do or do not meet one or more properties. The challenging element 820 may generate a classification output 822 that may be provided to the light chain generation element 804 and the heavy chain generation element 806. The challenging element 820 may evaluate the combined antibody sequences 818 with respect to training data that includes antibody sequence data 824. The challenging element 820 may compare the combined antibody sequences 818 generated by the sequence combination element 816 with at least a portion of the amino acid sequences included in the antibody sequence data 824. The classification output 822 generated based on the comparison may indicate an amount of correspondence between the combined antibody sequences 818 with respect to at least a portion of the amino acid sequences included in the antibody sequence data 824. For example, based on the similarities and differences between the combined antibody sequence 818 and at least a portion of the amino acid sequences included in the antibody sequence data 824, the classification output 822 generated by the challenging element 820 may indicate the amount of similarity or the amount of difference between the combined antibody sequence 818 and at least a portion of the amino acid sequences included in the antibody sequence data 824.

[0114] The challenging element 820 can generate a classification output 822 labeling the combined antibody sequences 818 based on an amount of correspondence between the combined antibody sequences 818 and the training data provided to the challenging element 820. The training data can include at least a portion of the amino acid sequences included in the antibody sequence data 824. The classification output 822 can be based on the type of generated adversarial network associated with the generated adversarial network 802. For example, for a first type of generated adversarial network, the challenging element 820 can generate a classification output 822 of 1 for combined antibody sequences 818 that correspond to at least a threshold amount with respect to the training data provided to the challenging element 820. Also, for a first type of generated adversarial network, the challenging element 820 can generate a classification output of 0 for combined antibody sequences 818 that correspond to less than a threshold amount with respect to the training data provided to the challenging element 820. In various examples, for a first type of generated adversarial network, the challenging element 820 can generate a classification output 822 that labels the combined antibody sequence 818 using a numerical scale from 0 to 1 based on the amount of similarity between the combined antibody sequence 818 and the amino acid sequences included in the training data provided to the challenging element 820.

[0115] Additionally, where generative adversarial network 802 implements a second generative adversarial network, such as a Wasserstein GAN, challenging element 820 can implement a distance function to generate a classification output 822 indicative of a measure of distance between combined antibody sequences 818 and amino acid sequences included in the training data provided to challenging element 820. For example, challenging element 820 can generate a classification output 822 that includes a number between -∞ and ∞ indicative of the distance between combined antibody sequences 818 and at least a portion of the amino acid sequences included in antibody sequence data 824. In various examples, the training data obtained from antibody sequence data 824 is referred to as ground truth data.

[0116] The protein sequences included in the antibody sequence data 824 may undergo data pre-processing 826 before being provided to the challenging element 820. In an embodiment, the data pre-processing 826 may place the antibody sequence data 824 through a classification system before being provided to the challenging element 820. The data pre-processing 826 may include pairing the amino acids in the antibody sequence data 824 with a numerical value that may represent the amino acid in the sequence and its structure-based position within the antibody. The numerical value may include a series of numbers with a start and an end. In an illustrative example, a T may be paired with the number 43, indicating that a threonine molecule is located at structure-based position 43 in a particular protein domain type.

[0117] The output generated by data preprocessing 826 may include a structured array 828. The structured array 828 may include a matrix indicating the amino acids associated with various positions of an antibody. In an example, the structured array 828 may include a matrix with columns corresponding to different amino acids and rows corresponding to positions of structural bases of the protein. For each element in the matrix, a 0 may be used to indicate that there is no amino acid at the corresponding position, and a 1 may be used to indicate that there is an amino acid at the corresponding position. If a position represents a gap in the amino acid sequence, the row associated with that position may include zeros for each column. Additionally, the combined antibody sequence(s) 818 may also be represented using a vector with the same or similar number scheme as used for the structured array 828. In some illustrative examples, the structured array 828 and the combined antibody sequence(s) 818 may be encoded using what is known as one-hot encoding. In various embodiments, the structured array 828 may include the amino acid sequence of an antibody light chain followed by the amino acid sequence of an antibody heavy chain. In a further embodiment, structured sequence 828 may comprise an antibody heavy chain amino acid sequence followed by an antibody light chain amino acid sequence. The antibody light chain and antibody heavy chain sequences in structured sequence 828 may correspond to the antibody light chain and antibody heavy chain sequences included in combined antibody sequence 818.

[0118] In various examples, the training of the light chain generating element 804 and the heavy chain generating element 806 can be performed asynchronously. For example, the training of the heavy chain generating element 806 can be stopped for a period of time while the training of the light chain generating element 804 continues. In one or more examples, the light chain generating element 804 and the heavy chain generating element 806 can be trained simultaneously for a period of time. During this period, the training of the heavy chain generating element 806 can proceed faster than the training of the light chain generating element 804. In such a situation, the training of the heavy chain generating element 806 can be stopped while the light chain generating element 804 is being trained. In some examples, the sequences generated by the heavy chain generating element 806 can be evaluated at various times to determine metrics regarding the quality of the amino acid sequences generated by the heavy chain generating element 806. In various examples, the training of the heavy chain generating element 806 can be stopped when the metrics meet one or more threshold metrics. The light chain generating element 804 can continue training until the sequences generated by the light chain generating element 804 meet one or more threshold metrics. After sequences from both the light chain generation element 804 and the heavy chain generation element 806 meet one or more threshold metrics, the light chain generation element 804 and the heavy chain generation element 806 can continue training. In one or more examples, the training of the light chain generation element 804 and the heavy chain generation element 806 can be trained until one or more metrics used to evaluate sequences generated by the light chain generation element 804 and the heavy chain generation element 806 diverge by at least a threshold amount.

[0119] In one or more exemplary embodiments, the training of the heavy chain production element 806 can be implemented with Hobble weights such that the training of the light chain production element 804 and the training of the heavy chain production element 806 proceed at a relatively similar rate. Additionally, the training of the heavy chain production element 806 can proceed at a slower gradient such that the training of the light chain production element 804 and the training of the heavy chain production element 806 proceed at a relatively similar rate.

[0120] After the generative allele network 802 undergoes a training process, a trained model 830 can be generated to generate amino acid sequences for antibodies. The trained model 830 can include a light chain generator 804 and a heavy chain generator 806 after a training process using the antibody sequence data 824. In an example, the training process for the generative allele network 802 can complete a classification output 822 after demonstrating at least a threshold amount of correspondence between the combined antibody sequence 818 and the amino acid sequences included in the antibody sequence data 824. In a further embodiment, the training of the generative allele network 802 can be completed when the combined antibody sequence 818 comprises a particular property. By way of illustration, the amino acid sequence generated by the sequence combination element 816 can be analyzed by a software tool that can analyze the amino acid sequence to determine at least one of a biophysical property of the amino acid sequence, a structural property of the amino acid sequence, or an attachment to a corresponding amino acid sequence of one or more protein germlines. The property of the combined antibody sequence 818 determined by the analysis of the software tool in association with the particular property can be used to determine whether the training of the generative allele network 802 is complete.

[0121] A sequence input 832 can be provided to the training model 830, which can generate a combinatorial antibody sequence library 834. The sequence input 832 can correspond to a random or pseudo-random sequence of numbers that can be used to generate the combinatorial antibody sequence library 834. The combinatorial antibody sequence library 834 can include amino acid sequences of antibodies comprising at least one light chain and at least one heavy chain corresponding to the individual antibodies included in the combinatorial antibody sequence library 834. The amino acid sequences included in the combinatorial antibody sequence library 834 produced by the trained model 830 can be represented as a matrix structure that is identical to or similar to the matrix structure used to represent the structured sequences 828 and / or the combinatorial antibody sequences 818. In various embodiments, a matrix produced by the trained model 830 that includes the amino acid sequences included in the combinatorial antibody sequence library 834 can be decoded to generate strings of amino acids corresponding to the sequences of the antibodies.

[0122] In some embodiments, the amino acid sequences included in the combinatorial antibody sequence library 834 can be subjected to one or more filter operations. The one or more filter operations can analyze the amino acid sequences included in the combinatorial antibody sequence library for one or more sequences corresponding to one or more particular properties. For example, the amino acid sequences included in the combinatorial antibody sequence library 834 can be analyzed to identify sequences with particular amino acids at particular positions. The amino acid sequences included in the combinatorial antibody sequence library 834 can also be analyzed to identify one or more sequences with one or more particular amino acid strings at one or more positions. In various embodiments, the amino acid sequences included in the combinatorial antibody sequence library 834 can be analyzed to identify one or more sequences with a set of biophysical properties based on similarity between at least one of the sequences included in the combinatorial antibody sequence library 834 and the amino acid sequence of an additional antibody known to have a set of biophysical properties.

[0123] In one or more embodiments, the amino acid sequences generated by the trained model 830 and / or the amino acid sequences generated during training of the light chain generation element 804 and the heavy chain generation element 806 may be evaluated according to one or more criteria. For example, the amino acid sequences that generated the trained model 830 and / or the amino acid sequences generated during training of the light chain generation element 804 and the heavy chain generation element 806 may be evaluated based on at least one of a match with an amino acid sequence generated for one or more germline genes, a measure of immunogenicity of the amino acid sequence, or a match with a CDR H3 amino acid sequence. A PCA model may be used to determine when to stop training at least one of the light chain generation element 804 or the heavy chain generation element 806 with respect to the correspondence with the CDR H3 region. In various examples, the measure of immunogenicity may correspond to MHC class II binding affinity.

[0124] FIG. 9 is a diagram illustrating a framework 900 using transfer learning techniques to generate an antibody amino acid sequence from paired antibody heavy and light chain amino acid sequences, according to an embodiment. The framework 900 may include a first generating adversarial network 902. The first generating adversarial network 902 may include a first light chain generating element 904, a first heavy chain generating element 906, a first challenging element 908, and a first sequence combining element 910. In various embodiments, the first challenging element 908 may be a classifier. The first light chain generating element 904 may implement a model to generate an antibody light chain amino acid sequence based on inputs provided to the first light chain generating element 904. The first heavy chain generating element 906 may implement a model to generate an antibody heavy chain amino acid sequence based on inputs provided to the first heavy chain generating element 906. The first light chain generating element 904 and the first heavy chain generating element 906 may use input data 912 for generating the amino acid sequence. The input data 912 may include a vector generated using a random or pseudo-random generator. In an example embodiment, the input data 912 may include a noise signal that includes a series of numbers.

[0125] The first sequence combination element 910 can combine the amino acid sequences generated by the first light chain generation element 904 and the amino acid sequences generated by the first heavy chain generation element 906 to generate a combined antibody sequence. The first sequence combination element 910 can provide the combined antibody sequence to the first challenging element 908. The first challenging element 908 can then generate an output indicating whether the combined antibody sequence satisfies various properties. The output generated by the first challenging element 908 can be provided as feedback to at least one of the first light chain generation element 904 and the first heavy chain generation element 906. In this manner, one or more models implemented by the first light chain generation element 904 and / or the first heavy chain generation element 906 can be modified based on the feedback provided by the first challenging element 908. In various embodiments, the first challenging element 908 can compare the amino acid sequence generated by the first sequence combined with element 910 to amino acid sequences of antibodies corresponding to the training data of the first generated adversarial network 902 and generate an output indicative of the amount of correspondence between the amino acid sequence generated by the first sequence combined with element 910 and the amino acid sequences of antibodies included in the training data. The training data can include antibody sequence data 914. The antibody sequence data 914 can correspond to amino acid sequences of a number of antibodies. For a given antibody, the antibody sequence data 914 can include pairings of antibody light chains and antibody heavy chains. In an illustrative example, the antibody sequence data 914 can include amino acid sequences of antibodies produced by one or more mammals. The antibody sequence data 914 can also include amino acid sequences of one or more isotypes of classes of antibodies, such as IgA antibodies, IgD antibodies, IgE antibodies, IgG antibodies, and / or IgM antibodies.

[0126] The first generative adversarial network 902 can be trained in the same or similar manner as described with respect to the generative adversarial network 802 of FIG. 6. For example, at least a portion of the antibody sequence data 914 can be provided to the first challenging component 908 and compared to the output generated by the first sequence combination component 910. The output generated by the first sequence combination component 910 can be based on the amino acid sequence of the light chain of the antibody generated by the first light chain generation component 904 and the amino acid sequence of the heavy chain of the antibody generated by the first heavy chain generation component 906. The trained model 916 can be generated in response to iteratively determining parameters and / or weights for one or more functions implemented by at least one of the first light chain generation component 904 or the first heavy chain generation component 906. By way of example, the trained model 916 can include a trained light chain generation component 918 and a trained heavy chain generation component 920.

[0127] In various examples, the amino acid sequences generated by the trained model 916 may be further refined. Illustratively, the trained model 916 may be modified by being subjected to another training process using a different set of training data than the training data used in the initial training process. For example, the data used for further training of the trained model 916 may include a subset of the data used to generate the first trained model 916. In further examples, the data used for further training of the trained model 916 may include a different set of data than the data used to generate the first trained model 916. In illustrative examples, the trained model 916 may be further refined to generate amino acid sequences of antibodies with one or more specific properties. The one or more identified attributes may include values ​​of one or more biophysical properties and / or one or more expression levels. In these scenarios, the trained model 916 may be further trained using a training dataset that includes amino acid sequences of antibodies with one or more specific properties.

[0128] In the illustrative example of FIG. 9 , the improvement of the trained model 916 can be represented by training a second generative adversarial network 922 that includes the trained model 916. For example, the second generative adversarial network 922 can include a second light chain generation element 924 that corresponds to the trained first light chain generation element 918, and a second heavy chain generation element 926 that corresponds to the trained first heavy chain generation element 920. In various embodiments, the second light chain generation element 924 can include the trained light chain generation element 918 after one or more modifications have been made to the trained light chain generation element 918. Additionally, the second heavy chain generation element 926 can include the trained heavy chain generation element 920 after one or more modifications have been made to the trained heavy chain generation element 920. Changes can be made to the trained light chain generation element 918 in relation to the architecture of the trained light chain generation element 918, such as, for example, one or more additional hidden layers or changes to one or more network filters. The second generative adversarial network 922 may also include a second arrangement that combines element 928 with a second challenging element 930. The second challenging element 930 may include a classifier.

[0129] The first further input data 932 may be provided to the second light chain generation element 924, which may generate one or more light chain sequences 934. The first further input data 932 may include a random or pseudorandom sequence that the second light chain generation element 924 uses to generate the light chain sequences 934. Additionally, the second further input data 936 may be provided to the second heavy chain generation element 926, which may generate one or more heavy chain sequences 938. The second further input data 932 may include a random or pseudorandom sequence that the second heavy chain generation element 926 uses to generate the heavy chain sequences 938. The second sequence combination element 928 may combine one or more light chain sequences 934 with one or more heavy chain sequences 938 to generate one or more combined sequences 940. One or more of the combined sequences 940 can correspond to the amino acid sequence of an antibody that includes at least one light chain and at least one heavy chain.

[0130] The second challenging element 930 may generate a classification output 942 indicating that the amino acid sequences included in the combined sequences 940 satisfy various properties or that the amino acid sequences included in the combined sequences 940 do not satisfy various properties. In an exemplary embodiment, the second challenging element 930 may generate a classification output 942 based on the similarities and differences between one or more of the combined sequences 940 and the amino acid sequences provided as training data to the second challenging element 930. The classification output 942 may indicate an amount of similarity or an amount of difference between the combined sequences 940 and the training data amino acid sequences provided to the second challenging element 930.

[0131] The amino acid sequences provided to the second challenging element 930 as training data can be included in the further antibody sequence data 944. The further antibody sequence data 944 can include amino acid sequences of proteins with one or more particular properties. For example, the further antibody sequence data 944 can include amino acid sequences of antibodies with a threshold level of expression in humans. In a further example, the further antibody sequence data 944 can include amino acid sequences of antibodies with one or more biophysical properties and / or one or more structural properties. By way of example, the antibodies included in the further antibody sequence data 944 can have negatively charged regions, hydrophobic regions, a relatively low probability of aggregation, a particular ratio of high molecular weight (HMW), a melting temperature, one or more combinations thereof, and the like. In one or more further examples, the further antibody sequence data 944 can include binding affinity information that can be used in transfer learning. In one or more examples, the further antibody sequence data 944 can correspond to antibodies with at least a threshold amount of binding affinity for one or more molecules, such as MHC class II molecules. In various examples, the additional antibody sequence data 944 may include a subset of the antibody sequence data 914 used to generate the trained model 916. By providing amino acid sequences to the second challenging element 930 that have one or more particular properties, the second light chain generation element 924 and the second heavy chain generation element 926 may be trained to produce amino acid sequences of antibodies that have at least a threshold probability of having one or more particular properties.

[0132] Furthermore, in many situations where it is desired to produce an antibody amino acid sequence with one or more specific properties, the number of sequences available for training the generative allele network may be limited. In such situations, the accuracy, efficiency and / or effectiveness of the generative allele network for generating an antibody amino acid sequence with a specific property may not be satisfactory. Thus, if there is not a sufficient number of amino acid sequences available for training the generative allele network, the amino acid sequence generated by the generative allele network may not have the desired properties. By implementing the techniques and systems described with respect to FIG. 9, the first generative allele network 902 can perform part of the process of training a model to generate an antibody with one or more specific properties, and further training is performed in an accurate and efficient manner with the second generative allele network 922 to generate an antibody amino acid sequence with one or more specific properties.

[0133] Prior to being provided to the second challenging element 930, the amino acid sequences included in the further antibody sequence data 944 may be subjected to data pre-processing 946 that generates a structured sequence 948. For example, the further protein sequence data 944 may be arranged according to a classification system prior to being provided to the second challenging element 930. The data pre-processing 946 may include paired amino acids included in the amino acid sequences of the antibodies included in the further protein sequence data 944 along with numerical values ​​that may represent structure-based positions within the antibody. The combined sequence 940 may also be represented using vectors following the same or similar number scheme as used for the structured sequence 948.

[0134] After the second generative adversarial network 922 has undergone a training process, a modified trained model 950 can be generated that can generate an amino acid sequence of an antibody. The modified trained model 950 can represent the trained model 916 after being trained with additional protein sequence data 944. Additional sequence inputs 952 can be provided to the modified trained model 950, which can generate an antibody sequence 954. The additional sequence inputs 952 can include a random or pseudo-random sequence of a particular length. In a further embodiment, the antibody sequence 954 can be evaluated to determine whether the antibody sequence 954 comprises a particular set of properties. The evaluation of the antibody sequence 954 can generate an indication of a property of the antibody sequence 954, such as a biophysical property of the antibody, a biophysical property of a region of the antibody, and / or the presence or absence of an amino acid at a particular position of the antibody.

[0135] The illustrative example of FIG. 9 shows training of a model using multiple training sets in a framework that includes two generative adversarial networks. In further embodiments, training of a model using multiple training datasets can also be represented using a single generative adversarial network. Furthermore, although the illustrative example of FIG. 9 shows training of a model using a generative adversarial network with two training datasets, in various embodiments, two or more datasets can be used to train a model using one or more generative adversarial networks according to the implementations described herein.

[0136] 9 illustrates a second generating adversarial network 922 using a trained model 916 with a light chain generating element 918 separate from the trained heavy chain generating element 920, in further embodiments, the trained model 916 used as a generating element by the second generating adversarial network 922 may be a single generating element that may be used to generate amino acid sequences that include both light and heavy chains. In these embodiments, the second generating adversarial network 922 may include a single generating element in place of both the second light chain generating element 924 and the second heavy chain generating element 926, and the second sequence generating element 928 may not be present in the second generating adversarial network 922. In various embodiments, in scenarios where interactions between amino acids of the light and heavy chains introduce one or more complexities that are more efficiently captured using a generating adversarial network with a single generating element, a single generating element may be implemented by the second generating adversarial network 922 in place of separate light and heavy chain generating elements. Furthermore, in one or more embodiments, additional layers can be added to the generation of the amino acid sequences of the antibody to the second generated allele network 922. In various embodiments, one or more additional layers can be added to the second generated allele network 922 after or as part of the second sequence combination element 928 to generate the combined sequences 940.

[0137] FIG. 10 illustrates a framework 1000 for linking the amino acid sequences of the heavy and light chains of an antibody, according to some embodiments. The framework 1000 may include a light chain generation component 1002 that generates data corresponding to a first amino acid sequence 1004 of the light chain of the antibody. The light chain generation component 1002 may be part of a generative adversarial network. Furthermore, the light chain generation component 1002 may implement one or more first models in generating the data corresponding to the amino acid sequence of the light chain of the antibody. The one or more first models may include one or more functions with one or more variables, one or more parameters, one or more weights, or one or more combinations thereof. The light chain generation component 1002 may generate the data corresponding to the amino acid sequence of the light chain of the antibody based on input data obtained by the light chain generation component 1002. The input data may include numerical data generated by a random generator or a pseudorandom generator.

[0138] The framework 1000 may also include a heavy chain generation component 1006 that generates data corresponding to a second amino acid sequence 1008 of a heavy chain of the antibody. The heavy chain generation component 1006 may be part of a generative adversarial network. In various embodiments, the heavy chain generation component 1006 may implement one or more second models in generating the data corresponding to the amino acid sequence of the heavy chain of the antibody. The one or more second models may include one or more additional functions with one or more variables, one or more parameters, one or more weights, or one or more combinations thereof. The heavy chain generation component 1006 may generate the data corresponding to the amino acid sequence of the heavy chain of the antibody based on additional input data obtained by the heavy chain generation component 1006. The additional input data may include numerical data generated by a random or pseudorandom generator.

[0139] Additionally, the framework 1000 may include a linking element 1010 that combines the first amino acid sequence 1004 and the second amino acid sequence 1008 to generate data corresponding to a third amino acid sequence 1012. The linking element 1010 may add the second amino acid sequence 1008 to the first amino acid sequence 1004. For example, the first amino acid sequence 1004 may include a first string with each character in the first string representing an amino acid at each position of the light chain of the antibody. Additionally, the second amino acid sequence 1008 may include a second string with each character in the second string representing an amino acid at each position of the heavy chain of the antibody. The third amino acid sequence 1012 generated by the linking element 1010 may include a third string generated by appending the second string included in the second amino acid sequence 1008 to the last string of the first string included in the first amino acid sequence 1004. For example, first amino acid sequence 1004 ends with VESG and second amino acid sequence 1008 begins with EIQM. Linking element 1010 can combine first amino acid sequence 1004 and second amino acid sequence 1008 by adding second amino acid sequence 1008 beginning with EIQM after VESG of first amino acid sequence 1004. In this manner, third amino acid sequence 1012 includes a number of amino acids corresponding to the combination of a first number of amino acids in first amino acid sequence 1004 and a second number of amino acids in second amino acid sequence 1008.

[0140] The third amino acid sequence 1012 may be provided to a challenging element 1014, which may evaluate the third amino acid sequence 1012 against training data 1016. The challenging element 1014 may be included in a generative adversarial network. In an exemplary embodiment, the challenging element 1014 may be a classifier of the generative adversarial network. The training data 1016 may include amino acid sequences of antibodies. The amino acid sequences included in the training data 1016 may correspond to antibodies produced by various organisms and analyzed to determine the amino acid sequences of the antibodies. In various examples, the training data 1016 may include at least one of an amino acid sequence of an antibody light chain, an amino acid sequence of an antibody heavy chain, or an amino acid sequence of a combination of an antibody light chain and an antibody heavy chain. By evaluating the amino acid sequence generated by the linking element 1010, e.g., the third amino acid sequence 1012, against the training data 1016, the challenging element 1014 may generate a classification output 1018. The classification output 1018 may correspond to a measure of similarity between the third amino acid sequence 1012 and the amino acid sequences included in the training data 1016 .

[0141] In various examples, the classification output 1018 may be provided to at least one of the light chain generation component 1002 or the heavy chain generation component 1006. The light chain generation component 1002 and / or the heavy chain generation component 1006 may utilize the classification output 1018 to modify one or more models implemented by the light chain generation component 1002 and / or the heavy chain generation component 1006. In this manner, one or more models implemented by the light chain generation component 1002 and / or the heavy chain generation component 1006 may be modified to generate antibody light chain and / or antibody heavy chain amino acid sequences that correspond to the amino acid sequences included in the training data 1016.

[0142] In one or more scenarios, the framework 1000 may include one or more additional computational layers 1020. The additional computational layers 1020 may modify the output from the linking element 1016 before the output from the linking element 1010 is provided to the challenging element 1014. In various examples, the additional computational layers 1020 may be part of the linking element 1010. The additional computational layers 1020 may be utilized in situations where a relationship between an amino acid sequence and one or more biophysical properties is not accounted for by the linking element 1010. Additionally, the one or more additional computational layers 1020 may be utilized in situations where a non-linear relationship exists between the heavy chain amino acid sequence and the light chain amino acid sequence generated by the light chain generation element 1002 and the heavy chain generation element 1006. Additionally, the one or more additional computational layers 1020 may be utilized in scenarios where there are various interactions between the first amino acid sequence 1004 and the second amino acid sequence 1008 that may be captured by the one or more additional computational layers 1020.

[0143] 11-14 illustrate an exemplary method for generating amino acid sequences of proteins using machine learning techniques. The exemplary process is illustrated as a collection of blocks in a logic flow graph that represents a sequence of operations that may be implemented in hardware, software, or a combination thereof. The blocks are referenced by numerals. In the context of software, the blocks represent computer-executable instructions stored on one or more computer-readable media, which perform the recited operations when executed by one or more processing units (such as hardware microprocessors). Generally, computer-executable instructions include routines, programs, objects, elements, data structures, etc., that perform a particular function or implement a particular data type. The order in which the operations are described is not intended to be construed as a limitation, and any number of the described blocks can be combined in any order and / or in parallel to perform the process.

[0144] FIG. 11 is a flow diagram illustrating another exemplary method 1100 for generating protein sequences, according to some implementations. At 1102, the method 1100 includes obtaining a training dataset including amino acid sequences of proteins. The training dataset can be obtained by extracting amino acid sequences of proteins from one or more databases. In various embodiments, the amino acid sequences included in the training dataset can correspond to proteins with one or more properties. For example, the amino acid sequences included in the training dataset can include one or more structural properties. In a further example, the amino acid sequences included in the training dataset can include one or more biophysical properties. In a further example, the amino acid sequences included in the training dataset can have one or more regions that include a particular amino acid sequence.

[0145] At 1104, the method 1100 includes generating encoded amino acid sequences based on the training dataset. In various embodiments, the encoded amino acid sequences can be generated by applying a classification system to the amino acid sequences included in the training dataset. In an example, the classification system can identify one or more regions of the amino acid sequences. Further, generating the encoded amino acid sequences can include generating a matrix for each amino acid sequence indicating the amino acids included at each position of the individual amino acid sequences.

[0146] At 1106, the method 1100 includes generating a model to generate additional amino acid sequences corresponding to the amino acid sequences in the training set. The model can be generated using the encoded amino acid sequences generated from the training data set. Additionally, a generative allele network can be used to generate the model. In various embodiments, the model can be used to generate amino acid sequences of proteins with one or more properties that are the same or similar to at least one property of the proteins corresponding to the amino acid sequences in the training data set.

[0147] At 1108, the method 1100 may include generating additional amino acid sequences using the model and the input vector. In an example, the input vector may include a random or pseudorandom sequence. Additionally, at 1110, the method 1100 may include evaluating the additional amino acid sequences according to one or more criteria and determining an index of the additional amino acid sequences. The techniques and operations utilized to evaluate the additional amino acid sequences may be different than those utilized by the generative allele network to generate the model. In an embodiment, computer readable instructions, such as instructions associated with a software tool or software platform, may be executed to evaluate the additional amino acid sequences. The additional amino acid sequences may be evaluated to determine whether a protein corresponding to the additional amino acid sequences has one or more particular properties. In certain embodiments, the additional amino acid sequences may be evaluated to determine the number of variations of each additional amino acid sequence from the amino acid sequence of the protein derived from the germline gene.

[0148] 12 is a flow diagram illustrating another exemplary method 1200 for generating antibody sequences, according to an embodiment. At 1202, method 1200 includes obtaining a training data set including amino acid sequences of antibodies. The amino acid sequences may be obtained from one or more databases that store amino acid sequences of antibodies.

[0149] At 1204, method 1200 includes generating a model to generate additional amino acid sequences of antibodies with one or more properties similar to the properties of the antibodies in the training dataset. The model may be generated using a generative allele network. In an embodiment, the additional amino acid sequence may correspond to an antibody with one or more particular structural properties. Additionally, the additional amino acid sequence may correspond to an antibody with one or more biophysical properties. In yet a further example, the additional amino acid sequence may correspond to an amino acid sequence of an antibody derived from a germline gene.

[0150] At 1206, the method 1200 may include generating additional amino acid sequences using the model and the input vector. In various scenarios, the input vector may include a random or pseudorandom sequence of a particular length. The model can take the input vector and use it to generate an output that corresponds to an amino acid sequence of an antibody.

[0151] FIG. 13 is a flow diagram illustrating an exemplary method 1300 of generating an amino acid sequence of a protein that binds to a particular target molecule, according to some embodiments. The method 1300 may include, at operation 1302, obtaining first data indicative of a composition of the target molecule. The target molecule may correspond to a protein from which a protein that binds to the target molecule is being generated. The composition of the target molecule may correspond to a sequence of atoms that comprise the target molecule. In various examples, the composition of the target molecule may include a sequence of a subgroup of atoms that comprise the target molecule. For example, the target molecule may include a protein, and the composition of the target molecule may include a sequence of amino acids. In an illustrative example, the target molecule may include an antigen, and the method 1300 may be directed to generating an amino acid sequence of an antibody that binds to the antigen. In a further illustrative example, the target molecule may include a substrate, and the method 1300 may be directed to generating an amino acid sequence of an enzyme that binds to the substrate.

[0152] Further, at 1304, the method 1300 may include obtaining second data indicative of binding interactions between each of the first proteins of the group of first proteins and one or more additional molecules. The second data may include experimentally obtained data indicative of binding between the first proteins and one or more additional molecules. In various examples, the second data may be simulated and computationally derived to indicate binding between the first proteins and one or more additional molecules. The binding interactions may include at least one of binding affinity or avidity. In various examples, the binding interactions may indicate a sequence of amino acids included in a binding region of an antibody and a sequence of additional amino acids in an epitope region of an antigen, where the binding region has at least a threshold amount of binding interactions with the epitope region. Furthermore, the binding interactions may indicate coupling between amino acids included in the binding region and additional amino acids located in the epitope region. In an exemplary embodiment, the second data may include an equilibrium constant between each of the first proteins and one or more additional molecules. Illustratively, the second data may include an equilibrium dissociation constant between each of the first proteins and one or more additional molecules. The second data may indicate that each first protein can bind to one other molecule. The second data may also indicate that each first protein can bind to multiple other molecules. Furthermore, the second data may indicate that multiple first proteins can bind to a single additional molecule.

[0153] In various embodiments, the second data may indicate the portion of the first protein and the portion of the further molecule where binding occurs. For example, the second data may indicate the atoms of the first protein involved in the binding interaction with the atoms of the further molecule. If the other molecule comprises a protein, the second data may indicate the amino acids of the first protein and the amino acids of the other molecule involved in the binding interaction. Furthermore, in further embodiments, the first protein may comprise an antibody, and the further molecule may comprise an antigen. In these scenarios, the second data may indicate the amino acids contained in one or more binding regions of each of the first proteins and the amino acids contained in one or more epitope regions of each of the antigens.

[0154] The second data may be indicative of structural characteristics of the first protein and other molecules involved in the binding interaction. Illustratively, the second data may be indicative of functional groups of the first protein and functional groups of other molecules involved in the binding interaction. Furthermore, in situations where the additional molecule comprises a protein, the second data may be indicative of at least one of the secondary or tertiary structure of the first protein and the additional molecule involved in the binding interaction. In illustrative examples, the second data may be indicative of structures that are part of the binding interaction, such as a sheet, a helix, a bend, a coil, a turn, a bridge, a loop, or one or more combinations thereof.

[0155] Further, at 1306, the method 1300 may include determining a composition of the additional molecule that is similar to at least a threshold amount with respect to at least a portion of the composition of the target molecule. The amount of similarity between the target molecule and the additional molecule may be determined by determining the number of one or more atoms contained in the target molecule and the additional molecule and comparing the atom numbers. For example, the number of carbon atoms contained in the target molecule and the number of carbon atoms contained in the additional molecule can be compared to each other. Continuing with this example, the amount of difference between the number of carbon atoms contained in the target molecule and the number of carbon atoms in the additional molecule can correspond to the amount of similarity between the target molecule and the additional molecule.

[0156] Furthermore, the amount of similarity between the target molecule and the further molecule can be determined by determining the functional groups of the target molecule and the further molecule and comparing the number and / or position of one or more types of functional groups contained in the target molecule and the further molecule. In this situation, the amount of similarity between the target molecule and the further molecule can be based on the difference between the number of one or more functional groups of the further molecule and the number of functional groups of the further molecule. For example, the amount of similarity between the target molecule and the further molecule can be based on the difference between the number of aldehyde groups contained in the target molecule and the number of aldehyde groups contained in the further molecule. In another example, the number of aromatic groups contained in the target molecule and the number of aromatic groups contained in the further molecule can be used to determine the amount of similarity between the target molecule and the further molecule. The difference and / or similarity between the positions of the functional groups can also be used to determine the amount of similarity between the target molecule and the further molecule. For example, the carboxyl groups located at carbon positions 2 and 10 of the target molecule can be compared to the position of the carboxyl group of the further molecule. Continuing with this example, the degree of similarity between the target molecule and the further molecule can be based on whether the carboxyl groups of the further molecule are also located at positions 2 and 10.

[0157] In a scenario where the target molecule and the further molecule are proteins, the amount of similarity between the target molecule and the further molecule can be determined by comparing the amino acid sequence of the further molecule and the amino acid at each position of the amino acid sequence of the further molecule. In an embodiment, the individual amino acids at each position of the amino acid sequence of the target molecule can be compared to the individual amino acids at each position of the amino acid sequence of the further molecule. In a further embodiment, the individual amino acids at the positions of one or more regions of the target molecule can be compared to the individual amino acids at the positions of one or more regions of the further molecule that correspond to one or more regions of the target molecule. For example, the amino acids located in an epitope region of the target molecule can be compared to the amino acids located in one or more regions of the further molecule that may correspond to the epitope region. The number of positions of at least a portion of the amino acid sequence of the target molecule that have amino acids that are identical to at least a corresponding portion of the amino acid sequence of the further molecule can correspond to the amount of identity between the further molecule and the target molecule. In that situation, the amount of similarity between the further molecule and the target molecule can correspond to the amount of identity between the amino acid sequence of one or more portions of the further molecule and the target molecule.

[0158] Although the amount of similarity between the target molecule and the additional target molecule has been described with reference to various examples, the amount of similarity between the target molecule and the additional target molecule can be determined based on one or more combinations of a number of criteria.For example, the amount of similarity between the target molecule and the additional molecule can be determined by analyzing at least one of the following: a number of atoms contained in the target molecule and the additional molecule, a number of bond sequences (e.g., single bonds, double bonds, triple bonds) contained in the target molecule and the additional molecule, a number of functional groups contained in the target molecule and the additional molecule, the position of secondary structural features of the target molecule and the additional molecule, amino acids contained in secondary structural features of the target molecule and the additional molecule, the additional molecule of the target molecule and the additional molecule, the tertiary structure of the target molecule and the additional molecule, or the identity of one or more regions of the target molecule and the additional molecule.

[0159] The threshold amount of similarity between the additional molecule and the target molecule can be based on the likelihood that a protein that binds to the target molecule also binds to the additional molecule. In embodiments where the target molecule and the additional molecule are antigens, the threshold amount of similarity can correspond to a minimum amount of identity between one or more regions of the target molecule and one or more regions of the additional molecule. In an illustrative example, the threshold amount of similarity between the target molecule and the additional target molecule can correspond to a minimum amount of identity between an epitope region of the target molecule with respect to one or more regions of the additional target molecule.

[0160] At 1308, the method 1300 may include determining a subset of the first group of proteins that have at least a threshold amount of binding interaction with the additional molecule. The threshold amount of binding interaction may correspond to a maximum equilibrium dissociation constant. In this situation, determining the subset of the first group of proteins may include determining an equilibrium dissociation constant between each first protein and the additional molecule. The equilibrium dissociation constant may be compared to a threshold equilibrium dissociation constant. If the equilibrium dissociation constant is less than the threshold equilibrium dissociation constant, the corresponding first protein may be added to the subset of the first group of proteins.

[0161] Further, at 1310, the method 1300 may include generating a model using the generative adversarial network to generate amino acid sequences of additional proteins having at least a threshold amount of binding interactions with the target molecule. The generative adversarial network may include a generator element that generates amino acid sequences of antibodies based on the antigen sequence. The generator element may use an input vector including noise data generated by a random or pseudo-random generator to generate the amino acid sequences. The amino acid sequences generated by the generator element may be evaluated by a challenging element of the generative adversarial network. The challenging element may be a classifier. The challenging element may evaluate the amino acid sequences produced by the generator element against antibody amino acid sequences with at least a threshold amount of binding to antigens that have at least a threshold amount of similarity with respect to the target antigen. For example, the challenging element may analyze the amino acid sequences generated by the generator element against amino acid sequences of a subset of the group of first proteins that have at least a threshold amount of binding interactions with the additional molecule determined in operation 1308. The model may include one or more functions with one or more variables, each variable with a respective weight.

[0162] The method 1300 may also include, at 1312, using the model to generate a plurality of second amino acid sequences of a second protein corresponding to the target molecule. For example, the target molecule may include an antigen, and the model may be used to generate amino acid sequences of an antibody that has at least a threshold probability of having at least a threshold binding interaction with the antigen. In an illustrative example, the model may be used to generate amino acid sequences of an antibody that has at least a threshold equilibrium dissociation constant with the antigen.

[0163] FIG. 14 is a flow diagram illustrating an exemplary method 1400 of generating an antibody amino acid sequence by combining separately generated amino acid sequences of an antibody heavy chain and a light chain, according to an embodiment. The method 1400 may include, at 1402, generating first data corresponding to a plurality of first amino acid sequences related to an antibody light chain using a generative adversarial network. The generative adversarial network may include a first generating element implementing a first model for generating the plurality of first amino acid sequences. The first model may include a first function having one or more first variables and one or more first weights. The antibody light chain may include at least one of an antibody light chain variable region or a constant region. Additionally, the antibody light chain may include an antibody light chain integrity determining region (CDR). The generated adversarial network may use input data to generate the plurality of first amino acid sequences. The input data may include a sequence of numbers generated by a random or pseudorandom generator.

[0164] At 1404, the method 1400 may include generating a plurality of second amino acid sequences corresponding to the heavy chain of the antibody using the generative adversarial network. The generative adversarial network may also include a second generating element implementing a second model to generate the plurality of second amino acid sequences. The second model may include a second function different from the first function. The second function may include one or more second variables and one or more second weights. The antibody heavy chain may include at least one of an antibody heavy chain variable region or a constant region. The antibody heavy chain may also include an antibody heavy chain CDR. The generative adversarial network may use additional input data in generating the plurality of second amino acid sequences. The additional input data may include a number sequence generated by a random or pseudorandom generator.

[0165] At 1406, method 1400 may include combining the first amino acid sequence with the second amino acid sequence using the generative allele network to generate a third amino acid sequence of the antibody comprising a light chain amino acid sequence and a heavy chain amino acid sequence. The first amino acid sequence may be combined with the second amino acid sequence by linking the second amino acid sequence to the first amino acid sequence. In one or more examples, the first amino acid sequence, the second amino acid sequence, and the third amino acid sequence may be coded according to a classification system.

[0166] At 1408, the method 1400 may include analyzing the third amino acid sequence with respect to the further amino acid sequences included in the training data by the generative adversarial network. The third amino acid sequence may be analyzed by a classifier, and an output may be provided to at least one of the first or second generation components. For example, based on the output by the classifier, the first generation component may modify a first model used to generate the first amino acid sequence. Furthermore, based on the output by the classifier, the second generation component may modify a second model used to generate the second amino acid sequence. In this manner, the output from the classifier may be used as feedback by at least one of the first or second generation components to generate amino acid sequences that are more likely to correspond to the further amino acid sequences included in the training data.

[0167] The output generated by the classifier over time may indicate progress in training the first model and training the second model. After the first model is completed, the first trained model may be used to generate an amino acid sequence for a light chain of the antibody, and after the second model is completed, the second trained model may be used to generate an amino acid sequence for a heavy chain of the antibody. The amino acid sequences generated by the first trained model and the second trained model may be combined, and the combined amino acid sequence may be analyzed by a software tool. The software tool may determine one or more metrics for the combined amino acid sequences. By way of example, the one or more metrics may include at least one of a number of hydrophobic amino acids, a number of positively charged amino acids, a number of negatively charged amino acids, a number of uncharged amino acids, a level of expression, a melting temperature, or a level of autoaggregation.

[0168] Furthermore, the first training model and the second training model can be further trained using further training data that is different from the initial training data used to generate the first training model and the second training model. For example, the further training data can include amino acid sequences of antibodies with one or more characteristics. For example, the further training data can include amino acid sequences of antibodies with hydrophobic regions, relatively low probability of aggregation, a certain ratio of high molecular weight (HMW), melting temperature, threshold level of expression, one or more combinations thereof, etc. In these embodiments, the output from the classifier of the generative adversarial network based on the further training data can be used to further modify the models implemented by the first and second generating elements such that the amino acid sequences generated by the generative adversarial network correspond to the amino acid sequences of the antibodies included in the further training data.

[0169] FIG. 15 is an example of a scheme for structurally aligning an antibody amino acid sequence for input to a generative machine learning architecture, according to an embodiment. The structure of FIG. 15 corresponds to the application of a classification system to the heavy chain domain of an antibody. In one or more exemplary examples, the classification system can assign 149 positions to encode the variable region of the antibody heavy chain. Furthermore, the classification system used to generate the structure 1500 in the exemplary example of FIG. 15 can assign 123 positions to encode the constant region of the antibody heavy chain. Furthermore, the classification system used to generate the structure 1500 in the exemplary example of FIG. 15 can assign 123 positions to encode the hinge region of the antibody heavy chain.

[0170] The amino acids bound to each position in the amino acid sequence may be represented by letters in structure 1500. Additionally, positions that are not bound to any amino acid may also be shown in structure 1500. Gaps in the amino acid sequence associated with structure 1500 may represent antibody structures that correspond to the structures shown in FIG.

[0171] In the illustrative example of Figure 15, structure 1500 may include a first region 1702 including amino acids of a first heavy chain framework region from a first position to a second position, and a second region 1704 including amino acids of a first heavy chain CDR from a second position to a third position. Additionally, structure 1500 may include a third region 1706 including amino acids of a second heavy chain framework region from a third position to a fourth position, and a fourth region 1708 including amino acids of a second heavy chain CDR from a fourth position to a fifth position. Additionally, structure 1500 may include a fifth region 1710 including amino acids of a third heavy chain framework region from a fifth position to a sixth position, and a sixth region 1712 including amino acids of a third CDR from a sixth position to a seventh position. The structure 1500 may also include a seventh region 1714 including amino acids of the fourth heavy chain framework region from position 7 to position 8, and an eighth region 1716 including amino acids of the fourth heavy chain constant region from position 8 to position 9. Additionally, the structure 1500 may include a ninth region 1718 including a hinge region of the heavy chain of the antibody from position 9 to position 10. In various examples, the structure 1500 may include a tenth region 1720 including amino acids of the third heavy chain constant region from position 11 to position 12, and an eleventh region 1722 including amino acids of the third heavy chain constant region from position 11 to position 12. Each region of the structure may include a predetermined number of positions, at least a portion of which may bind to a particular amino acid.

[0172] Figure 16 is an illustration of a machine 1600 in the form of a computer system on which a set of instructions may be executed that causes the machine 1600 to perform any one or more of the methodologies described herein, according to an exemplary embodiment. Specifically, Figure 16 shows a schematic diagram of a machine 1600 in the form of an exemplary computer system on which instructions 1624 (e.g., software, programs, applications, applets, applications, or other executable code) may be executed that causes the machine 1600 to perform any one or more of the methodologies described herein. For example, the instructions 1624 may cause the machine 1600 to perform the frameworks 100, 200, 300, 400, 500, 600, 700, 800 described with respect to Figures 1, 2, 3, 4, 5, 6, 7, 8, 9, and 10, respectively, and the methods 1100, 1200, 1300, and 1400 described with respect to Figures 11, 12, 13, and 14, respectively. Further, the coding shown in FIG. 15 may be generated using machine 1600 by instructions 1824.

[0173] The instructions 1824 transform the general, unprogrammed machine 1600 into a specific machine 1600 programmed to perform the functions described and illustrated in the manner described. In an embodiment, the machine 1600 may alternatively operate as a stand-alone device or be coupled (e.g., networked) to other machines. In a networked deployment, the machine 1600 may operate in the capacity of a server machine or a client machine in a server-client network environment, or as a peer machine in a peer-to-peer (or distributed) network environment. The machine 1600 may include, but is not limited to, a server computer, a client computer, a personal computer (PC), a tablet computer, a laptop computer, a netbook, a set-top box (STB), a personal digital assistant (PDA), an entertainment media system, a mobile phone, a smartphone, a mobile device, a wearable device (e.g., a smart watch), a smart home device (e.g., a smart appliance), other smart devices, a web appliance, a network router, a network switch, a network bridge, or any machine capable of executing, sequentially or otherwise, the instructions 1624 that specify the operations to be performed by the machine 1600. Further, although only a single machine 1600 is shown, the term "machine" should be interpreted to include a collection of machines 1600 that execute the instructions 1016, either separately or together, to perform any one or more of the methods discussed herein.

[0174] An example of a computing device 1600 may include logic, one or more elements, circuits (e.g., modules), or mechanisms. A circuit is a tangible entity configured to perform certain operations. In one embodiment, a circuit may be arranged in a predetermined manner (e.g., internally or relative to external entities such as other circuits). In one embodiment, one or more computer systems (e.g., stand-alone, client, or server computer systems) or one or more hardware processors (processors) may be configured with software (e.g., instructions, application portions, or applications) as circuitry that operates to perform certain operations described herein. In one embodiment, the software may reside (1) on a non-transitory machine-readable medium or (2) in a transmission signal. In one example, the software, when executed by the hardware underlying the circuitry, causes the circuitry to perform certain operations.

[0175] In one embodiment, the circuitry may be implemented mechanically or electronically. For example, the circuitry may include dedicated circuitry or logic specifically configured to perform one or more of the techniques described above, such as including special purpose processors, field programmable gate arrays (FPGAs), or application specific integrated circuits (ASICs). In one embodiment, the circuitry may include programmable logic (e.g., circuitry contained within a general purpose processor or other programmable processor) that can be temporarily configured (e.g., by software) to perform specific operations. It will be appreciated that the decision to implement a circuitry mechanically (e.g., in dedicated and permanently configured circuitry) or temporarily configured circuitry (e.g., configured by software) may be driven by cost and time considerations.

[0176] Thus, the term "circuitry" is understood to encompass tangible entities, i.e., entities that are physically constructed, permanently configured (e.g., hardwired), or temporarily (e.g., transitorily) configured (e.g., programmed) to behave in a particular way or perform particular operations. In one example, where multiple temporarily configured circuits are provided, each circuit need not be configured or instantiated at any one instance in time. For example, where a circuit includes a general-purpose processor configured via software, the general-purpose processor may be configured as different circuits at different times. Thus, the software may configure the processor, for example, to configure a particular circuit at one time and a different circuit at a different time.

[0177] In one embodiment, a circuit can provide information to and receive information from other circuits. In this example, a circuit can be considered to be communicatively coupled to one or more other circuits. If multiple such circuits are present simultaneously, communication can be achieved by signal transmission (e.g., via appropriate circuits and buses) connecting the circuits. In an embodiment in which multiple circuits are configured or instantiated at different times, communication between the circuits can be achieved, for example, by storing and retrieving information in a memory structure accessed by the multiple circuits. For example, one circuit can perform an operation and store the output of that operation in a memory device to which it is communicatively coupled. The other circuit can then access the memory device at a later time to retrieve and process the stored output. In one embodiment, a circuit can be configured to initiate or receive communication with an input device or an output device and can operate on a resource (e.g., a collection of information).

[0178] Various operations of example methods described herein may be performed, at least in part, by one or more processors that are configured temporarily (e.g., by software) or permanently configured to perform the associated operations. Such processors, whether configured temporarily or permanently, may constitute processor-implemented circuitry that operates to perform one or more operations or functions. In one embodiment, circuitry referred to herein may include processor-implemented circuitry.

[0179] Similarly, the methods described herein may be at least partially processor-implemented. For example, at least a portion of the operations of the methods may be performed by one or more processors or processor-implemented circuitry. Performance of certain operations may be distributed among one or more processors that are not only present within a single machine, but also located across multiple machines. In one embodiment, the processor or processors may be located in a single location (e.g., a home environment, an office environment, or a server farm), while in other embodiments, the processors may be distributed across multiple locations.

[0180] The one or more processors may also operate to support the performance of associated operations in a "cloud computing" environment or as a "software as a service."

[0181] (SaaS) For example, at least a portion of the operations may be performed by a collection of computers (as an example of machines including a processor) accessible via a network (e.g., the Internet) and one or more appropriate interfaces (e.g., application program interfaces (APIs)).

[0182] Exemplary embodiments (e.g., devices, systems, or methods) can be implemented in digital electronic circuitry, computer hardware, firmware, software, or any combination thereof. Exemplary embodiments can be implemented using a computer program product (e.g., a computer program tangibly embodied in an information carrier or machine-readable medium for execution by or to control the operation of a data processing apparatus, such as a programmable processor, a computer, or multiple computers).

[0183] The computer program may be written in any type of programming language, including compiled or interpreted languages, and may be deployed in any form, whether as a stand-alone program or as a software module, subroutine, or other unit suitable for use in a computing environment. The computer program may be deployed to be executed on one computer or on multiple computers at one site, or distributed across multiple sites and interconnected by a communications network.

[0184] In one embodiment, the operations may be performed by one or more programmable processors executing computer programs to perform functions by operating on input data and generating output. Examples of method operations may also be performed by, and examples of apparatus may be implemented as, special purpose logic circuitry (e.g., a field programmable gate array or an application specific integrated circuit).

[0185] A computing system may include clients and servers. Clients and servers are generally remote from each other and typically interact through a communication network. The relationship of client and server arises from computer programs running on the respective computers and have a client-server relationship to each other. It will be appreciated that in embodiments deploying a programmable computing system, both hardware and software architectures require consideration. In particular, it will be appreciated that the choice of whether to implement a particular function in permanently configured hardware (e.g., ASICs), temporarily configured hardware (e.g., a combination of software and programmable processors), or a combination of permanently and temporarily configured hardware may be a design choice. The following illustrates hardware (e.g., computing device 1600) and software architectures that may be deployed in an exemplary embodiment.

[0186] In one embodiment, computing device 1600 may operate as a stand-alone device or computing device 1600 may be connected (eg, networked) to other machines.

[0187] In a networked deployment, the computing device 1600 may operate in a server-client network environment in the capacity of either a server or a client machine. In one embodiment, the computing device 1600 may operate as a peer machine in a peer-to-peer (or other distributed) network environment. The computing device 1600 may The computing device 1600 may be a personal computer (PC), a tablet computer, a set-top box (STB), a personal digital assistant (PDA), a mobile phone, a wearable device, a network router, a network switch, a network bridge, or any machine capable of executing instructions that specify operations to be performed (e.g., to be performed) by the computing device 1600. Additionally, although only a single computing device 1600 is shown, the term "computing device" should be interpreted to include a collection of machines 1600 that separately or together execute a set (or sets) of instructions to perform any one or more of the methodologies discussed herein.

[0188] The exemplary computing device 1600 may include a processor 1602 (e.g., a central processing unit CPU), a graphics processing unit (GPU), or both), a main memory 1804, and a static memory 1806, some or all of which may communicate with each other via a bus 1608. The computing device 1600 may further include a display unit 1610, an alphanumeric input device 1612 (e.g., a keyboard), and a user interface navigation device 1614 (e.g., a mouse). In one embodiment, the display unit 1610, the input device 1612, and the UI navigation device 1814 may be touch screen displays. The computing device 1600 may further include a storage device (e.g., a drive unit) 1616, a signal generating device 1618 (e.g., a speaker), a network interface device 1620, and one or more sensors 1621, such as a global positioning system sensor, a compass, an accelerometer, or other sensor.

[0189] Storage device 1616 may include machine-readable medium 1622 on which is stored one or more sets of data structures or instructions (e.g., software) embodied or utilized by any one or more of the methods or functions described herein. Instructions 1624 may also reside, completely or at least partially, within main memory 1604, within static memory 1606, or within processor 1602 during execution by computing device 1600. In an embodiment, one or any combination of processor 1602, main memory 1604, static memory 1606, or storage device 1616 may constitute a machine-readable medium.

[0190] Although the machine-readable medium 1622 is depicted as a single medium, the term "machine-readable medium" may include a single medium or multiple media (e.g., a centralized or distributed database and / or associated caches and servers) configured to store one or more instructions 1624, and the term "machine-readable medium" may also include any tangible medium that can store, encode, or carry instructions for execution by a machine, cause a machine to perform any one or more of the methods of the present disclosure, or store, encode, or carry data structures used by or associated with such instructions. The term "machine-readable medium" includes, but is not limited to, solid-state memories, and optical and magnetic media. Particular examples of machine-readable media may include non-volatile memory, including, for example, semiconductor memory devices (e.g., electrically programmable read-only memory).

[0191] EPROM, Electrically Erasable Programmable Read-Only Memory (EEPROM), flash memory devices, built-in hard disks, magnetic disks such as removable disks, magneto-optical disks, CD-ROMs, and DVD-ROMs

[0192] The instructions 1624 may further be transmitted or received over a communications network 1626 using a transmission medium via the network interface device 1820 using any one of a number of transport protocols (e.g., Frame Relay, IP, TCP, UDP, HTTP, etc.). Examples of communications networks include local area networks (LANs), wide area networks (WANs), packet data networks (e.g., the Internet), mobile telephone networks (e.g., cellular networks), Plain Old Telephone (POTS) networks, and wireless data networks (e.g., the IEEE 802.11 family of standards known as Wi-Fi®, the IEEE 802.16 family of standards known as WiMax®), and peer-to-peer (P2P) networks. The term "transmission media" includes intangible media capable of storing, encoding, or transmitting instructions for execution by a machine, including digital or analog communication signals or other intangible media to facilitate communication of the software.

[0193] Exemplary embodiments 1. The method of claim 1, further comprising: acquiring, by a computing system including one or more computing devices having one or more processors and memory, first data indicative of a first amino acid sequence of a first antigen; acquiring, by the computing system, second data indicative of binding interactions between individual antibodies of a first plurality of antibodies and one or more antigens; determining, by the computing system, a second amino acid sequence of a second antigen of the one or more antigens that has at least a threshold amount of identity with respect to at least a portion of the first amino acid sequence of the first antigen; and determining, by the computing system and based on the second data, determining a group of antibodies from the first plurality of antibodies that have at least a first threshold amount of binding interaction with the second antigen; generating, by the computing system and using a generative adversarial network, a model for generating amino acid sequences of additional antibodies that have at least a second threshold probability of having at least a second threshold amount of binding interaction with the first antigen, wherein the model is generated based on the first amino acid sequence of the first antigen and the group of the first plurality of antibodies; and then generating, by the computer system and using the model, a second plurality of amino acid sequences of antibodies corresponding to the first antigen.

[0194] 2. The method of 1, wherein the binding interaction comprises at least one of binding affinity or avidity.

[0195] 3. The method according to 1 or 2, wherein the binding interaction represents an amino acid sequence of a binding region of one antibody of the first plurality of antibodies and a further amino acid sequence of an epitope region of one antigen of the one or more antigens, and the binding region binds to said epitope region.

[0196] 4. The method according to 3, wherein the binding interaction indicates a coupling between an amino acid contained in the binding region and a further amino acid contained in the epitope region.

[0197] 5. The method of any one of 1 to 4, wherein the binding interaction comprises an equilibrium constant between an individual antibody of the first plurality of antibodies and said one or more antigens.

[0198] 6. The method according to any one of 1 to 5, further comprising a step of evaluating one or more indicators for the amino acid sequences of the second plurality of antibodies using a computer system and software tools, wherein the one or more indicators include at least one of the following: the number of hydrophobic amino acids contained in each amino acid sequence of the second plurality of amino acid sequences; the number of positively charged amino acids contained in each amino acid sequence of the second plurality of amino acid sequences; the number of negatively charged amino acids contained in each amino acid sequence of the second plurality of amino acid sequences; the number of uncharged amino acids contained in each amino acid sequence of the second plurality of amino acid sequences, the expression level of each antibody, the melting temperature of each antibody, or the level of autoaggregation of each antibody.

[0199] 7. The method of claim 1, further comprising: acquiring, by a computer system including one or more computing devices having one or more processors and a memory, first data indicative of a composition of a target molecule; acquiring, by said computing system, second data indicative of binding interactions between each first protein of a plurality of first proteins and one or more additional molecules; and determining, by said computer system, compositions of the one or more additional molecules that have at least a threshold amount of similarity with respect to at least a portion of the composition of the target molecule; determining, by said computer system and based on said second data, a composition of the plurality of first proteins included in said second data. determining a group of the plurality of first proteins included in the second data that have at least a first threshold amount of binding interaction with the additional molecule; generating, by the computer system and using a generative adversarial network, a model for generating amino acid sequences of additional proteins that have at least a threshold probability of having at least a second threshold amount of binding interaction with the target molecule, wherein the model is generated based on the composition of the target molecule and the plurality of first protein groups, and generating, by the computer system and using the model, a plurality of second amino acid sequences of second proteins corresponding to the target molecule.

[0200] 8. The method of claim 7, wherein the target molecule comprises a protein.

[0201] 9. The method of claim 8, wherein the protein comprises an antigen, the plurality of first proteins comprises a first antibody, and the plurality of second proteins comprises a second antibody.

[0202] 10. A method for generating, by a computer system including one or more computing devices having one or more processors and memory, and using a generative adversarial network, a plurality of first amino acid sequences, each of the plurality of first amino acid sequences corresponding to a light chain of an antibody; generating, by the computer system and using the generative adversarial network, a plurality of second amino acid sequences, each of the plurality of second amino acid sequences corresponding to a heavy chain of an antibody; and generating, by the computing system and using the generative adversarial network, a plurality of second amino acid sequences, each of the plurality of second amino acid sequences corresponding to a heavy chain of an antibody. a step of combining a first amino acid sequence with a second amino acid sequence of a plurality of second amino acid sequences to generate a third amino acid sequence, the third amino acid sequence corresponding to an antibody comprising a light chain corresponding to the first amino acid sequence and a heavy chain corresponding to the second amino acid sequence; and a step of analyzing, by the computer system, the third amino acid sequence with respect to a further plurality of amino acid sequences using the generative adversarial network to generate an output, the further plurality of amino acid sequences being included in training data for the generative adversarial network, the output indicating a measure of similarity between the third amino acid sequence and at least a portion of the further plurality of amino acid sequences.

[0203] 11. The method of claim 10, wherein the step of combining the first amino acid sequence with the second amino acid sequence comprises linking the second amino acid sequence to the first amino acid sequence.

[0204] 12. The method of claim 10 or 11, wherein the generative opponent network includes a first generator element implementing a first model for generating a plurality of first amino acid sequences and a second generator element implementing a second model for generating a plurality of second amino acid sequences, the first model including a first function having one or more first variables and one or more first weights, and the second model including a second function different from the first function, the second function including one or more second variables and one or more second weights.

[0205] 13. The method of claim 12, wherein the third amino acid sequence is analyzed by a classifier and an output is provided to at least one of the first generation element or the second generation element.

[0206] 14. The method of claim 13, wherein the first generator modifies the first model based on the output.

[0207] 15. The method of claim 13, wherein the second generating element modifies the second model based on the output.

[0208] 16. The method of any one of 10 to 15, wherein the first amino acid sequence comprises at least a portion of a first variable region of an antibody light chain, and the second amino acid sequence comprises at least a portion of a first variable region of an antibody heavy chain.

[0209] 17. The method according to any one of 10 to 16, wherein the first amino acid sequence comprises at least a portion of a first variable region and a first constant region of an antibody light chain, and the second amino acid sequence comprises at least a portion of a second variable region and a second constant region of an antibody heavy chain.

[0210] 18. A method according to any one of claims 10 to 17, comprising: determining, by a computer system and based on the output, that training of a first model is complete, such that the first model is a first trained model; determining, by the computer system and based on the output, that training of a second model is complete, such that the second model is a second trained model; generating, by the computer system and using the first trained model, a first further amino acid sequence of a further antibody light chain; generating, by the computer system and using the second trained model, a second further amino acid sequence of a further antibody heavy chain; and combining, by the computer system, the first further amino acid sequence and the second further amino acid sequence to generate a third further amino acid sequence, wherein the third further amino acid sequence comprises a further antibody light chain and a heavy chain.

[0211] 19. The method of claim 18, further comprising a step of evaluating one or more indicators for the amino acid sequences of the third plurality of antibodies by a computer system, wherein the one or more indicators include at least one of the following: the number of hydrophobic amino acids contained in each amino acid sequence of the third plurality of amino acid sequences; the number of positively charged amino acids contained in each amino acid sequence of the third plurality of amino acid sequences; the number of negatively charged amino acids contained in each amino acid sequence of the third plurality of amino acid sequences; the number of uncharged amino acids contained in each amino acid sequence of the third plurality of amino acid sequences, the expression level of the third additional amino acid sequence, the melting temperature of the third additional amino acid sequence, or the level of self-aggregation of the third additional amino acid sequence.

[0212] 20. The method of claim 18, further comprising analyzing, by a computer system and using a generative adversarial network, a third additional amino acid sequence with respect to a further plurality of amino acid sequences and generating a further output, wherein the further plurality of amino acid sequences are included in further training data of the generative adversarial network, the further training data including amino acid sequences of an antibody that differ from the amino acid sequences included in the training data, and the further output indicating a further measure of similarity between the third additional amino acid sequence and at least a portion of the further plurality of amino acid sequences.

[0213] 21. One or more hardware processors; and when executed by said one or more hardware processors, causing said one or more processors to: 1. A system comprising: one or more non-transitory computer-readable media having stored thereon computer-executable instructions to perform the operations of: acquiring first data indicative of a first amino acid sequence of a first antigen; acquiring second data indicative of binding interactions between individual antibodies of a first plurality of antibodies and one or more antigens; determining a second amino acid sequence of a second antigen of the one or more antigens that has at least a threshold amount of identity for at least a portion of the first amino acid sequence of the first antigen; determining a group of antibodies from the first plurality of antibodies included in the second data that have at least a first threshold amount of binding interactions with the second antigen based on the second data; and generating a model using a generative adversarial network to generate amino acid sequences of additional antibodies that have at least a threshold probability of having a binding interaction with a first antigen of at least a second threshold, wherein the model is generated based on the first amino acid sequence of the first antigen and the group of antibodies of the first plurality; and using the model to generate amino acid sequences of the second plurality of antibodies corresponding to the first antigen.

[0214] 22. The system according to 21, wherein the binding interaction comprises at least one of binding affinity or avidity.

[0215] 23. The system according to 21 or 22, wherein the binding interaction represents an amino acid sequence of a binding region of one antibody of the first plurality of antibodies and a further amino acid sequence of an epitope region of one antigen of the one or more antigens, and the binding region binds to said epitope region.

[0216] 24. The system according to 23, wherein the binding interaction indicates coupling between an amino acid contained in the binding region and a further amino acid contained in the epitope region.

[0217] 25. The method of any one of 21 to 24, wherein the binding interaction comprises an equilibrium constant between an individual antibody of the first plurality of antibodies and said one or more antigens.

[0218] 26. A method according to any one of 1 to 5, wherein the operation includes a step of evaluating one or more indicators for the amino acid sequences of the second plurality of antibodies using a software tool, wherein the one or more indicators include at least one of the following: the number of hydrophobic amino acids contained in each amino acid sequence of the second plurality of amino acid sequences; the number of positively charged amino acids contained in each amino acid sequence of the second plurality of amino acid sequences; a number of negatively charged amino acids contained in each amino acid sequence of the second plurality of amino acid sequences; the number of uncharged amino acids contained in each amino acid sequence of the second plurality of amino acid sequences, the expression level of each antibody, the melting temperature of each antibody, or the level of autoaggregation of each antibody.

[0219] 27. A method for determining a composition of a target molecule comprising: one or more hardware processors; and, when executed by the one or more hardware processors, providing to the one or more processors: acquiring first data indicative of a composition of a target molecule; acquiring, by the computing system, second data indicative of a binding interaction between each first protein of a plurality of first proteins and one or more additional molecules; and determining, by the computing system and based on the second data, a composition of the one or more additional molecules that has at least a threshold amount of similarity with respect to at least a portion of the composition of the target molecule; determining, by the computing system and based on the second data, a composition of the one or more additional molecules that has at least a first threshold amount of similarity with respect to at least a portion of the composition of the target molecule. determining groups of the plurality of first proteins included in the second data that have at least a threshold probability of having a binding interaction of at least a second threshold amount with the target molecule; generating, by the computer system and using a generated adversarial network, a model to generate amino acid sequences of additional proteins that have at least a threshold probability of having a binding interaction of at least a second threshold amount with the target molecule, wherein the model is generated based on a composition of the target molecule and the plurality of first proteins, and generating, by the computer system and using the model, a plurality of second amino acid sequences of a second protein that correspond to the target molecule.

[0220] 28. The system according to claim 27, wherein the target molecule comprises a protein.

[0221] 29. The system of 28, wherein the protein comprises an antigen, the plurality of first proteins comprises a first antibody, and the plurality of second proteins comprises a second antibody.

[0222] 30. A system comprising one or more hardware processors and, when executed by the one or more hardware processors, the system includes: generating, by the one or more processors and using a generative adversarial network, a first amino acid sequence of each of the plurality of first amino acid sequences that corresponds to a light chain of an antibody; generating, by the computer system and using the generative adversarial network, a plurality of second amino acid sequences, each of the plurality of second amino acid sequences that corresponds to a heavy chain of an antibody; generating, by the computing system and using the generative adversarial network, a third amino acid sequence by combining a first amino acid sequence of the plurality of first amino acid sequences with a second amino acid sequence of each of the plurality of second amino acid sequences, the third amino acid sequence corresponding to an antibody comprising a light chain corresponding to the first amino acid sequence and a heavy chain corresponding to the second amino acid sequence; and analyzing, by the computer system, using the generative adversarial network, the third amino acid sequence with respect to a further plurality of amino acid sequences to generate an output, the further plurality of amino acid sequences being included in training data for a generative adversarial network, the output indicating a measure of similarity between the third amino acid sequence and at least a portion of the further plurality of amino acid sequences.

[0223] 31. The system according to 30, wherein the step of combining the first amino acid sequence with the second amino acid sequence comprises linking the second amino acid sequence to the first amino acid sequence.

[0224] 32. The system described in 30 or 31, wherein the generative opponent network includes a first generation element implementing a first model for generating a plurality of first amino acid sequences and a second generation element implementing a second model for generating a plurality of second amino acid sequences, the first model including a first function having one or more first variables and one or more first weights, and the second model including a second function different from the first function, the second function including one or more second variables and one or more second weights.

[0225] 33. The system of claim 32, wherein the third amino acid sequence is analyzed by a classifier and an output is provided to at least one of the first generation element or the second generation element.

[0226] 34. The system of claim 33, wherein the first generation element modifies the first model based on the output.

[0227] 35. The system of claim 33, wherein the second generation element modifies the second model based on the output.

[0228] 36. A system described in any one of 30 to 35, wherein the first amino acid sequence includes at least a portion of a first variable region of a light chain of the antibody, and the second amino acid sequence includes at least a portion of a first variable region of a heavy chain of the antibody.

[0229] 37. The system of any one of claims 30 to 36, wherein the first amino acid sequence comprises at least a portion of a first variable region and a first constant region of an antibody light chain, and the second amino acid sequence comprises at least a portion of a second variable region and a second constant region of an antibody heavy chain.

[0230] 38. The system of any one of claims 30 to 37, wherein the operations include determining, based on the output, that training of a first model is complete such that a first model is a first trained model; determining, by the computer system and based on the output, that training of a second model is complete such that a second model is a second trained model; generating, by the computer system and using the first trained model, a first further amino acid sequence of a further antibody light chain; generating, by the computer system and using the second trained model, a second further amino acid sequence of a further antibody heavy chain; and combining, by the computer system, the first further amino acid sequence and the second further amino acid sequence to generate a third further amino acid sequence, wherein the third further amino acid sequence comprises a further antibody light chain and a heavy chain.

[0231] 39. The method of claim 38, wherein the operation includes evaluating one or more indicators for the amino acid sequences of the third plurality of antibodies, wherein the one or more indicators include at least one of the following: the number of hydrophobic amino acids contained in each amino acid sequence of the third plurality of amino acid sequences; the number of positively charged amino acids contained in each amino acid sequence of the third plurality of amino acid sequences; the number of negatively charged amino acids contained in each amino acid sequence of the third plurality of amino acid sequences; the number of uncharged amino acids contained in each amino acid sequence of the third plurality of amino acid sequences, the expression level of the third additional amino acid sequence, the melting temperature of the third additional amino acid sequence, or the level of self-aggregation of the third additional amino acid sequence.

[0232] 40. The method of claim 38, further comprising: analyzing a third further amino acid sequence with respect to a further plurality of amino acid sequences using a generative adversarial network to generate a further output; wherein the further plurality of amino acid sequences are included in further training data for the generative adversarial network, the further training data including amino acid sequences of an antibody that differ from the amino acid sequences included in the training data; and wherein the further output indicates a further measure of similarity between the third further amino acid sequence and at least a portion of the further plurality of amino acid sequences.

[0233] 41. A method comprising the steps of: obtaining a training dataset comprising amino acid sequences of proteins; generating a structured amino acid sequence based on the training dataset; using the structured amino acid sequence and a generated allele network to generate a model for generating further amino acid sequences corresponding to the amino acid sequences included in the training dataset; generating the further amino acid sequences and using the model and an input vector to generate the further amino acid sequences; and evaluating the further amino acid sequences according to one or more indicators to determine an indicator for the further amino acid sequences.

[0234] 42. The method of claim 41, further comprising determining, for a protein derived from a germline gene, a number of amino acid sequence variations contained in the further amino acid sequence.

[0235] 43. The method of claim 41 or 42, wherein the generative adversarial network comprises a Wasserstein generative adversarial network.

[0236] 44. The method of any one of claims 41 to 43, wherein the structured amino acid sequence is represented by a matrix comprising a first number of rows and a second number of columns, each row of the first number of columns corresponding to a position in the sequence, and each column of the second number of columns corresponding to an individual amino acid.

[0237] 45. A method according to any one of 41 to 44, wherein one or more properties of the protein corresponding to the further amino acid sequence have at least a threshold similarity to one or more properties of the proteins included in the training dataset.

[0238] 46. ​​The method of any one of claims 41 to 45, wherein the one or more properties include at least one of a structural location property, a tertiary structure property, or a biophysical property.

[0239] 47. The method of any one of 41 to 46, wherein the protein comprises an antibody, an affibody, an affilin, an affimer, an affitin, an alphabody, an anchorin, an avimer, a monobody, a designed ankyrin repeat protein (DARPins), a nanoCLAMP (clostridal antibody mimetic protein), an antibody fragment, or a combination thereof.

[0240] 48. A method comprising: obtaining a training dataset comprising amino acid sequences of antibodies; generating a model using a generative adversarial network for generating additional amino acid sequences of antibodies having one or more properties similar to properties of the antibodies in the training dataset; and generating the additional amino acid sequences using the model and an input vector.

[0241] 49. The method of claim 48, further comprising applying a classification system to the amino acid sequences of the training dataset, said classification system indicating the number of first positions that bind to a heavy chain region of the antibody and the number of second positions that bind to a light chain region of the antibody.

[0242] 50. The method of 48 or 49, further comprising: generating a first model using a first generative adversarial network and a first training data set to generate a plurality of heavy chain regions of the antibody; generating a second model using a second generative adversarial network and a second training data set to generate a plurality of light chain regions of the antibody; combining at least a portion of the plurality of heavy chain regions with at least a portion of the light chain region to generate an antibody sequence.

[0243] 51. Training a first model of an element of a first generative adversarial network of a first generative adversarial network using a first training data set including a first number of amino acid sequences of a light chain of an antibody to generate a first trained model; training a second model of a second generative element of the generative adversarial network using a second training data set including a second number of amino acid sequences of a heavy chain of an antibody to generate a second trained model, wherein the training of the second generative element proceeds at a first rate different from a second rate of training of the first generative element; training a second model of an element of a first generative adversarial network of a first generative adversarial network using a second training data set including a second number of amino acid sequences of a heavy chain of an antibody to generate a second trained model, wherein the training of the second generative element proceeds at a first rate different from a second rate of training of the first generative element; generating a first further number of one further amino acid sequence using said second generator; generating a second further number of second further amino acid sequences corresponding to a heavy chain of an antibody; and combining a first amino acid sequence of said first further number of first amino acid sequences with a second amino acid sequence of said second further number of second amino acid sequences using said generator enemy network to generate a third further amino acid sequence, wherein said third amino acid sequence corresponds to an antibody comprising a light chain corresponding to said first amino acid sequence and a heavy chain corresponding to said second amino acid sequence.

[0244] 52. The method of claim 51, wherein a second generating element is trained using a number of Hobble weights to reduce the rate at which the second generating element is trained relative to the additional rate at which the second generating element is trained without the Hobble weights.

[0245] 53. The method of claim 51 or 52, wherein the second generating element is trained by slowing the gradient of the second generating element.

[0246] 54. A method according to any one of claims 51 to 53, comprising the steps of: training a second generator element during a first period of time; determining the quality of a first plurality of amino acid sequences generated at the end of the first period to be a first level; training the first generator element during a second period of time that includes the first period and is longer than the first period; determining the quality of a second plurality of amino acid sequences generated at the end of the second period to be the first level; training the second generator element during a third period of time that follows the end of the second period; determining the quality of a third plurality of amino acid sequences generated at the end of the third period to be a second level; training the first generator element during a fourth period of time that includes the third period and is longer than the third period; determining the quality of a fourth plurality of amino acid sequences generated at the end of the fourth period to be the second level.

[0247] 55. The method of claim 54, wherein the total time spent training the second generation element is less than the total time spent training the first generation element.

[0248] 56. A method comprising the steps of: obtaining a training dataset comprising amino acid sequences of a protein; generating a structured amino acid sequence based on the training dataset; generating a model for generating further amino acid sequences corresponding to the amino acid sequences included in the training dataset; generating the further amino acid sequences using the structured amino acid sequence and the generated allele network; generating the further amino acid sequences using the model and an input vector; determining an amount of similarity between each of the further amino acid sequences and an amino acid sequence of an antibody generated in association with expression of a germline gene; determining the length of each complementarity determining region (CDR) H3 region of each of the individual amino acid sequences; and evaluating the further amino acid sequences based on the amount of similarity of each of the further amino acid sequences and the respective lengths of the CDR H3 regions.

[0249] 57. The method of 56, comprising evaluating the further amino acid sequence based on a measure of the immunogenicity of the further amino acid sequence.

[0250] 58. The method according to 57, wherein the measure of immunogenicity corresponds to a measure of major set histocompatibility complex (MHC) class II binding.

[0251] 59. One or more hardware processors; and when executed by said one or more hardware processors, causing said one or more processors to: training a first model of a first generation adversarial network element of a first generation adversarial network with a first training data set including a first number of amino acid sequences of a light chain of an antibody to generate a first trained model; training a second model of a second generation element of said generation adversarial network with a second training data set including a second number of amino acid sequences of a heavy chain of an antibody to generate a second trained model, wherein training said second generation element proceeds at a first rate different from a second rate of training of said first generation element; generating a first further number of first further amino acid sequences corresponding to the light chain of the antibody using said first generation element. 16. A system comprising: one or more non-transitory computer-readable media having stored thereon computer-executable instructions to perform the operations including: generating, using the second generation element, a second further number of second further amino acid sequences corresponding to a heavy chain of an antibody; and combining, using the generation allele network, a first amino acid sequence of the first further number of first amino acid sequences with a second amino acid sequence of the second further number of second amino acid sequences to generate a third further amino acid sequence, wherein the third amino acid sequence corresponds to an antibody comprising a light chain corresponding to the first amino acid sequence and a heavy chain corresponding to the second amino acid sequence.

[0252] 60. The system described in 59, wherein a second generating element is trained using a number of Hobble weights to reduce the rate at which the second generating element is trained relative to the additional rate at which the second generating element is trained without the Hobble weights.

[0253] 61. The system of 59 or 60, wherein the second generating element is trained by slowing the gradient of the second generating element.

[0254] 62. The system of any one of claims 59 to 61, wherein the operations include training a second generator element during a first period of time; determining a quality of a first plurality of amino acid sequences generated at the end of the first period to be a first level; training the first generator element during a second period of time that includes the first period and is longer than the first period; determining a quality of a second plurality of amino acid sequences generated at the end of the second period to be the first level; training the second generator element during a third period of time that includes the first period of time and is longer than the first period of time; determining a quality of a third plurality of amino acid sequences generated at the end of the third period to be a second level; training the first generator element during a fourth period of time that includes the third period of time and is longer than the third period of time; determining a quality of a fourth plurality of amino acid sequences generated at the end of the fourth period to be the second level.

[0255] 63. The system of claim 62, wherein the total time spent training the second production element is less than the total time spent training the first production element.

[0256] 64. A system comprising one or more hardware processors and one or more non-transitory computer readable media that, when executed by the one or more hardware processors, cause the one or more processors to perform operations including obtaining a training dataset comprising amino acid sequences of proteins; generating structured amino acid sequences based on the training dataset; generating a model for generating additional amino acid sequences corresponding to the amino acid sequences included in the training dataset; generating the additional amino acid sequences using the structured amino acid sequences and generated allele network; generating the additional amino acid sequences using the model and an input vector; determining an amount of similarity between each additional amino acid sequence and an amino acid sequence of an antibody generated in association with expression of a germline gene; determining a length of each complementarity determining region (CDR) H3 region of each of the individual amino acid sequences; and evaluating the additional amino acid sequences based on the amount of similarity of each of the additional amino acid sequences and the respective lengths of the CDR H3 regions.

[0257] 65. The system according to 64, comprising a step of evaluating the further amino acid sequence based on the measure of immunogenicity of the further amino acid sequence.

[0258] 66. The system according to 65, wherein the measurement of immunogenicity corresponds to a measurement of major set histocompatibility complex (MHC) class II binding. EXAMPLES

[0259] We validated learning the rules of human antibody formation using a Generative Adversarial Network (GAN) trained from over 400,000 human antibody light and heavy chain sequences. The resulting model outperforms common in silico techniques by capturing residue diversity across the entire variable region, allowing for the generation of extremely large and diverse libraries of novel antibodies that mimic somatic hypermutation human repertoire responses. This method allows for the rational design of novel (de novo) human antibody libraries by explicitly controlling various properties of our search library. Through transfer learning, we can bias the GAN to generate molecules with improved stability and developability, reduced predicted MHC class II binding, and key properties of interest such as specific complementarity determining region (CDR) characteristics. These approaches provide a mechanism to better study the complex relationships between antibody sequences and molecular behavior, both in vitro and in vivo. We validated the method by successfully expressing a proof-of-concept library of approximately 100,000 GAN-generated antibodies by phage display. We present the sequences and homology model structures of exemplary generated antibodies expressed in stable CHO pools and evaluated multiple biophysical properties. The creation of drug discovery libraries using our in silico approach allows us to control the pharmaceutical properties of these therapeutic antibodies so that they can provide a more rapid and cost-effective response to biological threats.

[0260] Antibodies are an important class of biologics based therapeutics with distinct advantages of specificity and efficacy. However, high cost, long development times, and the availability of monoclonal antibody therapeutics present significant challenges. To rapidly respond to known and novel pathogens and diseases and provide affordable, high-quality treatments to patients worldwide, molecules must be engineered for their design, but at the same time developable and safe for patients. In addition to these overall costs and development times, many antibodies, due to their biophysical properties, undergo aggregation, unfolding, precipitation, and other physical changes when processed into pharmaceutical products, resulting in poor yields or requiring individualized preparation protocols or formulations. Despite the extensive research being conducted to discover pharmacologically active antibodies and to understand their physical and biological behavior, they remain difficult to identify and developably optimize for a given disease or pathogen.

[0261] Therapeutic antibody discovery often involves either display methods or B cells isolated from humans or animals exposed to the antigen or disease target of interest. Although B cell isolation and deep sequencing workflows have been improved over the years in terms of cost, effort, and speed, there are still inherent limitations when considering this process as an antibody discovery platform. A sufficient immune response is required from the specific subjects used, and the number and diversity of subjects used may not be sufficient to produce a sufficient diversity of antibody sequences. There is also the challenge of overcoming B cell-driven survival against specific epitopes, as therapeutic epitopes will not be utilized in the immune response unless they compete with the dominant epitope, resulting in only a panel of antibodies focused on a limited number of epitopes. Library approaches can explore a wider range of sequence space, but most examples of synthetic libraries have yielded sequence profiles that are quite different from those expressed in the human immune system. In both cases, there is little or no ability to control the chemical, biophysical, and biological properties of the identified candidates. As a result, the antibodies that are discovered often possess the above characteristics, which seriously complicate their developability and stability.

[0262] Recent synthetic library approaches involve random mutagenesis (commonly known as positional frequency analysis, PFA), where certain residues are allowed to change type according to the statistical rules of their frequency of occurrence by position in the antibody. PFA and other related methods do not consider interactions between residues, except in cases where these interactions limit protein expressibility. While this broadly explores sequence space, it ignores how residue types interact to form stabilizing features such as hydrogen bonds or ionic bonds. The random assignments are also made without consideration of the properties of the final antibody entity, which may result in unusual and potentially problematic properties on the protein surface.

[0263] Another drawback of most synthetic library approaches is that they only pay attention to the complementarity determining regions (CDRs) of antibodies. Although the CDRs are the most important parts of the antibody variable region in determining binding interactions, many CDR positions defined by Kabat are part of the core immunoglobulin (Ig) fold, and many of the framework residues can also play important roles in direct antigen binding, molecular stability, and CDR orientation. By limiting mutations to the CDRs, existing libraries ignore the potential improvements in biological activity and developability that some framework mutations can provide.

[0264] Even when a therapeutic antibody is identified, it can be difficult to improve the production and purification behavior of the antibody by modifying the sequence. Many papers have been published that attempt to develop predictable relationships between the antibody's sequence and / or calculated molecular structure and various physical properties of the molecule, but the relationships are difficult to understand because they involve complex nonlinear interactions between the constituent amino acid residues. Such studies often deal with very small numbers of molecules, often less than 200, and often less than 50, from a set of non-diverse sequences, such as a small number of parent sequences, a few parent sequences with a small number of highly related sequence variants, or even a single antibody with mutational scanning. While the approach provides information about individual antibodies or a small group, it is highly unlikely that the complexity of residue interactions can be generalized to other antibodies. Such understanding requires exploration of the ultra-high dimensional space of the larger antibody sequences. Computational approaches to optimizing molecular behavior also tend to ignore whether the modified molecule resembles a human antibody, an assessment of which is left to expensive in vitro testing studies.

[0265] Deep learning is one way to better understand the complex relationships between sequence and protein behavior and has been the focus of many recent publications. In the context of discovery and libraries, generative models such as Generative Adversarial Networks (GAN) and Autoencoder Networks (AE) are of particular interest as they have shown feasibility for generating unique sequences for proteins and nanobodies and antibody CDRs. However, these efforts have focused on short sequences of proteins or portions of antibodies. Using these approaches on the full antibody sequence space comes with a unique set of challenges for machine learning models.

[0266] Antibody backgrounds are derived from different germline lines, are much larger in size, and are composed of multiple chains, resulting in a more complex sequence and structure space. In a machine learning setting, more complexity generally requires more data to solve. However, sequence data associated with experimental data is more limited for antibodies and is much more costly to obtain than for small molecules.

[0267] Disclosed herein is Antibody-GAN, a novel synthetic approach to design a new class of antibody therapeutics: "humanoid" antibodies. Antibody-GAN employs a modified Wasserstein-GAN for both single-chain (light or heavy) and paired-chain (light and heavy) antibody sequence generation. The GAN allows encoding of key properties of interest into the library as a feature-biased search platform. Our antibody-GAN architecture (1) captures the complexity of the variable regions of standard human antibody sequence space, (2) provides a basis for generating novel antibodies spanning greater sequence diversity than has been explored by standard in silico generation approaches, and (3) provides a unique way to bias the physical properties of generated antibodies towards developability and improved chemical and biophysical properties through transfer learning (continuous training of a model with a subset of data that has specific desired properties).

[0268] We demonstrate a GAN library biased towards properties such as reduced negative surface area patches, identified as potential causes of aggregation, thermal instability, and reduced half-life, and dissociation from MHC class II binding, which may reduce the immunogenicity of the generated antibodies. Additionally, we show library bias towards higher isoelectric point (pI) to reduce aggregation and prevent precipitation in therapeutic formulations, and longer CDR3 length, known to increase diversity and create more effective therapeutics for target classes.

[0269] To demonstrate the viability of antibody-GANs for generating humanized antibody sequences, we used GANs to generate a proof-of-concept validation library of 100k sequences from four germline subgroups. These sequences were generated using two single-chain GANs, each trained on a set of 400,000 heavy or light chain sequences from human-repertoire antibodies. GAN sequences were expressed in phage as antibody antigen-binding fragments (Fabs). Two less-representative germline subgroups were optimized for germline identity using transfer learning. From this initial library, we present the sequences, structures, and biophysical properties of two antibodies with diverse surface patch properties expressed in stable Chinese Hamster Ovary (CHO) cells.

[0270] A general generative allele network for antibody design

[0271] In a typical antibody-GAN architecture, a set of real training variable domain (Fv) antibody sequences is fed into the GAN's classifier along with the output of the generator. The generator takes a vector of random seeds as input and outputs random synthetic antibody sequences. During training, the classifier is progressively trained to accurately attempt to distinguish between real and synthetic sequences, and the generator is progressively trained to generate synthetic sequences that are indistinguishable from real human repertoire sequences in the training set. After initial training of the antibody-GAN from the entire training set, transfer learning can be used to bias the GAN towards generating molecules with desired properties.

[0272] As a demonstration of the general architecture and training approach, we trained an antibody-GAN with a set of 400,000 randomly selected human repertoire sequences per chain from the Observed Antibody Space Project (OAS). Prior to training, all sequences were structurally aligned using the AHo numbering system to allow direct comparison of residues at the same structural positions across the dataset. This greatly simplifies the relationships that the GAN must capture for both generation and identification. Further training details are provided in the Methods section.

[0273] The antibody-GAN (GAN), OAS training set, and a set of sequences with 100% germline frameworks and PFA-generated CDRs (PFA) were compared by selecting a random set of 10,000 sequences classified as germline HV3-30 from the training set and all three synthetic sets. These were assessed for the distribution of germline identity of framework residues. In the human repertoire, sequences that have undergone somatic hypermutation during B-cell maturation were found to deviate from the germline identity of the framework. Molecules in the antibody-GAN model (GAN) deviate from the germline in close proximity to OAS. Note that the PFA set uses the exact germline framework, and therefore the germline identity is always 100%.

[0274] As an index of binding paratope diversity within a given set, we used the diversity of heavy variable (HV) CDR3s, assessed using (1) pairwise Levenshtein distances calculated only from HV CDR3 residues across all three sets, and (2) scores from the first two components of a principal component analysis (PCA) model on aligned HV CDR3 sequences from OAS, GAN, and PFA data. In general, the OAS set shows the greatest diversity in HVs, while the GAN and PFA sets have similar diversity with a main peak in OAS, and the diversity of GAN is slightly greater than that of PFA.

[0275] The distribution of sequence variability in the OAS human repertoire set is more similar to that seen in the GAN set than in the PFA set, and is significantly different from the other two sets, especially in the dense regions of the plot. The explained variance for PCA elements 1 and 2 is 10% and 4%, respectively. While these are only a small portion of the overall variance in HV CDR3, they are a good representation of the covariation relationships between HV CDR3 residues, indicating that the antibody-GAN approach captures significant relationships in human repertoire HV CDR3 that are not present in the PFA approach. KL-diversity is a measure of how different the two distributions are from each other, with values ​​of 0 indicating identical distributions and values ​​further away from 0 indicating more divergent distributions. For the OAS and GAN sets, the KL-diversity of the distribution on PC1, the element that captures most of the variance in CDR3, is 0.57. The KL-diversity of PC1 for the OAS and PFA sets is 1.45. The distributions of PC1 and PC2 can be determined for both the GAN and PFA sets compared to OAS. The PFA set deviates notably from the OAS and GAN sets, raising questions about how well the PFA approach reproduces human paratopes and the diversity of these paratopes.

[0276] Biasing and Controlling Antibody Discovery Libraries

[0277] Our generative deep learning approach to humanoid antibody library generation results in antibody libraries that are more similar to humans than existing synthetic library approaches, while allowing us to control the library's properties. A subset of these libraries was generated using a deep learning technique known as transfer learning, which biases the network from general antibody-GAN-trained properties to specific properties of interest. The corresponding heatmaps show the dissimilarity in the proportion of sequences in a given bin between each library and the OAS.

[0278] The bias towards the length of the primary binding paratope CDR H3 of the antibody sequence can be determined. We compared the following four libraries with OAS: the baseline antibody-GAN (GAN), the antibody-GAN with transfer training to small (GAN-C) and large (GAN +C) CDR H3 lengths, and the PFA-generated library (PFA). The baseline antibody-GAN library shows a total of 27% divergence in CDR H3 distribution from the OAS. Although still significantly different, it more closely reproduces the OAS distribution on CDR H3 than PFA (38% divergence) or the other two intentionally biased libraries. The GAN-C library was generated by model transfer trained on a small subset of about 1,000 sequences from a GAN library with a CDR H3 length of less than 12, resulting in a library shifted to 68% CDR H3 sequences. Similarly, GAN+C was transfer trained on approximately 1,000 sequences from a GAN library with CDR H3 length >22, resulting in a highly significant bias of 89% for longer CDR H3 sequences. GAN+C libraries also inherently bias towards diversity by generating antibodies with longer CDRs and therefore more residues. Antibodies with longer CDR H3s have also been shown to be more successful as therapeutics for diseases such as human immunodeficiency virus (HIV)57, and may be useful as discovery sublibraries for targets where this longer exposed paratope may be required.

[0279] The heavy chain immunogenicity bias can be measured using the principal set hierarchical compatibility class II (MHCII) binding scores derived from the in silico peptide fragment-MHCII binding affinity predictions (see Figure 23A for light chain bias). An in-house machine learning prediction tool for peptide-MHCII binding was used, as well as a binding prediction tool provided by the Immune Epitope Database (IEDB). Peptide-MHCII binding is the first step in the T cell-mediated immune response and is the most specific handle available to substantially reduce immunogenicity risk. The GAN library was only 2% divergent from OAS in predicted immunogenicity and statistically indistinguishable from the human repertoire training set (p<0.0001), while PFA showed a statistically significant 11% shift towards higher immunogenicity. The GAN-I library showed a total of 76% shift towards lower predicted MHCII binding than the human repertoire, using a transfer learning approach similar to that described above. It is believed that MHCII binding is the first step in an immunogenic response, and that reduced binding reduces the likelihood of an immunogenic response. As a result, biased GAN-I should generate molecules with a lower likelihood of an immunogenic response. This large bias toward less immunogenic sequences could be a significant bias toward higher quality antibody therapeutics, resulting in a library of safer therapeutics for patients.

[0280] To what extent this bias corresponds to a decrease in immunogenicity depends largely on the quality of the model used to select the transfer sample. As a control condition, the GAN+I library showed a 49% bias towards increased MHCII binding. Such highly immunogenic biased models are usually not of much interest for developing libraries, but they can provide a means to generate molecules that help to validate the underlying MHCII binding model, further highlighting the utility of the GAN method as a tool to explore molecular and therapeutic space.

[0281] For antibody drugs, the isoelectric point (pI, the pH at which the molecule is neutral) is an important measure for development, since pIs near the formulation pH are viscous and may result in aggregation or precipitation. In general, if the pH is slightly acidic, the overall charge is high, and recent formulations are centered around pH 5.5. To remain stable in solution, therapeutic antibodies ideally need an overall molecular pI of 8.5 or higher. The GAN library provides a distribution of pIs for the Fv portion of the antibody that is statistically indistinguishable from OAS, and the PFA library has a small shift of 11% toward higher Fv pI. We have shown that the GAN-P library can bias the library to a 79% lower Fv pI via transfer learning. However, the GAN+P library shows a 43% increase in sequences with a calculated Fv pI greater than 9, resulting in a significant bias toward potential developability.

[0282] Large surface patches in antibody therapeutics are associated with drug development problems such as aggregation, thermolability, increased viscosity, and increased clearance rates, but also with improved specificity, especially when the patches are charge-related. Thus, biasing libraries toward larger or smaller patches can have beneficial effects. They also serve as examples of general bias models toward desired structural properties. The bias for the maximum negative surface patch area of ​​a molecule, calculated using structure-based homology modeling, can be determined. Large negative patches have been shown to increase antibody viscosity upon high concentration treatment. Again, the GAN library is statistically equivalent to OAS at the maximum negative patch size, with only a 3% dissimilarity, indicating that this model can capture the human repertoire. The PFA library maintains a small but significant shift toward reduced negative surface patch area of ​​7%. The GAN-N library shows that the library can be purposefully shifted toward small negative surface patches to dissociate from the traditional developability problem, with a bias of 31%. The GAN+N library shows that it can shift in the opposite direction, toward larger negative patches, with a bias of 36%. Structure-based properties such as surface patches are less prone to bias than sequence-based properties due to (1) their non-Gaussian distribution and (2) an additional layer of abstraction and complexity away from the sequence. These issues can be addressed by increasing the number of sequences in the transfer learning training set, for example by repeated training and sampling. For more complex properties, layers can be added to the model itself during transfer-learning.

[0283] Diverse germline combinatorial library design and expression

[0284] Synthesis of a diverse de novo antibody discovery library containing specific sequences can be costly. Targeting of this specific sequence cannot be achieved with standard codon degeneracy approaches. To significantly reduce this cost, we used a chain-directed approach to library design to combinatorially combine heavy and light chains made with specific amino acid sequences designed by antibody-GAN, rather than designing each Fv separately. The antibody-GAN structure (Figure 21) is designed in a modular manner. After training with paired chain Fv sequences, the heavy and light chain generators can be used separately to generate single chain sequences, such that an independently generated heavy chain should pair with any independently generated light chain to generate a complete Fv that maintains the intended properties of the library. All antibody-GAN and transfer-learning antibody-GAN libraries were generated from models trained in this manner.

[0285] Alternatively, antibody-GAN models can be split into single-chain models first. These must be trained on single-chain sequences, which may be useful when generating diverse libraries of different germlines if there are no interesting properties associated with the complete Fv that one wishes to bias. Since there is little publicly available data on the base sequences of the paired chains that indicate properties such as developability, expression, and stability, we synthesized a simple and unbiased initial discovery library expressed in phage. The goal of this initial library is to recreate the human repertoire. In so doing, we create a dataset that will be of great use in biasing future libraries. Subsequent GAN libraries and molecules were therefore generated using the single-chain version of the antibody-GAN.

[0286] In our initial library, we selected heavy chain germlines IGHV3-30 and IGHV1-2 to be paired in combination with light chain germlines IGKV3-20 and IGKV1-39. The number of training set examples for IGHV1-2 and IGKV1-39 is less than that of the other two germlines, and there are not enough examples to train a model of sufficient quality. This problem is further complicated by the fact that the other germlines have even fewer training examples. This can be improved again by using transfer learning.

[0287] Because the HV3-30 and KV3-20 germlines are well represented in the OAS training set, the model generates sequences with sufficient framework quality. For the HV1-2 and KV1-39 germlines, respectively, a greater departure in framework quality from the OAS was determined when generated by the base model without transfer learning. Only when the model is transferred trained and continues training only on the germline subgroups of interest can it generate sequences whose framework quality is closer to the OAS for HV1-2 and KV1-39.

[0288] While a full-scale production library can combinatorially combine more than 10,000 individual single-stranded sequences from the germline to form billions of molecules, the proof-of-concept miniature library was created by selecting 158 sequences from each of these four germline sequences and combinatorially combining them to build a library of approximately 100,000 total sequences.

[0289] The Fab fragment display levels of the four germline paired sublibraries were determined, each containing approximately 25,000 GAN-generated sequences. Display levels were estimated by capturing serial dilutions of purified phage on anti-human Fab-coated ELISA plates and detecting with anti-M13 antibody conjugated to HRP. The average display levels of IgG Fab from each sublibrary in polyclonal phage were normalized for the total phage concentration. A slight bias towards higher expression at higher concentrations is seen for the germline sublibraries, including the KV1-39 sublibrary. Whether this difference is indeed significant and related to a better tolerance of the KV1-39 sequences, or whether it represents differential binding of the anti-human Fab capture antibody used in the ELISA remains to be investigated.

[0290] To confirm that the expressed Fabs were indeed the de novo designed sequences, approximately 30 colonies expressed in monoclonal phages from each of the four sublibraries were selected and sequenced. Variable region clads of selected sequences from the two sublibraries expressing the KV3-20 light chain into the GAN-designed KV3-20 sequence were primarily determined because: 1) colony selection was random and provided good coverage of the designed sequence space, and 2) any amino acid mutations compared to the GAN-designed sequence were contained in only a small fraction of the expressed sequences; most matched our synthetic design exactly.

[0291] We determined a similar clad of selected sequences from the two sublibraries expressing the HV3-30 heavy chain. The same observations can be made with this set. The selected sequences are well spread across the design space, with even fewer amino acid mutations than the KV3-20 set, and more precisely match the de novo GAN designed sequences. Again, expression is shown for sampled colonies and whether or not those sequences were paired with the KV3-20 light chain. Phage library sequences that were not paired with a light chain from the KV3-20 library were paired with light chain sequences from the KV1-39 library.

[0292] A further subset of antibodies was selected from the HV3-30 / KV3-20 sublibrary and expressed in stable CHO pools for biophysical analysis. Figure 18D shows the 3D structures of two GAN antibodies selected for their particularly interesting patch properties. The molecule mAb GAN-1285 has a ~600 Å patch size shown in red. 2 Molecules with such large maximum negative patches are relatively rare in the basic distribution of antibody-GANs, but are interesting to study from the perspective of exploitability. In contrast, the molecule mAb GAN-1528 has a maximum negative surface patch of approximately 140 Å. 2 It is.

[0293] Biophysical validation of CHO-expressed GAN antibody

[0294] Aiming to validate our GAN approach, we present biophysical data of mAb GAN-1285 and mAb GAN-1528 (Figures 18D, 19) after stable CHO expression and purification for the analysis of interesting properties of the maximally negative surface patches. Figure 20 shows the behavior of these two antibodies in four key assays in our platform: differential scanning fluorimetry (DSF), self-interacting nanoparticle spectroscopy (SINS), polyethylene glycol (PEG) solubility, and size exclusion chromatography (SEC). These assays are commonly used to evaluate the stability and developability of therapeutic antibodies.

[0295] DSF results were measured for mAb GAN-1285 and mAb GAN-1528, as well as the platform control antibody, MS-43. DSF evaluates the temperature at which a particular region of an antibody begins to unfold. More stable, and therefore more developable, antibodies tend to have one or more regions unfold at higher temperatures and have a higher first-unfolding transition temperature. The same constant region of these three molecules, presumed to be the IgG CH2 region, shows an unfolding response at about 72°C. A molecule with a very large negative surface patch, mAb GAN-1285, first unfolds around 60°C and is much less thermostable. This is consistent with the observation that negative surface patches are known to be associated with thermoinstability.

[0296] The SINS assay is commonly used to interrogate molecular self-interactions, which can lead to problems in manufacturing processes as well as viscosity and filterability issues. Interestingly, the two GAN molecules showed the same SINS profile as the PBS negative control, and showed a low tendency to self-interact, especially compared to the positive control molecule MS-63, which is known for its high self-interaction behavior.

[0297] The remaining assays, PEG solubility and SEC, indicate that both antibodies are reasonably soluble and exhibit relatively low amounts of high molecular weight (HMW) formation, although there are potentially significant differences between the two antibodies in both assays.

[0298] Consideration

[0299] Herein, we describe a new class of in silico derived human antibodies, termed "humanoid", due to the explicit requirement that the generated sequences should mimic human antibody sequence patterns. Although antibodies have excellent antigen specificity and have often been applied as scaffolds for therapeutic drugs, B cells are not subject to selective pressure in vivo to produce antibodies with ideal biotherapeutic properties. To reduce development costs and significantly shorten response times for known or novel diseases and pathogens, the discovery libraries must include therapeutic antibodies with desirable properties such as expressibility in the host system, compatibility with common protein manufacturing processes while achieving high purity and high yield, and high stability under long-term storage conditions. In addition, these therapeutic libraries must also include antibodies that exhibit usability properties such as low viscosity for injection at high concentrations, long elimination half-life for reduced dosing frequency, and high bioavailability for conserving injection volumes. Herein, we describe an antibody-GAN approach to the in silico design of monoclonal antibodies that retain typical human repertoire properties such as diversity and immunogenicity, while enhancing the possibility to bias the library in silico to achieve other desirable biological properties.

[0300] We experimentally validate the in silico approach via phage Fab-display of an initial library of about 100,000 GAN sequences and present the biophysical properties of two exemplary GAN antibodies expressed in CHO. Although the biophysical data of the CHO-expressed molecules is insufficient to show the causal effect of structural differences on the biophysical properties, they show that the molecules are properly folded and that they exhibit the expected biophysical properties. These results show that antibody-GAN allows the study of a large, truly diverse set of thousands of full-length secreted antibodies on phage for biophysical properties, and hundreds of millions of antibody Fabs. These provide a true basis for identifying the causal effect, or lack thereof, of structural features and sequences on biophysical properties. This data can also provide a truly generalized in silico predictive model across antibodies.

[0301] Because there are many nonlinear pathways in antibody optimization, continued research will be necessary to determine precisely which antibody sequence, structure, or biophysical properties bias the developability, quality, and efficacy of antibody libraries. Existing datasets investigating the developability of antibody drugs are more commonly on the order of tens of antibodies, although they may consist of hundreds of molecules. To derive complex properties such as antibody viscosity and chemical and thermal stability from sequence and structure, many sample molecules, hundreds or more, are required. Until now, protein scientists have had to rely on previously discovered antibodies and their close variants, which provide only a small random sampling of the true antibody space. Antibody-GANs allow for the exploration of human antibody space in a rational way. By using transfer learning to bias GANs towards given properties, such as structures calculated from homology modeling or properties measured on physically expressed antibodies, questions such as how engineering for developability affects important properties such as affinity and bioavailability can now be understood. This provides a mechanism for the in silico and in vitro generation of a much broader range of sequences with deliberate bias to form deep, rich training sets for human antibody research.

[0302] Recent advances in the field of protein assays now provide ultra-high throughput methods in, for example, phage or yeast to express and query the stability of molecules, with many more advances to come. Inventors can now rationally design and generate vast experimental antibody data sets for these and future methods as we continue to understand the properties of developable and effective therapeutics.

[0303] As a training set generation tool, our antibody-GAN approach significantly advances our knowledge of antibody design and behavior. It also transforms how therapeutics are created by better recapitulating the properties of antibodies from in vivo sources and adding tunable properties for biologics, production, and treatment. The resulting humanoid discovery library will lead to better quality treatments and faster, more cost-effective responses to biological threats and disease targets.

[0304] method

[0305] Data source for the training set

[0306] Training set data were obtained from the Observed Antibody Space (OAS) repository. Raw nucleotide sequences were automatically translated, classified, and structurally aligned using in-house software (Abacus™). The AHo structure numbering system was used for structural alignment of variable regions.

[0307] To generate the training set, the variable regions were first filtered by our in-house software Abacus™ to remove all sequences not classified as human variable regions, and then further cleaned to remove sequences containing stop codons, truncations, or insertions. All sequences with less than 85% identity to the closest germline were also removed.

[0308] For every paired chain model, we extracted distinct sequences belonging to the closest germline target heavy and light germline frameworks, and these two subsets were randomly sampled and combined during training to generate paired sequences.

[0309] For any pairwise single-strand model, the first (initial) training set included all germlines. If transfer learning was required, it was performed with a extracted set of sequences whose closest germlines belonged to the particular germline of interest.

[0310] Developing and training an antibody GAN

[0311] The antibody-GAN code was developed in Python. The Keras and Tensorflow deep learning libraries were primarily used to build and train the antibody-GAN. The Pandas and Numpy libraries were used to process the data and build the training set. Other public libraries used in the development and analysis of the antibody-GAN include Sklrean, Scipy, and Seaborn.

[0312] The antibody-GAN architecture is based on the Wasserstein-GAN (with gradient penalization) architecture and therefore consists of a generator and a discriminator, which in the WGAN architecture are commonly called critics. The single-stranded network generator takes as input a noise vector of size 296. This vector is fed into a dense layer, followed by three upsampling layers, a 2D convolutional transpos layer, and a final SoftMax layer to generate a 2D array of size 148x22. This 2D array represents the 148 residues and 22 amino acids (including deletions and Xs) of the light or heavy chain of the antibody sequence in a one-hot encoding scheme, and the SoftMax layer corresponds to this 2D array. The antibody sequence, ordered by AHo numbering, has 149 residues in both chains. To simplify the network structure, we decided to remove one residue from each chain during the encoding process, which appears to be relatively constant in the human repertoire. During decoding, we return these constant residues to the discriminator (critic).

[0313] It takes a 148x22 antibody chain as input and passes it through two 2D convolutional layers, followed by flattening, a dense layer, and a single-node linear output.

[0314] The paired structure of antibody-GAN is similar to the single-chain version, except that there are two generators with the same structure (one per chain). The output of each independent chain generator is concatenated to a 296x221-one-hot encoding scheme of the antibody sequence with both heavy and light chains. The architecture can be extended when transferring training or learning to complex properties where nonlinear interactions between the two chains are required. Critics of paired GAN take a one-hot encoding scheme of the 296x22 paired chain antibody sequence as input and keep the architecture similar to the above.

[0315] During training of a single-stranded HV3-30 GAN (using a batch size of 128), we determined the fake (generated) and real (training set examples) generator and discriminator (critic) losses. Quality was assessed by a germline framework consensus on training epochs for this model. Training was terminated when the generated sequences began to show sufficient quality.

[0316] Human monoclonal antibodies have been shown to have higher variability in their heavy chains than in their light chains, which can guide the asynchronous optimization of the light and heavy chain generators during in-pair training.

[0317] Antibody-GANs generate higher quality heavy chains than light chains, which can be solved by freezing layers of the heavy chain generator that reach a state where they generate sequences of sufficient quality, and continuing training on the network until the light chain generator reaches the desired quality.

[0318] Creating a PFA set and selecting an OAS set

[0319] The IGHV3-30 / IGKV3-20 PFA-based set used above was generated using the IGHV3-30 and IGKV3-20 training sets extracted from the OAS, consisting of ~250,000 and ~150,000 sequences, respectively. A 100% germline framework for IGHV3-30 was used as the constant framework for all heavy chain PFA sequences, and a 100% IGKV3-20 germline framework was used for all light chains. Each residue in the CDRs (CDR1, CDR2, and CDR3) was then generated using position frequency analysis, where for any given position, residues were randomly sampled from a distribution representing the frequency of the amino acid in the training set. In this way, 10,000 heavy chain sequences and 10,000 light chain sequences were generated and randomly paired to generate a set of 10,000 sequences with complete variable regions.

[0320] The OAS sets were created by randomly downsampling 10,000 sequences from each of the IGHV3-30 and IGKV3-30 training sets, which were then paired together to create a set of 10,000 sequences with complete variable regions.

[0321] PCA of DR3

[0322] Once the aligned CDR3s of a given antibody were encoded into a vector representation, PCA analysis was performed. A two-component PCA model was built using the sklearn library on the one-hot encoding vectors from all sequences in the OAS set, the PFA set, and the base GAN set (total of 30,000 samples). Heavy and light chain models were built and trained separately.

[0323] Antibody-GAN binding source

[0324] CDR H3

[0325] Our in-house software, Abacus™, was used to assess the length of CDR H3 from the training, GAN-generated and PFA-generated sets.

[0326] Calculated immunogenicity

[0327] MHCII is a polymorphic transmembrane protein that binds and presents fragments of foreign extracellular proteins to the T cell receptor (TCR) to initiate adaptive immune responses. The MHCII binding score is a composite index that aims to quantify the immunogenicity risk in a continuous manner based on whether its constituent peptides are predicted to bind strongly and indiscriminately to MHCII proteins. The quality of the index depends on the selection of accurate peptide-MHCII binding predictors and a rational method to aggregate predictions between peptide fragments in the sequence and allelic variants of MHCII.

[0328] The inventors developed a machine learning algorithm for peptide-MHCII binding affinity prediction that was used to train NetMHCIIpan-3.2 and reported by Jensen et al. Several machine learning algorithms have been developed whose performance exceeds that of traditional matrix-based approaches to peptide-MHCII binding affinity prediction, including NetMHCII-pan and recently MARIA.40,54. The inventors use their own MHCII binding predictor for ease of integration with other sequence analysis tools and based on favorable accuracy comparisons with published benchmarks (not shown here). Predictions from our model generally correlate with the "IEDB recommended" algorithms for peptide-MHCII binding prediction.

[0329] To calculate sequence MHCII binding scores, we first split the sequence into its constituent 15-mer peptide fragments (15 sliding windows, 1 stride). For each 15-mer, we used allele-specific models to predict binding affinity for eight common allelic variants of MHCII (encoded by HLA alleles DRB1*0101, DRB1*0301, DRB1*0401, DRB1*0701, DRB1*0801, DRB1*1101, DRB1*1301, and DRB1*1501). This set of alleles was also used in de Groot and Martin's pioneering work on risk reduction of MHCII binding. We convert the binding affinity into a z-score for each allele using the mean and standard deviation of predicted affinities for a large reference set of 15-mers randomly selected from human protein sequences archived in UniProt.

[0330] We took the median z-score for each 15-mer allele and summed the median positive z-score across the array to obtain a final MHCII binding score. The median is an appropriate aggregation because peptide fragments that bind to several MHCII variants pose an immunogenicity risk to a larger patient population than fragments that bind only one fragment. Dhanda et al., creators of the Protein Deimmunization Engine on the IEDB website, use the median to aggregate MHCII binding scores across alleles. We ignore negative scores in the sum across all arrays because peptides that do not bind MHCII (large negative scores) should not offset peptides that bind strongly to MHCII (large positive scores). We measured the fraction of putative MHC-binding peptides relative to all unique 15-mers in each array. The low-immunogenic set (GAN-I) had fewer MHCII-binding peptides than the other sets, suggesting that the GAN learns which 15-mers to avoid regardless of our sequence score abstraction.

[0331] Structural modeling, calculated isoelectric points (pI), and negative patch surface areas Structural models were calculated as Fab structures using the antibody modeling tools within the Molecular Operating Environment (MOE, Chemical Computing Group, Montreal, Canada). The Fab structure was used instead of Fvs to generate more accurate Fv surface patches in the presence of constant domains. pI was calculated using the Ensemble Isoelectric Point method of the Protein Characterization Tool within MOE, the SVL method. Electronegative patch sizes were calculated using the Protein Patch method as the SVL call within MOE. The hydrophobic minimum area (p_hminarea) was set to the default setting of 50–30 Å. 2 and change the charge cutoff (p_qcuatoff) from the default setting of 40 to 20 Å. 2 was changed to.

[0332] Selection of GAN-library sequences

[0333] The human repertoire contains a small subset of sequences with missing residues, non-canonical cystines, non-canonical N-linked glycosylation sites, or potential N-linked glycosylation sites. Sequences with these properties were not extracted from the training set and therefore also included in the small subset in the GAN library. All phage libraries were filtered to remove GAN-generated sequences with any of these properties before final sequence selection.

[0334] Phage expression of GAN-library

[0335] Bacterial strains

[0336] E. coli One Shot™ TOP10 cells (FmcrA Δ(mrr-hsdRMS-mcrBC) Φ80lacZΔM15 Δ lacX74 recA1 araD139 Δ(araleu)7697 galU galK rpsL (StrR) endA1 nupG) were purchased from Thermo Fisher Scientific and used for phagemid DNA cloning. E. cloni® 10G electrocompetent cells (FmcrA Δ(mrr-hsdRMS-mcrBC) endA1 recA1 recA1 Φ80dlacZΔM15 ΔlacX74 araD139Δ(ara,leu)7697galU galK rpsL rpsL nupG λtonA (StrR)) were purchased from Lucigen Corporation and used for phagemid DNA cloning. E. coli SS520 electrocompetent cells (F'[proAB lacIqZ ΔM15 Tn10 (TetR)] araD139 Δ(ara-leu) 7696 gaE15 gaK16 Δ(lac) X74 rpsL (StrR) hsdR2 (rK- mK+) mcrA mcrB1) were purchased from Lucigen Corporation and used as a host for phage library production.

[0337] Cloning

[0338] The phagemid pADL-20c (Antibody Design Labs) was used for construction of the GAN sublibrary and was modified for expression of Fab antibody fragments as N-terminal pIII fusion proteins in E. coli. This vector utilizes the bacterial pectic acid lysate (pelB) signal sequence for periplasmic translocation of the fusion protein along with an ampicillin resistance gene for propagation and selection in transformed E. coli. Hexahistidine and FLAG tags were added to the C-terminus of the CH1 and kappa constant domains, respectively, and the amber stop codon of gIII was removed to express the fusion protein in SS520 host cells.

[0339] Synthetic gene fragments encoding the variable heavy and light chains were first amplified separately using PCR primers containing 22 base pair sequences complementary to the phagemid backbone. PCRs were then pooled by germline and sequentially assembled into phagemids using NEBuilder® HiFi DNA Assembly Master Mix (New England Biolabs). Transformation was performed using OneShot™ TOP10 or E. cloni 10G® cells, and the resulting phagemid DNA was purified using ZymoPURE™ II Plasmid MidiprepKit (Zymo Research).

[0340] Phage library production

[0341] E. coli SS520 host cells were electroporated with 250 ng of each sublibrary DNA as described by the manufacturer. An aliquot of each transformation was plated onto 2xYT agar plates supplemented with 100 μg / mL carbenicillin and 2% glucose and incubated overnight at 30°C. The resulting colonies were used for library size estimation and sequencing of the variable heavy and light chains using colony PCR. The remainder of the transformation was used to inoculate 2xYT-CG (2xYT medium containing 50 μg / mL carbenicillin and 2% glucose) at an OD600nm of 0.07 and incubated at OD600nm with shaking at 250 rpm and 37°C. 600nm The cultures were then infected with M13KO7 helper phage (Antibody Design Institute) at a multiplicity of infection (MOI) of 25 and incubated at 37°C without shaking for 30 min, followed by shaking at 200 rpm for 30 min. The cultures were centrifuged and then the medium was replaced with 2xYT-CK (2xYT supplemented with 50 μg / mL carbenicillin and 25 μg / mL kanamycin). After overnight incubation at 30°C and 200 rpm, phage particles were purified, concentrated by PEG / NaCl precipitation, and resuspended in PBS containing 0.5% BSA and 0.05% Tween-20. The phage concentration was determined as 1 unit at OD of 5×10 12The concentration of PEG-precipitated phages from each GAN sublibrary was measured using a spectrophotometer, assuming it to be equal to 1 × 10 phages / mL. 13 Phage / mL was normalized and serially diluted 10-fold in 2% non-skim milk in PBS in duplicate for use in polyclonal phage ELISA.

[0342] Monoclonal phage production

[0343] A single clone with a functional Fab fusion protein was inoculated into 500 μL of 2xYT-CTG (2xYT broth supplemented with 50 μg / mL carbenicillin, 15 μg / mL tetracycline, and 2% glucose) and grown overnight in a 96-deep well plate at 37° C. with rigorous shaking. A μL of the overnight culture was then transferred to a new deep well plate containing 100 μL of 2xYT-CTG and the OD 600nm The plates were incubated at 37°C with rigorous shaking until the MOI reached ~0.5. M13KO7 helper phage was added to each well at an MOI of 25 and the plates were incubated at 37°C for 1 h without agitation, after which the medium was replaced with 2xYT-CK and incubated overnight at 30°C with rigorous shaking. Phage supernatants were collected after centrifugation and diluted 1:1 in 2% non-fat dry milk in PBS for use in monoclonal phage ELISA.

[0344] Phage ELISA

[0345] The amount of Fab displayed on the phages was measured using ELISA. Briefly, 96-well Maxisorp® assay plates (Nunc) were coated overnight at 4° C. with anti-human Fab (Millipore Sigma) diluted 1:500 in PBS, then blocked for 1 h at room temperature (RT) in PBS containing 1% BSA. Diluted phage preparations were added and incubated for 1 h at room temperature before the captured virions were detected with anti-M13-HRP (Santa Cruz Biotechnology) diluted 1:5000 for 1 h at room temperature. All interval plate washes were performed three times in PBST (PBS with 0.1% v / v Tween-20). ELISA was developed by addition of TMB solution (Thermo Fisher Scientific) and quenched with 10% phosphoric acid. Absorbance was measured at A450 nm. Phage supernatant from untransformed E. coli SS520 host cells was included as a negative control.

[0346] Specific CHO-expressed molecules

[0347] CHOK1 glutamine synthetase (GS) knockout host cells (Horizon Discovery, Cambridge, United Kingdom) were maintained in CD OptiCHO (Thermo Fisher Scientific, Waltham, Mass.) containing 4 mM glutamine. Cells were cultured as described above.

[0348] The light chain (LC) and heavy chain (HC) containing the appropriate signal peptides were cloned sequentially using Gibson assembly in a two-step manner into an in-house developed bicistronic PiggyBac transposon expression vector70. Successful insertion of the intended coding sequence was confirmed by Sanger DNA sequencing. Plasmid DNA was purified using a conventional silica-based low-endotoxin Zymo Research kit (Irvine, CA).

[0349] Cells, DNA and RNA were added to BTX25-multiwell electroporation plates (Harvard Bioscience, Holliston, MA) using a Tecan Freedom EVO (Mannedorf, Switzerland) liquid handler. For each transfection, 2.4E6 cells were spun down and resuspended in 150 uL of PFCHO medium (Sigma-Aldrich, St. Louis, MO). 7.5 ug of DNA and 2.5 ug of pJV95 transposase RNA were added to the cells and then electroporated at 3175 uF capacitance, 290 V voltage, 950 Ω resistance in an ECM 830 electromanipulator coupled to a HT 100 high throughput adapter (BTX, Holliston, MA). Both molecules were transfected in triplicate. Cells were transferred to 2 mL of non-selective medium in 24 deep well plates (DWP) shaken at 220 rpm under standard growth conditions and cultured for 2 days before selection. After 2 days, cells were counted using a Guava flow cytometer (Luminex, Austin, TX), plates were spun down, and resuspended in 2 mL of selective CD OptiCHO medium. Cells were then counted and passaged every 4–5 days.

[0350] Thirteen days after the start of selection, cells that were >90% viable were cultured at 8 × 10 in 3 mL of 24 DWP under standard growth conditions. 5 Cells were seeded at 1000μg / mL in proprietary production medium. On days 3, 6, and 8, cells were fed with Cell Boost7a and 0.5% Cell Boost7b (Hyclone GE Healthcare Life Sciences) at 5% of the starting volume. Cell number and glucose were determined as above, and on day 70, 50% glucose was added to a final concentration of approximately 10g / L. On day 10, cells were counted, spun down, and filtered by centrifugation onto a 24-deep well filter plate (Thomson, Oceanside, CA). Titers were sampled by ultra-high performance liquid chromatography (UHPLC) protein A affinity. Replicate wells were pooled together for protein A purification.

[0351] Biophysical validation of CHO-expressed molecules

[0352] Sample preparation

[0353] Samples were buffer exchanged against 10 diavolumes of 20 mM sodium chloride, 150 mM sodium chloride, pH 7.1 (PBS) using centrifugal filters with a 30 kDa molecular weight cutoff (Amicon). After buffer exchange, samples were normalized to 1 mg / mL using a Lunatic protein concentration plate format instrument (Unchained Labs).

[0354] Differential Scanning Fluorescence

[0355] Thermal transition temperatures and weighted shoulder scores were determined by DSF according to the methods described above ( Kerwin, 2019 ).

[0356] Self-Interacting Nanoparticle Spectroscopy

[0357] SINS measurements were performed according to the method described above (Liu, 2013). Briefly, gold nanoparticles (Ted Pella) were conjugated overnight with an 80:20 ratio of anti-human and anti-goat antibodies (Jackson Immuno Research). Unreacted sites were blocked using 0.1% (w / v) aqueous polysorbate 20. The conjugated gold nanoparticles were then concentrated by centrifugation and removal of 95% of the supernatant. Analysis was performed with 5 μl of concentrated conjugated gold nanoparticles in PBS (20 mM phosphate, 150 mM NaCl, pH 7.1) at a protein concentration of 0.05 mg / mL. After 2 h of incubation, absorbance spectra from 400 to 600 nm were collected using a Spectrostar Nano plate reader with 2 nm steps. The wavelengths of maximum of the spectral peaks were reported.

[0358] Relative Solubility

[0359] Solubility was assessed according to the method described above (Kerwin, 2019). Analysis was performed in PBS buffer (20 mM sodium phosphate and 150 mM sodium chloride pH 7.1) and final PEG 10,000 concentrations ranging from 0% to 12%. Remaining soluble protein after PEG incubation has been reported.

[0360] Size-exclusion high-performance liquid chromatography

[0361] Size-exclusion high performance liquid chromatography (SEC) was performed on a Dionex UltiMate 3000 HPLC system using a Waters XBridge Protein BEH SEC 200 Angstrom, 3.5 μm column and a diode array detector collected at 280 nm. Separation was performed under native conditions using a mobile phase buffer of 100 mM sodium phosphate, 250 mM sodium chloride, 10% acetonitrile v / v (pH 6.8).

Claims

1. below: generating, by a computer system including one or more computing devices having one or more processors and memory, and using a first generating element of a generating adversarial network, a plurality of first amino acid sequences, each of the plurality of first amino acid sequences corresponding to a light chain of an antibody; generating, by said computer system and using a second generating member of said generating adversarial network, a plurality of second amino acid sequences, each second amino acid sequence of said second amino acid sequences corresponding to a heavy chain of an antibody; generating, by said computer system and using said generated allele network, a third amino acid sequence by combining a first amino acid sequence of said plurality of first amino acid sequences with a second amino acid sequence of said plurality of second amino acid sequences, said third amino acid sequence corresponding to an antibody comprising a light chain corresponding to said first amino acid sequence and a heavy chain corresponding to said second amino acid sequence; analyzing, by the computer system and using the generated adversarial network, the third amino acid sequence against a further plurality of amino acid sequences included in training data for the generated adversarial network; and generating, by said computer system, an output indicative of a measure of similarity between said third amino acid sequence and at least a portion of said further plurality of amino acid sequences; A method comprising:

2. The method described in claim 1, wherein the step of combining the first amino acid sequence with the second amino acid sequence includes linking the second amino acid sequence to the first amino acid sequence.

3. the first generator implements a first model to generate a plurality of first amino acid sequences, and the second generator implements a second model to generate a plurality of second amino acid sequences; the first model includes a first function having one or more first variables and one or more first weights; and the second model includes a second function different from the first function, the second function having one or more second variables and one or more second weights; The method according to claim 1 or 2.

4. The method described in claim 3, wherein the third amino acid sequence is analyzed by a discriminator and an output is provided to at least one of the first generation element or the second generation element.

5. The method described in claim 4, wherein the first generating element modifies the first model based on the output.

6. The method of claim 4 , wherein a second generator modifies the second model based on the output.

7. A method according to any one of claims 1 to 6, wherein the first amino acid sequence comprises at least a portion of a first variable region of an antibody light chain, and the second amino acid sequence comprises at least a portion of a first variable region of an antibody heavy chain.

8. A method according to any one of claims 1 to 7, wherein the first amino acid sequence comprises at least a portion of a first variable region and a first constant region of an antibody light chain, and the second amino acid sequence comprises at least a portion of a second variable region and a second constant region of an antibody heavy chain.

9. below: determining, by the computer system and based on the output, that training of the first model is complete, such that the first model is a first trained model; determining, by the computer system and based on the output, that training of a second model is complete, such that a second model is a second trained model; generating, by said computer system and using said first training model, a first further amino acid sequence of a further light chain of the antibody; generating, by said computer system and using said second training model, a second further amino acid sequence of a further heavy chain of the antibody; and generating a third additional amino acid sequence by combining the first additional amino acid sequence with the second additional amino acid sequence, said third additional amino acid sequence comprising an additional antibody light chain and a heavy chain; The method according to any one of claims 1 to 8.

10. The method of claim 9, further comprising a step of evaluating the third further amino acid sequence with respect to one or more indicators by the computer system, wherein the one or more indicators include at least one of at least one of a number of hydrophobic amino acids contained in the third further amino acid sequence, a number of positively charged amino acids contained in the third further amino acid sequence, a number of negatively charged amino acids contained in the third further amino acid sequence, a number of uncharged amino acids contained in the third further amino acid sequence, an expression level of the third further amino acid sequence, a melting temperature of the third further amino acid sequence, or a self-aggregation level of the third further amino acid sequence.

11. analyzing, by said computer system and using said generated enemy network, said third additional amino acid sequence with an additional plurality of amino acid sequences to generate an additional output, wherein: the additional plurality of amino acid sequences are included in additional training data for the generated allele network; the additional training data includes amino acid sequences of antibodies that are different from the amino acid sequences included in the training data; and the further output indicating a further measure of similarity between the third further amino acid sequence and at least a portion of the further plurality of amino acid sequences.

11. The method according to claim 9 or 10.

12. one or more hardware processors; and 1. A system including one or more non-transitory computer-readable media storing computer-executable instructions that, when executed by one or more hardware processors, cause the one or more processors to perform the following operations: generating a plurality of first amino acid sequences, each first amino acid sequence of said plurality of first amino acid sequences corresponding to a light chain of the antibody, using a first generating member of the generating adversarial network; generating a plurality of second amino acid sequences, each second amino acid sequence of the plurality of second amino acid sequences corresponding to a heavy chain of an antibody, using a second generating member of said generating adversarial network; combining a first amino acid sequence of the plurality of first amino acid sequences with a second amino acid sequence of the plurality of first amino acid sequences using the generated allele network to result in a third amino acid sequence, wherein the third amino acid sequence corresponds to an antibody comprising a light chain corresponding to the first amino acid sequence and a heavy chain corresponding to the second amino acid sequence; analyzing the third amino acid sequence with respect to a further plurality of amino acid sequences included in training data of the generated adversarial network; and generating an output indicative of a measure of similarity between said third amino acid sequence and at least a portion of said further plurality of amino acid sequences; A system that executes the above.

13. 13. The system of claim 12, wherein a second generating element is trained with a number of hobbled weights to reduce the rate at which the second generating element is trained relative to a further rate at which the second generating element whose weights are not hobbled is trained.

14. The system of claim 12 or 13, wherein the second generating element is trained by slowing the gradient of the second generating element.

15. The one or more non-transitory computer-readable media, when executed by the one or more hardware processors, cause the one or more hardware processors to perform the following operations: training the second production element for a first time period; determining that the quality of the plurality of first additional amino acid sequences produced at the end of said first period is at a first level; training a first production element for a second time period that includes the first time period and is longer than the first time period; determining that the quality of the second plurality of additional amino acid sequences produced at the end of said second period is at said first level; training the second production element for a third time period after the second time period; determining that the plurality of third additional amino acid sequences produced at the end of said third period of time is at a second level of quality; training the first production element for a fourth time period that includes the third time period and is greater than the third time period; determining that the quality of the plurality of fourth additional amino acid sequences produced at the end of said fourth period of time is at said second level; The system of any one of claims 12 to 14, further storing further computer executable instructions for performing further operations, including:

16. 16. The system of claim 15, wherein a total time required to train the second production element is less than a total time required to train the first production element.

Images