Identifying convergent antibody specificity sequence patterns

Abstract

This method uses variational autoencoders (VAEs) and deep generative modeling to learn meaningful representations from the immune repertoire. The system can map input sequences into a low-dimensional latent space, which reveals a large number of convergent sequence patterns. The system can identify patterns that reside in convergent clusters that are highly predictive of antigen exposure and / or antigen specificity. From the latent space, the system can generate novel functional antibody sequence variants in silico.

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority to U.S. Patent Application No. 62 / 843,010, filed May 3, 2019, which is incorporated by reference in its entirety. [Background technology]

[0002] Background of the disclosure Deep sequencing of antibody repertoires can be used in immunology, immunodiagnostics, and drug discovery processes. Identifying relevant information among these large datasets remains challenging. The central question is to what extent antibody sequences are convergently selected in individuals with different antigen exposures. Summary of the Invention

[0003] The solution can provide a meaningful representation from the immune repertoire of a mammalian subject, including an antigen-exposed subject, using a deep generative modeling approach, the variational autoencoder (VAE). Exemplary data is provided herein that demonstrates the application of this approach to the antibody repertoire of immunized mice. The system can map the antibody repertoire into a low-dimensional latent space, which reveals a large amount of convergent sequence patterns. The system can use a combination of a linear classifier and the variational autoencoder (VAE) with a mixture model to identify patterns present in convergent clusters that predict antigen exposure. In some embodiments, the system further includes the use of variational deep embedding (VaDE). In some embodiments, the mixture model is a Gaussian mixture model. The system can also use a linear classifier and the VAE, followed by a separate clustering step in the latent space, to identify patterns present in convergent clusters that predict antigen exposure. The converged antibody sequences can then be expressed in a recombinant antibody expression system (e.g., as full-length IgG in a mammalian display system) and demonstrated to be antigen-specific using techniques such as flow cytometry and enzyme-linked immunosorbent assay (ELISA). The system can also unravel convergent sequence space by generating thousands of novel and functional variants in silico.

[0004] According to at least one aspect of the present disclosure, a method can include providing a candidate identification system with a plurality of input amino acid sequences representing antigen-binding portions of an antibody. The method can include converting the plurality of input amino acid sequences into a latent space by an encoder executed by the candidate identification system. The method can include determining a plurality of sequence clusters in the latent space by a clustering engine executed by the candidate identification system. The method can include identifying convergent clusters by the clustering engine. The method can include selecting, by a candidate generation engine executed by the candidate identification system, samples in the latent space defined by the convergent clusters. The method can include generating candidate sequences by the candidate generation engine using a decoder based on the samples in the latent space.

[0005] In some implementations, the decoder may include a plurality of long short-term recurrent neural networks, and generating candidate sequences may include providing samples to each of the plurality of long short-term recurrent neural networks. In some implementations, converting the plurality of input amino acid sequences into a latent space may include converting the plurality of input amino acid sequences into a latent space using a combination of a linear classifier and a variational autoencoder and a mixture model. In some implementations, the system may use variational deep embedding (VaDE). In some implementations, the system may use one or more dense layers or long short-term memory layers. Determining the plurality of sequence clusters further includes determining the plurality of sequence clusters using a mixture model, such as Gaussian mixture modeling (GMM).

[0006] In some implementations, the system can include a memory storing processor-executable instructions and one or more processors. The system can receive a plurality of input amino acid sequences representing antigen-binding portions of antibodies by an encoder executed by the one or more processors. The system can convert the plurality of input amino acid sequences into a latent space by the encoder. The system can determine a plurality of sequence clusters in the latent space by a clustering engine executed by the one or more processors. The system can identify convergent clusters by the clustering engine. The system can select samples in the latent space defined by the convergent clusters by a candidate generation engine executed by the one or more processors. The system can generate candidate sequences by the candidate generation engine based on the samples in the latent space.

[0007] In some implementations, the candidate generation engine can include a decoder having multiple long-term short-term recurrent neural networks. The encoder can convert multiple input amino acid sequences into a latent space using a combination of a linear classifier and a variational autoencoder and a mixture model. In some implementations, the system can use variational deep embedding (VaDE). The clustering engine can determine multiple sequence clusters using a mixture model such as GMM.

[0008] The input amino acid sequence may be derived from any mammalian subject, including humans and non-human animals. The input amino acid sequence may be derived from a healthy subject or a subject with a disease or condition (e.g., a pathogenic infection, cancer, an autoimmune disease, an allergic reaction, or inflammation). The input amino acid sequence may be derived from a subject that has been previously exposed to an antigen. The input amino acid sequence may be derived from a healthy subject that has previously had a disease or condition (e.g., a pathogenic infection, cancer, an autoimmune disease, an allergic reaction, inflammation, or an inflammatory disease). The input amino acid sequence may be derived from an immunized subject, e.g., a subject that has received a vaccine.

[0009] The input amino acid sequence may comprise any antigen-binding portion of an antibody. In some embodiments, the input amino acid sequence comprises one or more complementarity determining regions (CDRs). In some embodiments, the input amino acid sequence comprises one or more heavy chain CDRs, e.g., CDRH1, CDRH2, CDRH3, or any combination thereof. In some embodiments, the input amino acid sequence comprises one or more light chain CDRs, e.g., CDRH1, CDRH2, CDRH3, or any combination thereof. In some embodiments, the input amino acid sequence comprises one or more heavy chain CDRs and one or more heavy chain CDRs. In some embodiments, the input amino acid sequence comprises one or more framework regions of a heavy and / or light chain variable region. In some embodiments, the input amino acid sequence comprises a full-length heavy chain variable region. In some embodiments, the input amino acid sequence comprises a full-length light chain variable region. In some embodiments, the input amino acid sequence comprises one or more constant regions of a heavy and / or light chain. In some embodiments, the input amino acid sequence comprises a full-length heavy chain or an antigen-binding portion thereof. In some embodiments, the input amino acid sequence comprises a full-length light chain or an antigen-binding portion thereof.

[0010] Also provided herein are proteins or peptides comprising the amino acid sequence produced by the methods provided herein. In some embodiments, the generated amino acid sequence is a heavy or light chain of an antibody, or any portion thereof. In some embodiments, the generated amino acid sequence comprises one or more complementarity determining regions (CDRs). In some embodiments, the generated amino acid sequence comprises CDRH1, CDRH2, CDRH3, or any combination thereof. In some embodiments, the generated amino acid sequence comprises CDRL1, CDRL2, CDRL3, or any combination thereof. In some embodiments, the protein or peptide comprising the amino acid sequence produced herein is an antibody or a fragment thereof. In some embodiments, the protein or peptide comprising the amino acid sequence produced herein is a full-length antibody. In some embodiments, the protein or peptide comprising the amino acid sequence produced herein is a fusion protein comprising one or more portions of an antibody. In some embodiments, the protein or peptide comprising the amino acid sequence produced herein is an scFv or Fc fusion protein. In some embodiments, the protein or peptide comprising the amino acid sequence produced herein is a chimeric antigen receptor. In some embodiments, the protein or peptide comprising the amino acid sequence produced herein is a recombinant protein. In some embodiments, the protein or peptide comprising the amino acid sequence generated herein binds to an antigen. In some embodiments, the antigen is associated with a disease or condition. In some embodiments, the antigen is a tumor antigen, an inflammatory antigen, a pathogenic antigen (e.g., a virus, a bacteria, a yeast, a parasite). In some embodiments, the protein or peptide comprising the amino acid sequence generated herein has one or more improved properties compared to a protein or peptide comprising the input amino acid sequence. In some embodiments, the protein or peptide comprising the amino acid sequence generated herein has improved affinity for an antigen compared to a protein or peptide comprising the input amino acid sequence.In some embodiments, the proteins or peptides comprising the amino acid sequences generated herein can be administered to treat inflammatory diseases, infectious diseases, cancer, genetic disorders, organ transplant rejection, autoimmune diseases, or immune disorders. In some embodiments, the proteins or peptides comprising the amino acid sequences generated herein can be used to manufacture medicaments for treating inflammatory diseases, infectious diseases, cancer, genetic disorders, organ transplant rejection, autoimmune diseases, and immune disorders. Also provided herein are cells comprising one or more proteins or peptides comprising the amino acid sequences generated herein. The cells can be mammalian cells, bacterial cells, yeast cells, or any cells capable of expressing proteins or peptides comprising the amino acid sequences generated herein. The cells can be immune cells, such as T cells (e.g., CAR-T cells). In some embodiments, the proteins or peptides comprising the amino acid sequences generated herein can be used to detect antigens in biological samples.

[0011] Also provided herein are proteins or peptides comprising one or more CDR sequences of the amino acid sequence shown in any of Figures 10, 12-13, or 18-22, or the amino acid sequence shown in any of Figures 10, 12-13, or 18-22. In some embodiments, the proteins or peptides comprising one or more CDR sequences of the amino acid sequence shown in any of Figures 10, 12-13, or 18-22, or the amino acid sequence shown in any of Figures 10, 12-13, or 18-225 are antibodies or fragments thereof. In some embodiments, the proteins or peptides comprising one or more CDR sequences of the amino acid sequence shown in any of Figures 10, 12-13, or 18-22, or the amino acid sequence shown in any of Figures 10, 12-13, or 18-225 are full-length antibodies. In some embodiments, the protein or peptide comprising one or more CDR sequences of the amino acid sequence shown in any of Figures 10, 12-13, or 18-22, or the amino acid sequence shown in any of Figures 10, 12-13, or 18-22, is a fusion protein comprising one or more portions of an antibody. In some embodiments, the protein or peptide comprising one or more CDR sequences of the amino acid sequence shown in any of Figures 10, 12-13, or 18-22, or the amino acid sequence shown in any of Figures 10, 12-13, or 18-22, is an scFv or Fc fusion protein. In some embodiments, the protein or peptide comprising one or more CDR sequences of the amino acid sequence shown in any of Figures 10, 12-13, or 18-22, or the amino acid sequence shown in any of Figures 10, 12-13, or 18-22, is a chimeric antigen receptor. In some embodiments, a protein or peptide comprising the amino acid sequence shown in any of Figures 10, 12-13, or 18-22, or one or more CDR sequences of the amino acid sequence shown in any of Figures 10, 12-13, or 18-22, is a recombinant protein.

[0012] In some embodiments, a protein or peptide that includes one or more CDR sequences of the amino acid sequence shown in any of Figures 10, 12, or 18-19, or the amino acid sequence shown in any of Figures 10, 12, or Tables 2-3, binds to an ovalbumin antigen. In some embodiments, a protein or peptide that includes one or more CDR sequences of the amino acid sequence shown in any of Figures 10, 12, or 18-19, or the amino acid sequence shown in any of Figures 10, 12, or 18-19, can be used to detect an ovalbumin antigen (e.g., in a biological sample).

[0013] In some embodiments, a protein or peptide comprising one or more CDR sequences of the amino acid sequence shown in any of Figures 10, 13, 20, or 21, or the amino acid sequence shown in any of Figures 10, 13, 20, or 21, binds to a RSV-F antigen. In some embodiments, a protein or peptide comprising one or more CDR sequences of the amino acid sequence shown in any of Figures 10, 13, 20, or 21, or the amino acid sequence shown in any of Figures 10, 13, 20, or 21, can be administered to treat a respiratory syncytial virus infection. In some embodiments, a protein or peptide comprising one or more CDR sequences of the amino acid sequence shown in any of Figures 10, 13, 20, or 21, or the amino acid sequence shown in any of Figures 10, 13, 20, or 21, can be used to manufacture a medicament for treating a respiratory syncytial virus infection. In some embodiments, a protein or peptide containing an amino acid sequence shown in any of Figures 10, 13, 20, or 21, or one or more CDR sequences of the amino acid sequence shown in any of Figures 10, 13, 20, or 21, can be used to detect RSV-F antigen (e.g., in a biological sample).

[0014] Also provided herein are cells comprising one or more proteins or peptides comprising an amino acid sequence shown in any of Figures 10, 12, 13 or 18-22, or one or more CDR sequences of the amino acid sequence shown in any of Figures 10, 12, 13 or 18-22. The cells can be mammalian cells, bacterial cells, yeast cells, or any cells capable of expressing a protein or peptide comprising an amino acid sequence shown in any of Figures 10, 12, 13 or 18-22, or one or more CDR sequences of the amino acid sequence shown in any of Figures 10, 12, 13 or 18-22. The cells can be immune cells, such as T cells (e.g., CAR-T cells).

[0015] [The present invention 1001] providing a plurality of input amino acid sequences representing antigen-binding portions of a plurality of antigen-binding molecules to a candidate identification system; transforming the plurality of input amino acid sequences into a latent space by an encoder executed by the candidate identification system; determining a plurality of clusters of sequences in the latent space by a clustering engine implemented by the candidate identification system; identifying, by the clustering engine, convergent clusters; selecting, by a candidate generation engine executed by the candidate identification system, samples in the latent space defined by the converged clusters; generating candidate amino acid sequences by the candidate generation engine using a decoder based on the samples in the latent space. [The present invention 1002] 1001. The method of claim 1001, wherein said antigen-binding molecule is an antibody or an antigen-binding fragment thereof. [The present invention 1003] 1001. The method of claim 10, wherein the antigen-binding molecule is a chimeric antigen receptor. [The present invention 1004] The method of claim 1002, wherein the input amino acid sequence represents a complementarity determining region (CDR). [The present invention 1005] The method of any one of claims 10 to 14, wherein the input amino acid sequence comprises a CDRH3 sequence. [The present invention 1006] 1005. The method of claim 1004 or 1005, wherein the input amino acid sequence comprises a CDRH1 sequence. [The present invention 1007] The method according to any one of claims 1004 to 1006, wherein the input amino acid sequence comprises a CDRH2 sequence. [The present invention 1008] The method according to any one of claims 1004 to 1007, wherein the input amino acid sequence comprises a CDRL1 sequence. [The present invention 1009] The method of any of claims 1004 to 1008, wherein the input amino acid sequence comprises a CDRL2 sequence. [The present invention 1010] The method of any of claims 1004 to 1009, wherein the input amino acid sequence comprises a CDRL3 sequence. [The present invention 1011] The method of any of claims 1004 to 1010, wherein said input amino acid sequence comprises a full-length heavy chain, or an antigen-binding portion thereof. [The present invention 1012] The method of any of claims 1004 to 1011, wherein the input amino acid sequence comprises a full-length light chain, or an antigen-binding portion thereof. [The present invention 1013] the decoder includes a plurality of long term short term recurrent neural networks; The method of any one of claims 1001 to 1012, wherein the step of generating candidate sequences further comprises providing the samples to each of the plurality of long-term short-term recurrent neural networks. [The present invention 1014] Any of the methods of claims 1001 to 1013, wherein the step of transforming the plurality of input amino acid sequences into the latent space further comprises transforming the plurality of input amino acid sequences into the latent space using variational deep embedding (VaDE). [The present invention 1015] Any of the methods of claims 1001 to 1014, wherein the step of determining the plurality of sequence clusters further comprises determining the plurality of sequence clusters using mixture modeling. [The present invention 1016] The method of the present invention 1015, wherein the mixture modeling includes Gaussian mixture modeling (GMM). [The present invention 1017] a memory for storing processor-executable instructions; One or more processors Including, The one or more processors: receiving, by an encoder executed by said one or more processors, a plurality of input amino acid sequences representing antigen-binding portions of antibodies; Transforming the plurality of input amino acid sequences into a latent space by the encoder; determining, by a clustering engine executed by the one or more processors, a plurality of clusters of sequences in the latent space; identifying convergent clusters by said clustering engine; selecting, by a candidate generation engine executed by the one or more processors, samples in the latent space defined by the convergent clusters; generating, by the candidate generation engine, candidate sequences based on the samples in the latent space. [The present invention 1018] The system of the present invention 1017, wherein the candidate generation engine includes a decoder having a plurality of long-term short-term recurrent neural networks. [The present invention 1019] The system of any one of claims 1017 to 1018, further comprising the encoder that converts the plurality of input amino acid sequences into the latent space using variational deep embedding (VaDE). [The present invention 1020] The system of any one of claims 1017 to 1019, comprising the clustering engine that determines the plurality of sequence clusters using Gaussian mixture modeling (GMM). [The present invention 1021] A protein or peptide, wherein the amino acid sequence of the protein or peptide is a candidate amino acid sequence generated by any of the methods of the present inventions 1001 to 1016 or any of the systems of the present inventions 1017 to 1020. [The present invention 1022] The protein or peptide of the present invention, wherein said protein or peptide binds to an antigen. [The present invention 1023] The protein or peptide of the present invention 1021 or 1022, wherein said protein or peptide is a chimeric antigen. [The present invention 1024] The protein or peptide of the present invention 1021 or 1022, wherein the generated candidate amino acid sequences comprise CDRH1, CDRH2, CDRH3, or any combination thereof. [The present invention 1025] 15. The protein or peptide of any of claims 1021, 1022, or 1024, wherein the generated candidate amino acid sequences comprise CDRL1, CDRL2, CDRL3, or any combination thereof. [The present invention 1026] 1026. The protein or peptide of any of claims 1011, 1022, 1024, or 1025, wherein said protein or peptide comprises an antibody or an antigen-binding fragment thereof. [The present invention 1027] 1027. The protein or peptide of any one of claims 1021 to 1026, wherein said protein or peptide is a fusion protein comprising one or more portions of an antibody. [The present invention 1028] The protein or peptide of any one of claims 1021 to 1026, wherein the protein or peptide comprises an scFv or Fc fusion protein. [The present invention 1029] 9. The protein or peptide of any one of claims 1021 to 1028, wherein the antigen is associated with a disease or condition. [The present invention 1030] The protein or peptide of the present invention, wherein said antigen is a tumor antigen. [The present invention 1031] The protein or peptide of the present invention, wherein the antigen is an anti-inflammatory antigen. [The present invention 1032] The protein or peptide of the present invention, wherein said antigen is a pathogenic antigen selected from a viral, bacterial, yeast, or parasitic antigen. [The present invention 1033] A protein or peptide comprising an amino acid sequence shown in any one of Figures 10, 12, 13 or 18 to 22. [The present invention 1034] A protein or peptide comprising one or more CDR sequences of the amino acid sequence shown in any of Figures 10, 12, 13, or 18 to 22. [The present invention 1035] A cell comprising any one of the proteins or peptides according to the present invention 1021 to 1034. [The present invention 1036] A cell comprising a nucleic acid sequence encoding any one of the proteins or peptides of the present invention 1021 to 1034. [The present invention 1037] The cell of claim 1035 or 1036, wherein the cell is a mammalian cell, a bacterial cell, a yeast cell, an insect cell, or a eukaryotic cell. [The present invention 1038] The cell of the present invention, wherein the cell is an immune cell. [The present invention 1039] The cell of the present invention, wherein the immune cell is a T cell. [The present invention 1040] The cell of the present invention, wherein the T cell is a CAR-T cell. [The present invention 1041] The protein or peptide of any of claims 1021 to 1034, or the cell of any of claims 1035 to 1040, wherein the protein or peptide, or the cell is administered to a subject to treat an inflammatory disease, an infectious disease, a cancer, a genetic disorder, an organ transplant rejection, an autoimmune disease, or an immune disorder. [The present invention 1042] The protein or peptide of any of claims 1021 to 1034, or the cell of any of claims 1035 to 1040, wherein the protein or peptide, or the cell is used for the manufacture of a medicament for treating an inflammatory disease, an infectious disease, cancer, a genetic disorder, an organ transplant rejection, an autoimmune disease, or an immune disorder. [The present invention 1043] Use of a protein or peptide according to any one of claims 1021 to 1034 for detecting an antigen in a biological sample. [The present invention 1044] Use of a protein or peptide according to any one of claims 1021 to 1034 for detecting an antigen in a subject in vivo. The foregoing summary and the following detailed description of the invention and detailed explanation of the drawings are exemplary and explanatory and are intended to provide further explanation of the invention as claimed. Other objects, advantages and novel features will become readily apparent to those skilled in the art from the following brief description of the drawings and detailed description. [Brief description of the drawings]

[0016] The accompanying drawings are not intended to be drawn to scale. Like reference numbers and designations in the various drawings indicate like elements. For purposes of clarity, every component may not be labeled in every drawing.

[0017] [Figure 1] 1 illustrates a block diagram of an exemplary candidate identification system. [Diagram 2] Flow diagram for generating in silico sequences. Antibody repertoires from bone marrow of 45 BALB / c mice immunized with various antigens are sequenced. The antibody sequences are then used to train a deep generative model that can both generate novel sequences and assign input sequences to different clusters based on their potential embeddings. Cluster assignments are used to identify sequences that are highly enriched in a particular repertoire or antigen cohort. Natural and in silico generated sequences of antigen-associated clusters are expressed as full-length IgG and validated as binding antibodies. The figure discloses SEQ ID NOs: 41-46, 37, and 47-52, respectively, in order of appearance. [Diagram 3] 2 illustrates an exemplary encoder architecture that may be used in the system illustrated in FIG. [Figure 4] 2 illustrates an exemplary decoder architecture that may be used in the system illustrated in FIG. 1 . [Diagram 5]2 shows an exemplary deep neural network of a variational autoencoder. The gray boxes indicate the input to the model, and the light red boxes indicate various (mathematical) operations. The purple boxes highlight the trainable layers of the model. The dark red indicates the output of the model. The gray boxes include layers where the weights are shared across all cluster dimensions. The variational autoencoder can receive CDR1, CDR2, and CDR3 as inputs. To handle CDRHs of various lengths, the system pads the array with dashes until it reaches a certain fixed length (the maximum length of each CDRH in the data). The system one-hot encodes and concatenates the padded array, and uses this as input to the variational autoencoder (VAE). As shown in FIG. 2, the VAE includes both a dense layer (e.g., a nonlinear activation function) and a linear layer. The dense layer can include, for example, filters or units ranging from 256 to 512 or other amounts. The linear layer can include 10 units, or some other number of units. [Figure 6] Identification and characterization of antigen-associated sequences. (A) 10-dimensional latent space of two antibody repertoires visualized by principal component analysis (PCA). Blue and red dots indicate sequences belonging to one OVA (2C) and RSV-F (2C) repertoire, respectively. In the zoomed-in area, two learned clusters containing only sequences specific to one repertoire and their respective sequence motifs are highlighted. (B) The antibody repertoire is converted to a vector based on the learned sequence clusters in the latent space. The recoded vectors are used as input for a linear support vector machine (SVM) classifier of antigen exposure. The confusion matrix shows the aggregate prediction results of each model during 5-fold cross-validation with cluster labels and raw sequences as features. (C) Heatmap includes all predicted and convergent sequence clusters for each cohort. The red dashed lines indicate mice that received only primary immunization. (D) Exemplary sequence logos of convergent clusters found in each antigen cohort. [Figure 7]Cluster-specific sequences across various repertoires containing antigen-specific antibodies are shown. (A) Dose-dependent absorbance curves for all antigens of supernatants prepared from four antigen-related heavy chain pools. (B) Flow cytometry histograms of six monoclonal cell populations each utilizing a different convergent OVA-related or RSV-F-related VH. Grey histograms represent negative controls, while colored histograms represent convergent antibodies. (C) Flow cytometry histograms of 12 monoclonal cell populations of convergent variants (CVs) using different VH sequences from the same cluster as RSV3. (D) Table shows the CDRH3 of selected CVs and the RSV-F immunized mouse repertoires in which they were found. Red letters indicate differences from the sequence of the originally found sequence RSV3. (E) Scatter plot shows the frequency rank distribution of CVs from the RSV3 cluster by mouse repertoire. Red dots highlight VHs confirmed to bind in (C). (F) Pie chart showing the 9 most utilized V gene germline in convergent clones for both RSV-F and OVA. The figure discloses SEQ ID NOs: 53-65, respectively, in order of appearance. [Figure 8] Deep generative modeling and in silico antibody sequence generation. (A) Schematic deep generative modeling of antibody sequence space, where clusters are selected or randomly sampled, based on selected parameters, and random samples are taken from a multivariate normal distribution. The encoder then converts the encodings to a multivariate polynomial distribution from which novel sequences are sampled. (B) The scatter plot shows two potential naturally occurring variants, and the yellow dots indicate the 10 most frequently in silico sampled encodings that were confirmed to be binding antibodies. The table on the right shows their CDRH3 sequences and their counts after 1,000,000 samplings (sequence numbers 66-75, respectively, in order of appearance). Red letters indicate differences relative to the initial biological sequence (RSV3, shown in black). [Figure 9]An exemplary work flow for generating and testing new VH sequences selected by the provided deep generative model is shown. Candidate heavy chains are selected from the bulk heavy chain sequencing dataset for each antigen based on the implemented bioinformatics sequence clustering framework. Sequences are gene synthesized and cloned into HDR donor vectors (step 1). For each antigen, light chain repertoires are amplified from RNA of mice immunized with the same antigen by multiplex PCR. The resulting light chain libraries are then cloned into the HDR donor vectors generated in step 1 to generate separate HDR donor VL libraries for each heavy chain (step 2). The resulting HDR donor libraries are then used as DNA repair templates for CRISPR / Cas9-based integration into PnP mRuby / Cas9 cells, thereby generating a library of hybridoma cell clones expressing antibodies with the same candidate heavy chain but different light chains. Antigen-specific clones are enriched by fluorescence-activated cell sorting. [Figure 10-1] Flow cytometry analysis of hybridoma libraries. (A)-(B) OVA, and (C)-(D) RSV-F hybridoma cell libraries, with sequence library enrichment dot plots shown in (A) and (C). Each antigen-specific monoclonal cell line is shown in histogram plots (B) and (D) relative to a negative control cell line that is not specific for the given antigen. FIG. 10B discloses SEQ ID NOs: 76-81, 79, 82-83, 79, 82, 84, 47-48, 85, 79, 82, and 86, and FIG. 10D discloses SEQ ID NOs: 87-95, in order of appearance, respectively. [Figure 10-2] See description of Figure 10-1. [Figure 11] ELISA data of convergent sequences confirmed to be antigen specific are shown. Supernatant ELISA profiles of antigen specific hybridoma monoclonal cell lines are shown for (A) OVA and (B) RSV-F. The starting cell line PnP-mRuby / Cas9 was used as a negative control. [Figure 12]Alignment of convergent sequences confirmed to be antigen-specific. VH amino acid alignment for antigen-specific antibodies. (A) Full-length VDJ alignment is shown for OVA and RSV variants (SEQ ID NOs: 96-104, respectively, in order of appearance). (B) Concatenated CDRH1-CDRH2-CDRH3 amino acid alignment of OVA and RSV-F (sequences of the CDRs are not intended to be contiguous). The color code used is derived from the Clustal coloring scheme using Geneious V10.2.6 software. The figure discloses the CDR sequences as SEQ ID NOs: 76-81, 79, 82-83, 79, 82, 84, 47-48, 85, 79, 82, and 86-95, respectively, in order of appearance. [Figure 13] Shown are amino acid sequence alignments of convergent variants from the RSV3 cluster (SEQ ID NOs: 105-116, and 104, respectively, in order of appearance). VH amino acid alignment of convergent natural variants (NVs) from the RSV3 cluster. Color codes used are derived from the Clustal coloring scheme in Geneious V10.2.6. [Figure 14] Figure 1. Reconstruction accuracy of the variational autoencoder. The bar chart shows the achieved reconstruction accuracy as a function of the number of clusters. Increasing the amount of clusters (k) improves the reconstruction accuracy, with diminishing returns after k=2000. [Figure 15] The workflow of the RSV3 CDRH3 antibody library screening workflow is shown. (A) The RSV3 CDRH3 library was generated by CRISPR-Cas9 homology-directed mutagenesis using ssODNs with degenerate codons representing the sequence space depicted by the logo shown. (B) Transfected cells were then selected in two consecutive steps for antibody expression and specificity or negativity to RSV-F. [Figure 16]Sampling results from an RSV3-generated CDRH3 library are shown. The histograms show the likelihood of sequences from the positive (blue) and negative (red) fractions of the RSV3 CDRH3 library screen according to the VAE decoder model. Positive variants are slightly but significantly more likely to occur (P<0.001, Mann-Whitney U test). The green histogram on the right shows the probability of a variant being observed in the organism repertoire. [Figure 17] Deep sequencing results from RSV3 CDRH3 library screening are shown. Sequence logos show aggregated sequences found in (A) the positive fraction and (B) the negative fraction of the RSV3 CDRH3 library screen. [Figure 18] The sequences confirmed to bind to OVA are shown in the figure, which discloses the "HC" sequences as SEQ ID NOs: 96-101 and the "LC" sequences as SEQ ID NOs: 117-122, in order of appearance, respectively. [Figure 19]

[0036] Figure 1 shows the surrogate VL chain sequences of OVA1 and OVA5. The figure discloses SEQ ID NOs: 123 to 129, respectively, in order of appearance. [Figure 20]

[0033] Figure 1 shows sequences that have been confirmed to bind to RSV. The figure discloses the "HC" sequences as SEQ ID NOs: 102-104 and the "LC" sequences as SEQ ID NOs: 130-132, in order of appearance, respectively. [Figure 21] The surrogate VL chain sequences for RSV1, 2, and 3 are shown (SEQ ID NOs: 133 to 135, respectively, in order of appearance). [Figure 22]Convergent antibody sequences screened for antigen binding are shown. This table shows convergent sequences experimentally screened for antigen binding. The rightmost three columns indicate whether the sequence could be identified by each method. A sequence would have been found as a public clone if it was shared with at least one other mouse in its cohort but not observed in any other antigen cohort. The number in parentheses indicates the number of sequences found within the convergent cluster. The figure discloses "CDR1" sequences as SEQ ID NOs: 76, 79, 79, 79, 47, 79, 87, 90, 93, and 136, "CDR2" sequences as SEQ ID NOs: 77, 80, 82, 82, 48, 82, 88, 91, 94, and 137, and "CDR3" sequences as SEQ ID NOs: 138-143, 38-39, 53, and 40, respectively, in order of appearance. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0018] Detailed Description The various concepts introduced above and discussed in further detail below are not limited to any particular manner of implementation and can be implemented in any of numerous ways, and examples of specific implementations and applications are provided primarily for illustrative purposes.

[0019] The method described herein uses a deep generative modeling approach, variational autoencoder (VAE), to provide a meaningful representation from the immune repertoire of a mammalian subject, including a subject exposed to an antigen. The system can map the antibody repertoire into a low-dimensional latent space, which reveals a large amount of convergent sequence patterns. In some embodiments, the system can use linear classifiers and variational deep embedding (VaDE) to identify patterns present in convergent clusters that predict antigen exposure. In some embodiments, statistical tests such as t-tests, Fisher's exact test, or permutation tests are used instead of linear classifiers to test for statistical significance. The converged antibody sequences can then be expressed in a recombinant expression system (e.g., as full-length IgG in a mammalian display system) and demonstrated to be antigen-specific using techniques such as flow cytometry and enzyme-linked immunosorbent assay (ELISA). The system can also unravel the convergent sequence space by generating thousands of novel and functional variants in silico. The method can be applied to the development of therapeutic and diagnostic (target identification) antibody agents with improved properties.

[0020] FIG. 1 shows a block diagram of an exemplary system 100 for generating in silico sequences, which may be referred to as candidate sequences. The candidate identification system 102 may include one or more processors 104 and one or more memories 106. The processor 104 may execute processor-executable instructions to perform the functions described herein. The processor 104 may execute an encoder 108, a clustering engine 110, a decoder 112, and a candidate selection engine 114. The memory 106 may store the processor-executable instructions, generate data, and store collected data. The memory 106 may store one or more classifier weights 122. The memory 106 may also store classification data 116, training data 118, and candidate data 120.

[0021] The system 100 can include one or more candidate identification systems 102. The candidate identification system 102 can include at least one logical device, such as a processor 104. The candidate identification system 102 can include at least one memory element 106 that can store data and processor-executable instructions. The candidate identification system 102 can include multiple computing resources or servers located in at least one data center. The candidate identification system 102 can include multiple logically grouped servers to facilitate distributed computing techniques. A logical grouping of servers can be referred to as a data center, a server farm, or a machine farm. The servers can also be geographically distributed. The candidate identification system 102 can be any computing device. For example, the candidate identification system 102 can be or include one or more laptops, desktops, tablets, smartphones, portable computers, or any combination thereof.

[0022] The candidate identification system 102 may include one or more processors 104. The processors 104 may provide information processing capabilities to the candidate identification system 102. The processors 104 may include one or more of a digital processor, an analog processor, a digital circuit that processes information, an analog circuit designed to process information, a state machine, and / or other mechanisms for electronically processing information. Each processor 104 may include multiple processing units or processing cores. The processors 104 may be electrically coupled to the memory 106 and may execute the encoder 108, the clustering engine 110, the decoder 112, and the candidate generation engine 114.

[0023] The processor 104 may include one or more microprocessors, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), or combinations thereof. The processor 104 may be an analog processor and may include one or more resistive networks. The resistive network may include multiple inputs and multiple outputs. Each of the multiple inputs and each of the multiple outputs may be coupled to a nanowire. The input nanowire may be coupled to the output nanowire via a memory element. The memory element may include a ReRAM, a memristor, or a PCM. As an analog processor, the processor 104 may perform matrix vector multiplication using analog signals.

[0024] The candidate identification system 102 may include one or more encoders 108. The encoder 108 may be an application, applet, script, service, daemon, routine, or other executable logic for encoding an input sequence into a latent space. The encoder 108 may include a neural network autoencoder. The encoder 108 is described further with respect to FIG. 3, among others. In overview, the encoder 108 may receive an unlabeled input sequence and map (or encode) the input sequence into a lower dimensional space. The encoder 108 may encode the input sequence into a lower dimensional space, for example, using a variational autoencoder (VAE). In some embodiments, the encoder uses variational deep embedding (VaDE). The encoder 108 may map the input sequence into, for example, a five-dimensional space. In some embodiments, the encoder may jointly optimize a deep generative model along with a mixture model, such as a Gaussian Mixture Model (GMM)-based clustering of the latent space.

[0025] The candidate identification system 102 may include one or more clustering engines 110. The clustering engine 110 may be an application, applet, script, service, daemon, routine, or other executable logic for determining clusters in a latent space. The clustering engine 110 may use K-means clustering to identify clusters generated by the encoder 108 from the input sequences in the latent space. The clustering engine 110 may use Gaussian mixture modeling (GMM) to identify clusters in the latent space.

[0026] The candidate identification system 102 may include one or more decoders 112. The decoder 112 may be an application, applet, script, service, daemon, routine, or other executable logic that may decode or otherwise create an output sequence from an input in a latent space. The decoder 112 is described further with respect to FIG. 4, among others. The decoder 112 may receive samples from the latent space and reconstruct a sequence (e.g., CDR1, CDR2, or CDR3). For example, the decoder 112 may convert the latent space samples into a one-hot encoding matrix that represents the CDR1, CDR2, or CDR3 sequence. In some implementations, the decoder 112 may include multiple different neural networks. For example, the decoder 112 may include a different neural network for each of the sequences generated from the latent space samples. The decoder 112 may include a different neural network for generating each of the CDR1, CDR2, and CDR3 sequences. The neural network of the decoder 112 may be a long short-term recurrent neural network.

[0027] The candidate identification system 102 may include a candidate generation engine 114. From the clusters identified by the clustering engine 110, using the decoder 112, the candidate generation engine 114 may generate in silico output sequences. For example, the candidate generation engine 114 may select samples from a latent space. The candidate generation engine 114 may select samples from within predefined clusters in the latent space. The candidate generation engine 114 may provide the samples to the decoder 112 to generate in silico sequences as output, which the candidate generation engine 114 may store in memory as candidate data 120.

[0028] The candidate identification system 102 may include one or more memories 106. The memory 106 may be or may include a memory element. The memory 106 may store machine instructions that, when executed by the processor 104, may cause the processor 104 to perform one or more of the operations described herein. The memory 106 may include, but is not limited to, an electronic, optical, magnetic, or any other storage device capable of providing instructions to the processor 104. The memory 106 may include, but is not limited to, a floppy disk, a CD-ROM, a DVD, a magnetic disk, a memory chip, a ROM, a RAM, an EEPROM, an EPROM, a flash memory, an optical medium, or any other suitable memory from which the processor 104 may read instructions. The instructions may include code from any suitable computer programming language, such as, but not limited to, C, C++, C#, Java, JavaScript, Perl, HTML, XML, Python, and Visual Basic.

[0029] The candidate identification system 102 can store the classifier weights 122 in the memory 106. The classifier weights 122 can be a data structure that includes weights and biases that define the neural networks of the encoder 108 and the decoder 112. Once trained, the classification engine 108 can, for example, store the classifier weights 122 in the memory 106 for later use in searching and generating in silico sequences.

[0030] During the training phase, the encoder 108 and the decoder 112 may process the training data 118 to generate weights and biases for one or more machine learning models in the encoder 108 and the decoder 112. Once trained, the encoder 108 and the decoder 112 may store the weights and biases in the memory 106 as classifier weights 122. The generation of the training data and training of the encoder 108 and the decoder 112 is further described below in relation to the memory 106, the training data 118, and examples. Once the encoder 108 and the decoder 112 are trained, the weights and biases may be saved in the memory 106 as classifier weights 122. The models of the classification engine 108 (e.g., convolutional neural networks, dense layers, and LSTM neural networks) are further described in particular with reference to FIG. 2 and FIG. 3.

[0031] FIG. 2 shows a flow diagram 200 for generating in silico sequences, for example, using the system illustrated in FIG. 1. The flow diagram 200 includes three stages. During the first stage 202, training or testing data is generated. During the second stage 204, deep embedding can be performed to train the encoder 108. During the third stage 206, the candidate generation engine 114 can identify antigen-associated clusters and then generate in silico sequences. For example, as further described in connection with the Examples section, antibody repertoires from the bone marrow of 45 BALB / c mice immunized with various antigens can be sequenced to generate the training data 118. The candidate identification system 102 can use the training data 118 to train the encoder 108 and the decoder 112 during the second stage 204. The trained encoder 108 can assign input sequences to distinct clusters based on the potential embeddings of the sequences. During the third stage 306, the candidate generation engine 114 can identify clusters enriched in a particular repertoire or antigen cohort. The candidate generation engine 114 can generate in silico sequences from the antigen-associated clusters.

[0032] FIG. 3 illustrates an exemplary architecture 300 of the encoder 108. The encoder 108 may receive an input 302 at a first layer of the architecture 300. Although FIG. 3 illustrates an input of a sequence including CDR1, CDR2, and CDR3 sequences, the input sequence may be any other sequence. The architecture 300 may include a padding layer 304. The padding layer 304 may zero-pad, dash-pad, or otherwise pad the input sequences such that all input sequences have the same length. For example, different variations of CDR1, CDR2, or CDR3 may have different sequence lengths. The padding layer 304 may add zeros, dashes, or other values ​​to the ends of variants that have a length shorter than the longest variant of each of the respective CDR1, CDR2, and CDR3 sequences. Each sequence that exits the padding layer 204 may have a predetermined length (or size). The architecture 300 may include a one-hot encoding layer 306 that converts the padded input sequence (output from the padding layer 304) into a one-hot encoding matrix. The one-hot encoding layer 306 may generate a one-hot encoding matrix that includes, for example, a row for each position of the padded input sequence. Each column of the one-hot encoding matrix may correspond to a different possible amino acid, thereby filling each respective value of the padded input sequence. In this example, the one-hot encoding matrix includes 21 columns, since there are 20 amino acids and another column for the padded value (e.g., 0) to be added to the sequence. Each row of the one-hot encoding matrix includes a 1 in the column corresponding to the amino acid present in each value of the padded input sequence. In some implementations, the one-hot encoding layer 306 may be an encoding layer that may use an encoding different from one-hot encoding, such as BLOSUM26 or Blomap.

[0033] The architecture 300 may include a concatenation layer 308, which concatenates the one-hot encoding matrices of CDR1, CDR2, and CDR3 (in this example) into a single one-hot encoding matrix.

[0034] The architecture 300 may include multiple interconnected layers 310, which may be trainable layers. Each of the layers 310 may include one or more neurons. As shown in FIG. 3, a portion of the layers 310 may include 21 neurons and a portion of the layers 310 may include 64 units. The architecture 300 may include multiple computation layers 312, which may combine or otherwise perform mathematical operations on the output from the layers 310. The architecture 300 may include a trainable normalization layer 314. The architecture 300 may include a layer 316 that flattens the output of the normalization layer 314 to generate an output vector, which may be fully interconnected with a layer 318 that includes multiple rectified linear units (ReLUs).

[0035] FIG. 4 illustrates an exemplary decoder architecture 400 of the decoder 112 illustrated in FIG. 1. The architecture 400 can receive or select samples from a latent space. As illustrated in the exemplary architecture 400 of FIG. 4, the latent space can be a five-dimensional latent space. The architecture 400 can include a different neural network 402 for each sequence recreated by the decoder 112. In the illustrated example, the decoder 112 has generated (or otherwise generated) a sequence including CDR1, CDR2, and CDR3 in silico. Thus, in the example illustrated in FIG. 4, the architecture 400 can include three neural networks, each of which corresponds to a respective one on CDR1, CDR2, or CDR3. The neural network 402 can include a dense layer or can include a long short-term recurrent neural network (LSTM-RNN) layer. A dense layer in a neural network can refer to a layer that is nonlinear. The dense layer may include a linear form such as wx+b, but the final result of the dense layer may then be passed to a nonlinear function called an activation function, such as y=f(w*x+b). Examples of nonlinear activation functions include, for example, unit step, sign, piecewise linear, logistic, hyperbolic tangent, regularized linear unit, or regularized softmax. For example, the output of each of the neural networks 402 may be input to a feedforward layer 404 using a softmax activation. The output of the layer 404 may be a one-hot encoding matrix, which uses the same one-hot encoding used in the encoder 108. The one-hot encoding output matrix may be converted to an array.

[0036] Once trained according to the methods described herein, deep learning models can be used to predict millions of antigen binders from extremely large libraries of in silico generated variants. These variants can be subjected to multiple developability filters, resulting in tens of thousands of optimized lead candidates. Due to its scalable throughput and ability to explore vast protein sequence space, the methods described herein can be applied to a wide variety of applications involving the engineering and optimization of antibodies and other protein-based therapeutics. EXAMPLES

[0037] Adaptive immunity can be driven by its ability to generate a highly diverse set of adaptive immune receptors (e.g., B-cell and T-cell receptors, as well as secreted antibodies), and the subsequent clonal selection and expansion of those receptors that can recognize foreign antigens. These principles can result in a unique, dynamic immune repertoire. Deep sequencing can provide evidence for the presence of commonly shared receptors across individual organisms within one species. Convergent selection of specific receptors for various antigens provides one explanation for the presence of common shared receptors across individual organisms. Convergent selection in mouse antibody repertoires can occur for a range of protein antigens and immunization conditions. In this example, variational encoding was specifically performed using a system similar to the system and architecture shown in Figures 1-3. In this example, a generative modeling technique that combines a variational autoencoder with a mixture model, such as Gaussian mixture model (GMM)-based clustering, is used. Using variational encoding, the system can map the antibody repertoire into a lower-dimensional latent space, allowing for the discovery of a large number of convergent antigen-specific sequence patterns (AASPs). Using a linear one-to-one support vector machine (SVM), the system identified sequence patterns that predicted antigen exposure with up to 95% accuracy. Recombinant expression of both native and variational encoding generated antibodies with AASP confirmed binding to the target antigen. This example illustrates that deep generative modeling can be applied to immunodiagnostics, as well as antibody discovery and engineering.

[0038] I. Results Targeted deep sequencing of rearranged B cell receptor (BCR) loci can reveal the repertoire of B cells or expressed antibodies in a given tissue or cell population. Using deep sequencing data, we analyzed the antibody repertoire in the bone marrow of 45 BALB / c mice, which were divided into cohorts immunized with either protein antigens: ovalbumin (OVA), chicken egg lysozyme (HEL), Blue Carrier Protein (BCP), or respiratory syncytial virus fusion protein (RSV-F). As shown in Figure 2 and outlined in Table 1, the OVA, HEL, and BCP cohorts were further subdivided into groups receiving zero, one, two, or three booster immunizations. Serum ELISA confirmed antigen-specific antibody titers in all mice, with mice receiving only primary immunizations showing significantly weaker titers. RNA was bulk extracted from bone marrow and variable heavy chain IgG sequencing libraries were prepared using a two-step RT-PCR protocol. Libraries were sequenced, quality processed, and aligned on an Illumina MiSeq, yielding a total of 243,374 unique combinations of all three complementarity determining regions (CDRs) across all mice.

[0039] Table 1. Immunization schedule TIFF0007602484000001.tif107161

[0040] Figure 6 shows a workflow to assess the extent to which convergence beyond exact sequence similarity occurs. Figure 6 shows the identification and characterization of antigen-associated sequences. (A) 10-dimensional latent space of two antibody repertoires visualized by principal component analysis (PCA). Blue and red dots indicate sequences belonging to one OVA (2C) and RSV-F (2C) repertoire, respectively. The enlarged region highlights two learned clusters that contain only sequences specific to one repertoire and their respective sequence motifs. (B) The antibody repertoire is converted to a vector based on the learned sequence clusters in the latent space. The recoded vectors are used as input for a linear support vector machine (SVM) classifier of antigen exposure. The confusion matrix shows the aggregate prediction results of each model during 5-fold cross-validation with cluster labels and raw sequences as features. (C) Heatmap includes all predicted and convergent sequence clusters for each cohort. The red dashed lines indicate mice that received only primary immunization. (D) Exemplary sequence logos of convergent clusters found in each antigen cohort.

[0041] Figure 7 shows cluster-specific sequences across various repertoires. (A) Dose-dependent absorbance curves of supernatants prepared from hybridoma cells expressing antibodies with convergent variable heavy chain (VH) pools for each antigen. (B) Flow cytometry histograms of six monoclonal cell populations each utilizing a different convergent OVA-related or RSV-F-related VH. Grey histograms represent negative controls, and colored histograms represent convergent antibodies. (C) Flow cytometry histograms of 12 monoclonal cell populations of convergent variants (CVs) using different VH sequences from the same cluster as RSV3. (D) Table shows the CDRH3 of selected CVs and the RSV-F immunized mouse repertoires in which they were found. Red letters indicate differences from the originally found sequence RSV3 sequence. (E) Scatter plot shows frequency rank distribution of CVs from the RSV3 cluster by mouse repertoire. Red dots highlight VHs confirmed to bind in (C). (F) Pie chart shows the nine most utilized V gene germ lines in convergent clones for both RSV-F and OVA.

[0042] In particular, the system and architecture shown in Figures 1-5 can encode and decode CDR1, CDR2, CDR3 sequences and appropriate combinations thereof to and from the latent space. Sequences in the latent space can be clustered according to GMM, with similar sequences falling into the same cluster and closely related clusters occupying similar regions in the latent space. In some embodiments, deep neural networks are used to encode (Figure 3) and decode (Figure 4) the sequences, optimized with respect to GMM pre-processing and their ability to reconstruct the input sequences. Increasing the dimensionality of the latent encoding improved the reconstruction ability of the model, and with a 10-dimensional encoding layer, the system achieved a reconstruction accuracy of over 93% (Figure 14). As illustrated in Figure 6A, principal component analysis (PCA) can be used to visualize the latent encoding and show that related sequences indeed mapped to the same clusters and regions in the latent sequence space. The amount of convergence was quantified by identifying latent clusters that were highly predictive of the respective antigen immunization. Sequences were grouped into their respective clusters, and the recoded repertoire was used to train and cross-validate a linear one-to-one SVM classifier. To establish a baseline for this workflow, a linear support vector machine (SVM) was also trained on the occurrence of public clones (those with exact sequence matches of CDRH1-CDRH2-CDRH3 aa) and obtained an accuracy of 42% for predicting antigen exposure (5-fold cross-validation). In contrast, when using a VAE-based cluster assignment and subsequently encoding the repertoire based on cluster enrichment, the resulting classifier was able to achieve a prediction accuracy of over 80% (Figure 6B). The model was retrained on the full data, and clusters were selected based on a non-parametric permutation test, selecting significantly enriched clusters with Bonferroni-corrected p-values ​​less than 0.05. Closer inspection revealed that not all mice expressed all of the respective convergent clusters, but rather a small but predictable subset (Figure 6C).The number of convergent clusters discovered correlates well with the size and complexity of the antigen. By comparing all sequences that map to a given cluster, the system demonstrates how VaDE is able to build biologically meaningful clusters. Again, comparing the aggregated sequence logos with those generated from a single repertoire shows the potential diversity of the convergent sequence space and highlights that the convergence is not focused on a specific CDR (Figure 6A and Figure 6C).

[0043] To confirm that antigen-predicted sequence convergence is indeed driven by antigen recognition, convergence V H A small subset of sequences was cloned from a single repertoire of variable light chains (V L) library (Figures 19-22 and 9). A mammalian expression system was utilized for antibody display and secretion through coupling to CRISPR-Ca9 mediated library integration to screen for antigen specificity. ELISA performed on the supernatant of the library cell lines demonstrated the correct specificity for all four antigens (Figures 6A, 6C). Furthermore, single antigen-specific clones were identified by fluorescence activated cell sorting (FACS) (Figures 7B, 9, and 10). The heavy chain pools of the two antigens, RSV-F and OVA, were then investigated in more detail through single clone isolation by fluorescence activated cell sorting (FACS). The antigen-specific clones identified in this step and their corresponding heavy chains were again confirmed by ELISA performed on the supernatant of the new monoclonal cell population (Figures 7B, 7C, and 10). This procedure allowed us to confirm the antigen specificity of six OVAs (out of six selected) and three RSV-F convergent VH sequences (out of four selected) (Figures 19-22). The VH chains were able to pair with VL chains from different mouse repertoires, further highlighting the convergence for VH chain dominant binding (Figures 19-22). Although all antigens were associated with different V gene germlines, we noticed that the convergent antibodies utilized different V gene segments in an antigen-dependent manner, highlighting the contribution of the original V gene germline to convergent selection (Figure 7F).

[0044] To confirm that antibody sequence variants mapping to the same convergent cluster are also antigen-specific, we identified 12 convergent Vs in one of the RSV-F binders identified from cluster mapping. H Mutants (derived from other mice immunized with the same antigen) were recombinantly expressed (RSV3, FIG. 13). These 12 convergent mutants are identical to the V L We confirmed that all 12 of these convergent mutants were indeed antigen-specific by flow cytometry (Fig. 7c). HUsing the standard clonotype definition of CDRH3 aa identity (2,4), we identified only 0 or 5 of the 12 variants, respectively, that converged between the repertoires (Fig. 7d). In contrast, the VAE model was able to discover variants of RSV3 with CDRH3 aa identity as low as 64% (4 of 11 mismatches), validating the large potential diversity revealed by the historical logo plots (Fig. 6d, Fig. 7f). In addition to their sequence diversity, these clones also confirmed the large range of occurrence of confirmed binders in the respective mouse repertoires, with high, medium, and low frequencies (Fig. 7e).

[0045] Finally, we aimed to understand the extent to which the VAE model can generalize to unseen data. First, we experimentally generated an antibody CDRH3 library of RSV3 clones through CRISPR-Cas9 homology-directed mutagenesis. The diversity of the library was designed to model the decoder-generated sequences of the RSV3 cluster, while also containing fully randomized positions (Figure 15a). Screening of the hybridoma antibody library by FACS, followed by deep sequencing, yielded 19,270 surface-expressed variants, of which 7,470 were confirmed to be antigen-binding (Figure 15b). When evaluating the probability of these novel variants under the VAE model, we found that binding CDRH3 was significantly more likely to be generated than non-binding variants (Figure 16). However, the library also contained aa that were not observed in nature, so the majority of the variants were less likely to be generated by our model than naturally occurring sequences (Figures 16, 17). However, the overlap between the distributions indicates that the VAE model is capable of generating some of these variants in silico, a fact we confirmed by sampling 1 million potential encodings.

[0046] II. Discussion The solution can reveal broad convergence and provide analytical tools and workflows for generating in silico sequences. H A screening workflow can be included, which combines bioinformatics and screening technologies based on antibody expression and display systems. The convergent clusters revealed by the encoder or the in silico sequences generated by the decoder can be used to evaluate optimal properties for drug development (e.g., antibody development potential). The convergent cluster antibodies can also be used through experimental assays to identify their cognate binding epitopes (e.g., peptide / protein antigen library arrays, mass spectrometry), which can serve as targets for drug development. The convergent clusters can also be used as diagnostics to evaluate the immune status or health / disease status of an individual.

[0047] In summary, the present system shows that extensive convergence occurs in mouse antibody repertoires across a range of antigens. Current approaches used to detect convergence look for exact CDR3 sequence identity or use a threshold of 80% sequence identity, and the full scale of convergence patterns can only be partially recovered, as dissimilarities of more than 40% are found in individual convergent motifs. Other clustering algorithms that can be used to extract convergence also often require the definition of an arbitrary similarity threshold. In the present solution, these parameters are learned from the data to form clusters of various degrees of similarity. Furthermore, these systems allow the discovery of convergent motifs buried deep within the repertoire. This highlights that a similar phenomenon may be more commonly observed in humans as the amount of available sequencing data increases. Furthermore, we show for the first time how to use deep generative modeling to generate novel and functional antibodies in silico, thereby enabling a significant expansion of antibody discovery capabilities from deep sequencing.

[0048] III. Method A. Immunization Six to eight week old female BALB / c mice (Charles River Laboratories) were separated into cohorts (10-12 mice) based on antigens, i.e. chicken egg white lysozyme (HEL), ovalbumin (OVA), blue carrier protein (BCP) and respiratory syncytial virus glycoprotein (RSV). Mice were given subcutaneous injections of 200 μg antigen and 20 μg monophosphoryl lipid A (MPLA) adjuvant. Final immunizations (boost 1, 2, or 3) were given with 50 μg antigen per intraperitoneal injection without adjuvant. Intermediate immunizations (boost 1 and / or 2) were given with 50 μg antigen and 20 μg MPLA. Sequential injections were given at 3 week intervals. All adjuvants and antigens were prepared, aliquoted before the experiment, and mixed on the corresponding day of injection. Mice were sacrificed 10 days after the final immunization, and bone marrow was extracted from the femur of the hind leg. The isolated bone marrow was then centrifuged at 400g for 5 minutes at 4°C. The supernatant was removed and 1.25 mL of Trizol was added. The bone marrow was then homogenized using an 18G x 2 inch needle (1.2 x 50 mm). The resulting 1 mL of Trizol solution was then frozen at -80°C until processing. The mouse cohorts and immunization groups are listed in Table 1.

[0049] B. RNA extraction from mouse bone marrow One mL of homogenate was used as input for the PureLink RNA Mini Kit (Life Technologies, 12183018A), followed by RNA extraction according to the manufacturer's guidelines.

[0050] C. Antibody Repertoire Library Preparation and Deep Sequencing Antibody variable heavy chain (VH) for deep sequencing H) libraries were prepared using a molecular amplification fingerprinting (MAF) protocol that allows comprehensive error and bias correction (Khan, TA, et al., "Accurate and predictive antibody repertoire profiling by molecular amplification fingerprinting" Sci Adv, 2016.2(3):p.e1501371). Briefly, a first step of reverse transcription was performed on total RNA using gene-specific primers corresponding to the constant heavy region 1 (CH1) of IgG subtypes, with an overhang region containing a reverse unique molecular identifier (RID). Then, multiplex PCR was performed to isolate V H A forward primer set annealing to the framework 1 (FR1) region of the cDNA was performed on the first strand cDNA with a forward molecular identifier (FID) and a partial Illumina adapter overhang region, and the reverse primer also contains a partial Illumina sequencing adapter. A final singleplex PCR step was performed to complete the addition of the full Illumina adapter. After library preparation, the overall library quality and concentration were determined on a fragment analyzer (Agilent). Libraries were then pooled and sequenced on an Illumina MiSeq using the reagent v3 kit (2 × 300 bp) with 10% PhiX DNA added for quality purposes.

[0051] D. Data Preprocessing and Sequence Alignment Prior to alignment, raw FASTQ files were processed by a custom CLC Genomics Workbench 10 script. First, low-quality nucleotides were removed using the quality trimming option with a quality limit of 0.05. Then, forward and reverse read pairs were merged and the resulting 350–600 base pairs were retained for further analysis. The preprocessed sequences were then error-corrected and aligned.

[0052] E. Variational Deep Embedding (VaDE) into Antibody Repertoires Also, with reference to Figures 1 and 3, in particular, after error and bias correction and alignment of the antibody repertoire sequencing data, all discovered combinations of CDRH1, CDRH2, and CDRH3 for each dataset were extracted. To handle CDRHs of various lengths, the sequences were padded with dashes until a certain fixed length (the maximum length of each CDRH in the data) was reached, as described above with reference to Figure 3. The padded sequences were one-hot encoded, concatenated, and used as input to a variational autoencoder (VAE), in particular with reference to Figure 3. The VAE model jointly optimizes its ability to reconstruct its input along with a Gaussian Mixture Model (GMM)-based clustering of the latent space according to the following formula: TIFF0007602484000002.tif6128, where TIFF0007602484000003.tif33128We also perform a posteriori variational approximation as follows, where we assume that q(z│x,y) is distributed according to a Gaussian distribution: q(y,z│x)=q(y│x)q(z|x,y)

[0053] The technique can increase the stability of the model by not performing a mean-field approximation during posterior modeling. The system can encode and decode every input sequence as if it belongs to every cluster (indicated by the one-hot encoded cluster label) using weights shared across all layers. The system then weights the final contribution to the overall loss by the separately predicted probability q(y│x), which describes the probability of the sequence belonging to a particular cluster (Figure 5). The decoder maps the input sequences and concatenated class labels to a lower dimensional (d=10) space using two dense layers with rectified linear unit (ReLU) activations, followed by a final 10-dimensional layer. Sequences are sampled and re-created from the latent space using the decoder. The decoding network (Figure 5) employs two separate dense layers with ReLU activations, followed by a dense layer with linear activations, the output of which can be reshaped and normalized with a softmax activation to reconstruct the probability of the initial one-hot encoded CDRH. A standard multiclass cross-entropy loss can be used as the reconstruction term. For example, all VAE models can be trained on a single GPU node of a parallel computing cluster (e.g., the ETH Zurich Parallel Computing Cluster). Training can include 200 epochs for all models using a stochastic optimization algorithm.

[0054] VaDE can jointly optimize a deep generative model along with Gaussian Mixture Model (GMM)-based clustering of the latent space, as shown in Figure 2. The encoder 108 concatenates the CDR1, CDR2, and CDR3 sequences and feeds them into a self-attention layer. The input and output of this layer form a normalized residual block. The normalized residual block is input into a position-wise fully connected feed-forward neural network layer. The output of this layer is then mapped to a lower-dimensional latent space using a linear transformation.

[0055] Also, referring to FIG. 1 and FIG. 4, in particular, the decoder 112 can recreate the sequence from the latent space. The decoder 112 can use three separate long short-term recurrent neural network (LSTM-RNN) layers, as shown in FIG. 4, whose outputs are processed using a feed-forward layer with softmax activation to reconstruct the initial one-hot encoding CDRs individually. Each VaDE model was trained, for example, on a GPU node of a parallel computing cluster. The training can include 100 or more pre-trainings and 1000 or more full trainings. To pre-train, a deep autoencoder model was used whose layers reflect the above-mentioned architecture illustrated in FIG. 3. After pre-training was completed, a GMM was learned in the latent space, and both the autoencoder layer weights and the GMM parameters were used to initialize the full model.

[0056] F. Predicting antigen exposure of a single antibody repertoire The repertoire dataset was split into five folds, with each fold roughly balanced in the number of different antigen groups, and each dataset appearing only once across all folds. This split was then used to perform a cross-validation procedure, where each of the five folds was reserved once as a test set, and the remaining four folds were used as training data. For each of the five training / test splits, a separate VAE model was learned by combining all sequences across all repertoires from the training set as input. Clustering assignments or sequences from both the training and test sets were then calculated for the trained model. Based on these cluster labels, each repertoire was recoded as an n-dimensional vector, where n is the number of possible clusters and the i-th element indicates the number of sequences that map to the i-th cluster in a given repertoire. These vectors were then used to train and validate a linear support vector machine (SVM) in a one-to-one setting. Also, to prevent resource-intensive nested cross-validation steps, we decided not to optimize the hyperparameters of the SVM, but instead to use the standard values ​​(with a linear kernel) given in the scikit-learn implementation of "SVC".For visualization purposes, we clustered the results of each cross-validation step into one single confusion matrix (Figure 6B).

[0057] G. Identification of antigen-related sequence clusters To identify antigen-related sequence clusters from the antibody repertoire and determine whether sequence reads were specifically enriched in one cluster given a particular cohort, a non-parametric permutation test was performed (Figure 6D). To account for multiple testing, a Bonferroni correction was applied to all p-values ​​for each cohort. For further validation, we proceeded by randomly selecting one CDR1-CDR2-CDR3 combination and its cognate full-length variable region from each cluster.

[0058] H. Generation of cluster-specific in silico mutants Cluster-specific novel variants were generated in silico by sampling data points in the latent space from a multivariate Gaussian distribution, where the parameters were given by the respective cluster parameters from the final VAE model. These sampled data points were then fed into a decoding network, resulting in a position probability matrix for each CDRH (see Figure 8A). For each data point, a given CDRH1, CDRH2, and CDRH3 were generated. This process was repeated millions of times. The log probability of a single sequence was approximated by taking an average of 500 samples of the lower bound of evidence (ELBO).

[0059] I. Hybridoma Cell Culture Conditions All hybridoma cell lines and libraries were cultured in high glucose Dulbecco's Modified Eagle Medium (DMEM; Thermo) supplemented with 10% (v / v) heat-inactivated fetal bovine serum (FBS; Thermo), 100 U / ml penicillin / streptomycin (Pen Strep; Thermo), 10 mM HEPES buffer (Thermo), and 50 μM 2-mercaptoethanol (Thermo). All hybridoma cultures were maintained in a cell culture incubator at a constant temperature of 37 °C in humidified air with 5% CO2. Hybridoma cells were typically cultured in 10 mL of medium in T-25 flasks (TPP, 90026) and passaged every 48 / 72 hours. All hybridoma cell lines were confirmed annually to be mycoplasma-free (Universal Mycoplasma Detection Kit, ATCC, 30-1012K). The cell line PnP-mRuby / Cas9 has been published in Mason et al., (2018).

[0060] J. Generation of antibody libraries by CRISPR-CAS9 homology-directed repair Candidate V HGenes were ordered from Twist Biosciences as gene fragments, which were resuspended in 25 μl Tris-EDTA, pH 7.4 (Sigma) before use. All oligonucleotides used in this study, as well as crRNA-JP and tracrRNA, were purchased from Integrated DNA Technologies (IDT) and adjusted to 100 μM (oligonucleotides) with Tris-EDTA or 200 μM (crRNA / tracrRNA) with nuclease-free double-stranded buffer (IDT, 11-01-03-01) before use. The homology-directed repair (HDR) donor template used throughout this study was based on the pUC57(Kan)-HEL23-HDR homology donor plasmid. To avoid library cloning artifacts and background antibody expression due to unmodified parent vector DNA, two consecutive stop codons were inserted into the V H and the variable light chain (V L ) sequence at the beginning of the coding region.

[0061] Each candidate V H For separate HDR-donor V L Libraries were assembled stepwise by Gibson cloning using Gibson Assembly Master Mix (NEB). Where necessary, fragments were amplified using KAPA Hifi HotStart Ready Mix (Kapa Biosystems) according to the manufacturer's instructions. First, the heavy chain gene was amplified from a gene fragment and cloned into a PCR-linearized parental HDR-donor vector (step 1). Then, total bone marrow RNA from mice immunized with one of the four respective antigen RTs was amplified using the V L The PCR was performed using Maxima reverse transcriptase (Thermo) with degenerate primers specific for the constant region. The resulting cDNA was used to amplify each V in a multiplex PCR reaction using degenerate multiplex primers. L The repertoire was amplified (Table 7). L Repertoire for each candidate V HThe library was cloned into the PCR-linearized HDR-donor vector generated in step 1 (step 2), and the final library was evaluated in terms of diversity and background clones. Typically, the fixed V H HDR Donor V L Library sizes ranged from 30,000 to 80,000 transformants per library.

[0062] Table 7. Primers used for library generation and sequencing TIFF0007602484000004.tif123170TIFF0007602484000005.tif251170

[0063] PnP-mRuby / Cas9 cells were electroporated with the 4D-Nucleofector System (Lonza) using the SF Cell Line 4D-Nucleofector Kit L (Lonza, V4XC-2012) with the program CQ-104. For each HDR-donor library, 10 6 Cells were harvested by centrifugation at 125 g for 10 min, washed with 1 mL Opti-MEM reduced serum medium (Thermo, 31985-062) and centrifuged again using the same parameters. Finally, cells were resuspended in 100 μl nucleofection mixture containing 500 pmol crRNA-J / tracrRNA complex and 20 μg HDR donor plasmid (5.9 kb) diluted in SF buffer. After electroporation, cells were cultured in 1 mL growth medium in 24-well plates (Thermo) for 2 days and transferred to 6-well plates (Costar) containing 2 mL growth medium for an additional day.

[0064] K. Screening of Hybridoma Antibody Libraries by Flow Cytometry Flow cytometry-based analysis and cell isolation of CRISPR-Cas9 modified hybridomas were performed on BD LSR Fortessa and BD FACS Aria III (BD Biosciences). Flow cytometry data were analyzed using FlowJo V10 (FlowJo LLC). Three days after transfection, hybridoma cell libraries specific for one antigen were pooled and enriched for antibody-expressing and antigen-specific cells in successive rounds of fluorescence-activated cell sorting (FACS). Typically, the number of sorted cells from the previous enrichment step was sampled in 40-fold excess with respect to the number of labeled cells for the subsequent sorting step. For labeling, cells were washed with PBS (Thermo, 10010023), incubated with labeled antibody or antigen for 30 min on ice protected from light, washed twice again with PBS, and analyzed or sorted. Labeling reagents and working concentrations are listed in Table 8. For cell numbers different from 106, the amount of antibody / antigen as well as the incubation volume were adjusted proportionally. Labeling of RSVF-specific cells required a two-step labeling procedure due to the indirect labeling of cells with RSVF-biotin / streptavidin-AlexaFluor647.

[0065] Table 8. FACS labeling agents TIFF0007602484000006.tif97160

[0066] L. Genotyping of single-cell hybridoma clones Genomic DNA of single cell hybridoma clones was isolated from 5x105 cells, washed with PBS, and resuspended in QuickExtract DNA extraction solution (Epicentre, QE09050). Cells were incubated at 68°C for 15 min and 95°C for 8 min, and the combined synthetic VL-Ck-2A-VH antibody region was amplified by PCR using flanking primers specific for the 5' and 3' homology arms, CATGTGCCTTTTCAGTGCTTTCTC (SEQ ID NO:33) and CTAGATGCCTTTCTCCCTTGACTC SEQ ID NO:34From this single amplifier, both the VH and VL regions were amplified with the primers TGACCTTCTCAAGTTGGC SEQ ID NO:36 and GAAACAACATATGACTCCTGTCTTC (SEQ ID NO:35) (Microsynth) can be used for Sanger sequencing.

[0067] M. Measurement of antibody specificity by ELISA A standard enzyme-linked immunosorbent assay (ELISA) was performed to measure the specificity of single hybridoma cell line supernatants containing secreted IgG. High-binding 96-well plates (Costar, CLS3590) were coated overnight with 4ug / ml of antigen in PBS at 4°C. Plates were then blocked with PBS containing 2% (m / v) non-fat dry milk powder (Aprichem, A0830) and 0.05% (v / v) Tween-20 (Aprichem, A1389) for 2 hours at room temperature. After blocking, plates were washed three times with PBS containing 0.05% (v / v) Tween-20 (PBST). Cell culture supernatants were 0.2μm sterile filtered (Sartorius, 16534) and serially diluted across the plate (1:3 steps) in PBS supplemented with 2% (m / v) milk (PBSM), starting with undiluted supernatant as the highest concentration. Plates were incubated for 1 h at room temperature and washed three times with PBST. HRP-conjugated rat monoclonal [187.1] anti-mouse kappa light chain antibody (Abcam, ab99617) was used as secondary detection antibody concentrated at 0.7 μg / ml (1:1500 dilution from stock) in PBSM. Plates were again incubated for 1 h at room temperature followed by three washing steps with PBST. ELISA detection was performed using 1-step Ultra TMB-ELISA substrate solution (Thermo, 34028) and the reaction was terminated with 1M H2SO4. Absorbance at 450 nm was measured with an Infinite 200PRO NanoQuant (Tecan) and data were analyzed using Prism V8 (Graphpad).

[0068] Although operations are shown in the figures in a particular order, such operations do not have to be performed in the particular order or sequence shown, and not all of the illustrated operations need be performed. Actions described herein may be performed in different orders.

[0069] The separation of various system components need not be required in all implementations, and the described program components may be included in a single hardware or software product.

[0070] After describing some exemplary implementations herein, it is clear that the above is illustrative and not limiting, and is presented by way of example. In particular, many of the examples presented herein include specific combinations of method acts or system elements, but those acts and those elements may be combined in other ways to achieve the same purpose. Acts, elements, and features discussed in relation to one implementation are not intended to be excluded from similar roles in other implementations or implementations.

[0071] The terms and terminology used herein are for the purpose of description and should not be considered limiting. The use of "including," "comprising," "having," "containing," "involving," "characterized by," "its," and variations thereof herein are meant to encompass the subsequently recited items, equivalents thereof, and additional items, as well as alternative implementations consisting of only the subsequently recited items. In one implementation, the systems and methods described herein consist of one, each combination of more than one, or all of the recited elements, acts, or components.

[0072] As used herein, the terms "about" and "substantially" will be understood by those of ordinary skill in the art and will vary to some extent depending on the context in which it is used. If there are uses of the term that are not clear to persons of ordinary skill in the art given the context in which it is used, "about" will mean up to plus or minus 10% of the particular term.

[0073] Any reference herein to system and method implementations or elements or acts in the singular may encompass implementations that include a plurality of those elements, and any reference herein to any implementations or elements or acts in the plural may encompass implementations that include only a single element. References in the singular or plural are not intended to limit the systems or methods disclosed herein, their components, acts, or elements to the singular or plural configuration. References to any act or element that is based on any information, act, or element may include implementations in which the act or element is based at least in part on the any information, act, or element.

[0074] Any implementation disclosed herein may be combined with any other implementation or embodiment, and references to "implementations," "some implementations," "an implementation," etc. are not necessarily mutually exclusive and are intended to indicate that a particular feature, structure, or characteristic described in connection with an implementation may be included in at least one implementation or embodiment. As used herein, such terms do not necessarily all refer to the same implementation. Any implementation may be combined with any other implementation, inclusively or exclusively, in any manner consistent with the aspects and implementations disclosed herein.

[0075] As used in this specification and the claims, unless clearly indicated to the contrary, the indefinite articles "a" and "an" should be understood to mean "at least one."

[0076] References to "or" may be construed as inclusive, such that any term listed using "or" can indicate either one, more than one, and all of the listed terms. References to at least one of a conjunction list of terms may be construed as an inclusive "OR" indicating either one, more than one, and all of the described terms. For example, a reference to "at least one of 'A' and 'B'" can include 'A' alone, 'B' alone, and both 'A' and 'B'. Such references used in conjunction with "comprising" or other open terminology can include additional items.

[0077] Where a reference sign follows a drawing, the detailed description, or a technical feature of any claim, the reference sign is included to enhance the clarity of the drawing, the detailed description, and the claims, and thus the presence or absence of the reference sign does not have any limiting effect on the scope of any claim element.

[0078] The systems and methods described herein may be embodied in other specific forms without departing from their characteristics. The above-described implementations are illustrative rather than limiting of the described systems and methods. Thus, the scope of the systems and methods described herein is indicated by the appended claims, rather than the above specification, and changes that come within the meaning and range of equivalence of the claims are intended to be embraced therein.

Claims

1. providing a plurality of input amino acid sequences representing antigen-binding portions of a plurality of antigen-binding molecules to a candidate identification system; transforming the plurality of input amino acid sequences into a latent space by an encoder executed by the candidate identification system; determining a plurality of clusters of sequences in the latent space by a clustering engine implemented by the candidate identification system; identifying, by the clustering engine, convergent clusters; selecting, by a candidate generation engine executed by the candidate identification system, samples in the latent space defined by the converged clusters; generating candidate amino acid sequences by the candidate generation engine using a decoder based on the samples in the latent space.

2. The method of claim 1 , wherein the antigen-binding molecule is an antibody or an antigen-binding fragment thereof.

3. The method of claim 1 , wherein the antigen binding molecule is a chimeric antigen receptor.

4. The method of any one of claims 1 to 3, wherein the plurality of input amino acid sequences comprises one or more complementarity determining regions (CDRs).

5. The method of any one of claims 1 to 4, wherein the plurality of input amino acid sequences comprises one or more of the following amino acid sequences: (a) heavy chain complementarity determining region 3 (CDRH3) sequence, (b) heavy chain complementarity determining region 1 (CDRH1) sequence; (c) heavy chain complementarity determining region 2 (CDRH2) sequence; (d) light chain complementarity determining region 1 (CDRL1) sequence, (e) a light chain complementarity determining region 2 (CDRL2) sequence, and / or (f) Light chain complementarity determining region 3 (CDRL3) sequence.

6. the plurality of input amino acid sequences (a) a full-length heavy chain, or an antigen-binding portion thereof, and / or (b) a full-length light chain, or an antigen-binding portion thereof The method of claim 4 or 5, comprising:

7. (a) the decoder includes a plurality of long term short term recurrent neural networks; and (b) the step of generating candidate sequences further comprises providing the samples to each of the plurality of long-term short-term recurrent neural networks; The method according to any one of claims 1 to 6.

8. 8. The method of claim 1, wherein the step of transforming the plurality of input amino acid sequences into the latent space further comprises transforming the plurality of input amino acid sequences into the latent space using variational deep embedding (VaDE).

9. The method of any one of claims 1 to 8, wherein determining the plurality of sequence clusters further comprises determining the plurality of sequence clusters using mixture modeling.

10. The method of claim 9 , wherein the mixture modeling comprises Gaussian mixture modeling (GMM).

11. a memory for storing processor-executable instructions; One or more processors Including, The one or more processors: receiving, by an encoder executed by the one or more processors, a plurality of input amino acid sequences representing antigen-binding portions of antibodies; Transforming the plurality of input amino acid sequences into a latent space by the encoder; determining, by a clustering engine executed by the one or more processors, a plurality of clusters of sequences in the latent space; identifying convergent clusters by said clustering engine; selecting, by a candidate generation engine executed by the one or more processors, samples in the latent space defined by the convergent clusters; generating, by the candidate generation engine, candidate sequences based on the samples in the latent space.

12. (a) the candidate generation engine includes a decoder having a plurality of long term short term recurrent neural networks; (b) the encoder transforms the plurality of input amino acid sequences into the latent space using variational deep embedding (VaDE); and / or (c) the clustering engine uses Gaussian Mixture Modeling (GMM) to determine the plurality of sequence clusters. The system of claim 11.

Images