Generating a small number of class examples for training data

Abstract

A method and system for training a model includes encoding training peptide sequences with an encoder model 203. New peptide sequences are generated with a generator model 202. The encoder model, the generator model, and the discriminator model are trained 206 to cause the generator model to generate new peptides that the discriminator misinterprets as training peptide sequences that include learning projection vectors with cross-entropy losses for bound and unbound sequences, respectively.

Description

[Technical field]

[0001] This application claims priority to U.S. Provisional Patent Application No. 63 / 170,697, filed April 5, 2021, and U.S. Patent Application No. 17 / 711,617, filed April 1, 2022, the disclosures of which are incorporated herein in their entireties.

[0002] The present invention relates to training neural networks, and more particularly to generating minority class examples for augmenting training data for neural networks. [Background technology]

[0003] Peptide-MHC (Major Histocompatibility Complex) protein interactions are involved in cell-mediated immunity, regulation of immune responses, and transplant rejection. Although computational tools exist for predicting binding interaction scores between MHC proteins and specific peptides, tools for generating novel binding peptides with newly specified properties from existing binding peptides are lacking. Summary of the Invention

[0004] A method for training the model includes encoding training peptide sequences with an encoder model. New peptide sequences are generated with a generator model. The encoder model, the generator model, and the discriminator model are trained to cause the generator model to generate new peptides that the discriminator misinterprets as training peptide sequences that include learning projection vectors with cross-entropy losses for bound and unbound sequences, respectively.

[0005] A method for developing a treatment includes training a generative adversarial network (GAN) model to generate binding peptide sequences associated with major histocompatibility complex (MHC) proteins associated with a viral pathogen or tumor. Novel binding peptide sequences are generated using the trained GAN. A treatment for the viral pathogen or tumor associated with the MHC protein using the novel binding peptide sequences is developed.

[0006] A system for training a model includes a hardware processor and a memory that stores a computer program, which when executed by the hardware processor, causes the hardware processor to train the encoder model, the generator model, and the discriminator model to encode training peptide sequences using the encoder model, generate new peptide sequences using the generator model, and cause the generator model to generate new peptides that the discriminator misinterprets as training peptide sequences that include learning projection vectors with cross-entropy losses for bound and unbound sequences, respectively.

[0007] These and other features and advantages will become apparent from the following detailed description of exemplary embodiments, which is to be read in conjunction with the accompanying drawings.

[0008] In the present disclosure, preferred embodiments are described in detail with reference to the following drawings, as described below. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram illustrating binding between peptides and major histocompatibility complexes (MHC) according to one embodiment of the present principles.

[0010] [Diagram 2] FIG. 2 is a block diagram of a generative adversarial network (GAN) that can be trained to generate binding peptide sequences, according to one embodiment of the present invention.

[0011] [Diagram 3] FIG. 3 is a block / flow diagram of a method for developing and administering a treatment for a given pathogen, according to one embodiment of the present invention.

[0012] [Figure 4] FIG. 4 is a block / flow diagram of a method for training a GAN to generate peptide sequences capable of binding to a given MHC protein, according to one embodiment of the present invention.

[0013] [Diagram 5] FIG. 5 is a block diagram of an exemplary peptide sequence discriminator neural network architecture, according to one embodiment of the present invention.

[0014] [Figure 6] FIG. 6 is a block diagram of an exemplary peptide sequence classifier neural network architecture, according to one embodiment of the present invention.

[0015] [Figure 7] FIG. 7 is a block diagram of an exemplary peptide sequence generator neural network architecture, according to one embodiment of the present invention.

[0016] [Figure 8] FIG. 8 is a diagram of a patient being treated with a therapy developed by generating novel binding peptides to specific major histocompatibility complexes in accordance with one embodiment of the present invention.

[0017] [Figure 9] FIG. 9 is a block diagram of a computing device containing program code for training a model and generating novel binding peptide sequences according to one embodiment of the present invention.

[0018] [Figure 10]FIG. 10 is a diagram of an exemplary neural network architecture that can be used to implement one or more models, according to one embodiment of the present invention.

[0019] [Figure 11] FIG. 11 is a diagram of an exemplary neural network architecture that can be used to implement one or more models, according to one embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0020] Protein interactions between peptides and major histocompatibility complex (MHC) are involved in cell-mediated immunity, regulation of immune responses, and transplant rejection. Machine learning systems, including regression-based and neural network-based methods, can generate predictions of binding interaction scores between MHC proteins and a given peptide. The machine learning systems described herein generate novel peptides with strong binding interaction scores with MHC proteins based on one or more starting peptides.

[0021] Such a generative system may assume that the provided binding peptides are sufficient to train a generative model, such as a conditional generative adversarial network (GAN), but may still generate novel binding peptides even if there is an imbalance in the provided training dataset, such as the number of binding peptides being significantly smaller than the number of non-binding peptides.

[0022] The training dataset can be expanded by introducing training examples of additional minority classes. Although the specific application for generating binding peptides is described in detail herein, it should be understood that the training dataset expansion described herein can be applied to a variety of different applications where the categories being identified have little training data, such as visual product defect classification and anomaly detection.

[0023] Novel binding peptides are generated using a deep generative system trained with a dataset containing both MHC-binding and non-binding peptides. Instead of predicting binding scores for a predefined set of peptides, the conditional GAN ​​is trained on the generator with a tempering softmax unit and the MHC-binding peptides with dual-class label projections.

[0024] The conditional Wasserstein GAN is trained with a dataset containing both binding and non-binding peptide sequences for MHC. The conditional Wasserstein GAN includes a generator and a discriminator, where the generator is a deep neural network that converts sampled latent code vectors z and sampled labels y into generated peptide sequences.

[0025] Referring now to Figure 1, there is shown a diagram of peptide-MHC protein binding. A peptide 102 is shown bound to an MHC protein 104, whose complementary two-dimensional interfaces suggest the complementary shapes of their three-dimensional structures. The MHC protein 104 may be attached to a cell surface 106.

[0026] MHC is a region of a DNA strand that codes for cell surface proteins used by the immune system. MHC molecules are used by the immune system to contribute to the interactions of white blood cells with other cells. MHC proteins influence the compatibility of organs, for example, for transplants, and are also important in creating vaccines.

[0027] On the other hand, peptides are parts of proteins. When a pathogen presents a peptide that is recognized by an MHC protein, the immune system responds to destroy the pathogen. Therefore, by finding peptide structures that bind to MHC proteins, it may be possible to intentionally induce an immune response without introducing the pathogen itself into the body. In particular, if there are existing peptides that successfully bind to MHC proteins 104, novel peptides 102 can be automatically identified according to desired properties and attributes.

[0028] The principles are described with a specific focus on generating binding peptides, but can be easily extended to include predicting ongoing binding affinities of peptide sequences, predicting naturally processed peptides of peptide sequences, predicting T-cell epitopes of peptide sequences, etc. Diversification of applications includes providing various supervisory signals for optimizing cross-entropy loss terms, which are described in more detail below.

[0029] Furthermore, the principles of the present invention are not limited to generating binding peptides, but can be extended to generate other few-class examples in other applications. For example, few-class product images may be generated for product inspection or anomaly detection. For such tasks, the input training data may include images, and the generator architecture may be modified to accommodate the input format.

[0030] Now referring to FIG. 2, an exemplary GAN 200 is shown. The GAN 200 includes a generator 202 and a discriminator 204. The generator 202 generates candidates for a training dataset, and the discriminator 204 tries to distinguish between the generated candidates and true samples from the provided training dataset 201. The encoder 203 converts sequences of the training dataset into vectors in an embedding space. The encoder can convert sequences of amino acids into feature representation matrices, for example, with each column of the matrix corresponding to an amino acid, using block permutation or a pre-trained amino acid embedding scheme. The encoder 203 and the generator 202 are trained together to fool the discriminator.

[0031] The generator 202 is trained to increase the error rate of the discriminator 204, and the discriminator 204 is trained to decrease the error rate in identifying the generated candidates. The trainer 206 uses a loss function to train the generator 202 and the discriminator 204. In Wasserstein GAN, the loss function is based on the Wasserstein metric.

[0032] In the context of peptide generation, the training dataset 201 may include both binding and non-binding peptide sequences that interact with MHC. The generator 202 may be a deep neural network that converts sampled latent code vectors z from a multivariate unit variance Gaussian distribution and sampled binding class labels (e.g., 1 for "binding" and 0 for "non-binding") into a peptide feature representation matrix, where each column corresponds to an amino acid.

[0033] The discriminator 204 may be a deep neural network with convolutional layers and fully connected layers between an input representation layer and an output layer that outputs a scalar value. The parameters of the discriminator 204 may be updated to identify the generated peptide sequences from the sampled peptide sequences in the training dataset 201. The parameters of the generator 202 are updated to fool the discriminator 204.

[0034] A dual projection GAN can be used to simultaneously learn two projection vectors with two cross-entropy losses for each class (e.g., "bound" and "unbound"). This is equivalent to maximizing the mutual information between the generated data examples and their associated labels, where one loss distinguishes between actual bound / unbound peptides in the training data and actual unbound / bound peptides in the training data, and the other loss distinguishes between generated bound / unbound peptides and generated unbound / bound peptides. The generator 202 is updated to minimize these two cross-entropy losses for each class.

[0035] A non-negative scalar weight λ(x) is learned for each data point x associated with two cross-entropy losses to balance the discriminator losses. A penalty term of -0.5log(λ(x)) may be added to penalize large values ​​of λ(x). The data-label pairs are then scaled over a joint distribution P XY obtained from

number

[0036] The discriminator and generator loss terms can be written as the following objective function:

number

number

number

number

number

[0037] The logic of the projection discriminator can be derived as follows:

number

number

number

number

[0038] Therefore, the projection discriminator is based on the parameters

number

number

number

number

number

number

[0039] term v y is inherent

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

[0040] Data matching and label matching are weighted by the model. Two losses

number

number

[0041] In a scalar value embodiment,

number

number

[0042] In an amortized embodiment, amortized equal variance weights are learned for each data point.

number

[0043] Softmax can be used in the final output layer of the generator 202, and an entropy regularization method can be used to implicitly control the temperature in the tempered softmax unit. In the forward pass, a straight-through estimator can be used to output individual amino acid sequences (e.g., peptides) with "bound" or "unbound" labels. In the backward pass, the temperature can be used to facilitate continued gradient calculations. At the beginning of training, a smaller penalty factor is set for the entropy regularization to encourage a more uniform amino acid release probability distribution. Later in training, a larger penalty factor can be used for the entropy regularization to encourage an amino acid release probability distribution with more peaks.

[0044] In addition to updating the discriminator 204 and generator 202 in the weighted framework, the encoder is trained to map the input peptide sequence x into a latent embedding code space z. The collected latent codes of the input peptide sequences can be made to follow a multivariate unit variance Gaussian distribution by minimizing the maximum mean discrepancy regularization term of the kernel. Each embedding code z is fed to the generator 202 to reconstruct the original peptide sequence x, and the encoder and generator 202 are updated by minimizing the cross-entropy loss as the reconstruction error.

[0045] During training, m binding peptide sequences are randomly sampled from the training set 201. A convex combination of the latent codes for the m peptides can be computed using the randomly sampled coefficients, where

number

number

[0046]

[0031] Referring now to Figure 3, a method for developing a therapy is shown. Block 302 trains a GAN 200 to generate novel binding peptide sequences using a training dataset that includes both binding and non-binding peptides. The generator 202 can generate novel binding peptides for a given MHC protein of a pathogen from the trained GAN 200 in block 304. Upon identifying a peptide that successfully binds to the MHC protein of the pathogen, block 306 generates a therapy based on the peptide. Block 308 treats a patient with the developed therapy, for example, by administering a drug that includes the identified peptide that binds to the MHC protein of the pathogen and encourages the patient's immune system to target the pathogen.

[0047] Referring now to FIG. 4, additional details regarding the training of block 302 are shown. Block 402 generates a training dataset. The training dataset can include a set of peptide sequences, each of which is labeled as binding or non-binding to an MHC protein. Block 403 trains an encoder to convert peptide sequences x into vectors embedded in a latent space z. As described above, the encoder maps the input peptide sequences x into the space z, which includes minimizing the maximum mean discrepancy regularization term of the kernel, to make them multivariate unit variance Gaussian distributed. The training of the encoder is performed in parallel with the training of the generator 202, since the reconstruction error is used to help minimize the cross-entropy loss.

[0048] Block 404 encodes the peptide sequences of the training dataset as vectors using the trained encoder. These vectors are then used as inputs to the generator 202. Block 408 learns the dual projection vectors of the GAN 200. The objective function of the GAN is optimized using two cross-entropy losses per class and a data-specific adaptive weight that balances the discriminator loss and the cross-entropy loss. The generator 202 is updated with a tempered softmax output to minimize the cross-entropy loss. This training over blocks 403, 404 and 408 is repeated in block 410 with a convex combination of the combined sequence embeddings used to generate combined peptides. The encoder and generator 202 are updated to fool the discriminator 204 and the classifier. The iterative process stops when a maximum number of iterations is reached.

[0049] Referring now to Figure 5, an exemplary architecture of the discriminator 204 is shown. A peptide sequence is input as a series of embedded amino acids 502 and is processed by a convolutional layer 504 and one or more fully connected layers 506. The output of the final fully connected layer is a label indicating whether the input amino acids 502 represent "real" sequences present in the training dataset 201 or sequences generated by the generator 202.

[0050]

[0033] Referring now to Figure 6, an exemplary architecture of the classifier is shown. Similar to the discriminator 204, a peptide sequence is input as a series of embedded amino acids 502. The input amino acids 502 are processed by a convolutional layer 604 and one or more fully connected layers 606, which are trained to identify whether a given peptide sequence binds to an MHC protein or not. The output of the final fully connected layer is a label indicating whether the input amino acid 502 represents a binding or non-binding sequence.

[0051]

[0036] Referring now to Figure 7, there is shown an exemplary architecture of a generator 700. Block 702 samples a random noise vector z and a class y as input to the generator. The vector can be sampled from a multivariate Gaussian distribution with zero mean and unit diagonal variance, and the joint class label can be fixed.

[0052] The sampled vectors and classes are processed by one or more fully connected layers 704 that are trained to convert the input into a representation of a peptide sequence. A set of output tempering softmax units 706 processes the output of the fully connected layers 704 to generate the respective amino acids 502 that form the peptide sequence.

[0053]

[0023] Referring now to Figure 8, Figure 8 illustrates the treatment of a patient 802. A treatment system 804 administers a treatment based on the peptide sequences generated by the GAN 200. In particular, a binding peptide is generated that corresponds to a pathogen or tumor in the patient 102. This binding peptide can be used as part of a treatment provided to the patient 102, where the peptide binds to an MHC protein on the pathogen or tumor cell and helps the patient's immune system identify and eliminate the pathogen or tumor.

[0054] The administration of treatment may be overseen by a medical professional 806 who may assist in connecting the treatment system 804. The medical professional 806 may also be involved in identifying the pathogen or tumor using diagnostic tools to isolate MHC proteins which are used to identify binding peptides.

[0055]

[0043] Referring now to Figure 9, there is shown an exemplary computing device 900, in accordance with one embodiment of the present invention, configured to perform classifier expansion.

[0056] Computing device 900 may be embodied in any type of computing or computer device capable of performing the functions described herein, including, but not limited to, a computer, a server, a rack-based server, a blade server, a workstation, a desktop computer, a laptop computer, a notebook computer, a tablet computer, a mobile computing device, a wearable computing device, a network appliance, a web appliance, a distributed computing system, a processor-based system, and / or a consumer electronics device. Additionally or alternatively, computing device 900 may be embodied in one or more computational sleds, memory sleds or other racks, sleds, computational chassis, or other components of a physically distributed computing device.

[0057] 9, computing device 400 illustratively includes a processor 910, an I / O subsystem 920, a memory 930, a data storage device 940, a communication subsystem 950, and / or other components and devices typically found in a server or similar computing device. Computing device 900 may, in other embodiments, include other or additional components, such as components typically found in a server computer (e.g., various input / output devices). Furthermore, in some embodiments, one or more of the illustrative components may be incorporated in or form a part of another component. For example, memory 930, or portions thereof, may be incorporated in processor 910 in some embodiments.

[0058] The processor 910 may be embodied with any type of processor capable of performing the functions described herein, and may be embodied with a single processor, multiple processors, a central processing unit (CPU), a graphics processing unit (GPU), a single or multi-core processor, a digital signal processor, a microcontroller or other processor or processing / control circuitry.

[0059] Memory 930 may be embodied with any type of volatile or non-volatile memory or data storage device capable of performing the functions described herein. In operation, memory 930 may store various data and software used during operation of computing device 900, such as an operating system, applications, programs, libraries, and drivers. Memory 930 is communicatively coupled to processor 910 via I / O subsystem 920, which is embodied with circuits and / or components that facilitate input / output operations with processor 910, memory 930, and other components of computing device 900. For example, I / O subsystem 920 may be embodied with or include a memory controller hub, an input / output control hub, a platform controller hub, an integrated control circuit, a firmware device, communication links (e.g., point-to-point links, bus links, wires, cables, light guides, printed circuit board traces, etc.), and / or other components and subsystems that facilitate input / output operations. In some embodiments, the I / O subsystem 920 may form part of a system on a chip (SOC) and may be integrated with the processor 910, memory 930, and other components of the computing device 900 on a single integrated circuit chip.

[0060] The data storage device 940 may be embodied in any type of device or devices configured for short-term or long-term storage of data, such as, for example, memory devices and circuits, memory cards, hard disk drives, solid state drives, or other data storage devices. The data storage device 940 may store program code 940A for model training and program code 940B for generating binding peptides. The communication subsystem 950 of the computing device 900 may be embodied in any network interface controller or other communication circuitry, device, or collection thereof that allows communication between the computing device 900 and other remote devices over a network. The communication subsystem 950 may be configured to perform such communication using any one or more communication technologies (e.g., wired or wireless communication) and associated protocols (e.g., Ethernet, InfiniBand, Bluetooth, Wi-Fi, WiMAX, etc.).

[0061] As shown, computing device 900 may include one or more peripheral devices 960. Peripheral devices 960 may include any number of additional input / output devices, interface devices, and / or other peripheral devices. For example, in some embodiments, peripheral devices 960 may include a display, a touch screen, graphics circuitry, a keyboard, a mouse, a speaker system, a microphone, a network interface, and / or other input / output devices, interface devices, video capture devices, and / or peripherals.

[0062] Of course, computing device 900 may include other elements (not shown) or omit certain elements, as would readily occur to one of ordinary skill in the art. For example, computing device 900 may include various other sensors, input devices, and / or output devices, depending on the particular implementation, as would be readily understood by one of ordinary skill in the art. For example, various types of wireless and / or wired input and / or output devices may be utilized. Additionally, various configurations of additional processors, controllers, memory, and the like may be utilized. These and other variations of processing system 900 will be readily apparent to those of ordinary skill in the art in view of the teachings of the invention provided herein.

[0063] 10 and 11, which show an exemplary neural network architecture that can be used to implement portions of the present model. A neural network is a generalized system whose functionality and accuracy improves with exposure to additional empirical data. A neural network is trained by exposure to empirical data. During training, the neural network stores and adjusts a number of weights that are applied to the input empirical data. Applying the adjusted weights to the data identifies the data as belonging to a distinct predefined class from a set of classes, or outputs the probability that the input data belongs to each of the classes.

[0064] Empirical data from a set of examples, also known as training data, is formatted as a string of numbers and fed to the input of the neural network. Each example is associated with a known outcome or output. Each example can be represented as a pair (x,y), where x represents the input data and y represents the known output. The input data can include a variety of different types of data and can include multiple different values. The network can have one input node for each of the values ​​that make up the input data of the examples, and can apply weights to each input value separately. For example, the input data can be formatted as a vector, an array, or a string, depending on the architecture of the neural network being built and trained.

[0065] Neural networks "learn" by comparing the neural network's output, generated from input data, with the values ​​of known examples and adjusting stored weights to minimize the difference between the output value and the known value. Adjustments are made through backpropagation to the stored weights, and the effect of the weights on the output value can be determined by calculating a mathematical gradient and adjusting the weights in a way that shifts the output toward the smallest difference. This optimization, called gradient descent, is one non-limiting example of how training can be performed. A subset of examples with known values ​​that were not used in training can be used to test and validate the accuracy of the neural network.

[0066] In operation, the trained neural network can be used on new data not previously used in training or validation by generalization. The adjusted weights of the neural network can be applied to the new data, and the weights estimate a function evolved from the training examples. The parameters of the estimated function obtained by the weights are based on statistical inference.

[0067] In a layered neural network, multiple nodes are arranged in the form of layers. An exemplary simple neural network has an input layer 1020 of source nodes 1022 and a single computational layer 1030 with one or more computational nodes 1032 that also function as output nodes, with a single computational node 1032 for each category into which the input examples can be classified. The input layer 1020 can have a number of source nodes 1022 equal to the number of data values ​​1012 in the input data 1010. The data values ​​1012 in the input data 1010 can be represented as column vectors. Each computational node 1032 in the computational layer 1030 generates a linear combination of weighted values ​​from the input data 1010 provided to the input node 1020 and applies a differentiable nonlinear activation function to the sum. An exemplary simple neural network can perform classification on linearly separable examples (e.g., patterns).

[0068] A deep neural network, such as a multi-layer perceptron, has an input layer 1020 of source nodes 1022, one or more computational layers 1030 with one or more computational nodes 1032, and an output layer 1040 where there is a single output node 1042 for each possible category into which an input example can be classified. The input layer 1020 may have a number of source nodes 1022 equal to the number of data values ​​1012 in the input data 1010. The computational nodes 1032 of the computational layer 1030 are also referred to as hidden layers because they are between the source nodes 1022 and the output nodes 1042 and are not directly observed. Each node 1032, 1042 in the computational layer generates a weighted linear combination of values ​​from the values ​​output by the nodes in the previous layer and applies a nonlinear activation function that is differentiable over the range of the linear combination. The weights applied to the values ​​from each previous node may be, for example, w1, w2, ... w n-1 ,w nThe output layer provides the network's overall response to the input data. Deep neural networks can be fully connected, where each node in a computational layer is connected to every other node in the previous layer, or they can have other configurations of connections between layers. If links between nodes are missing, the network is considered to be partially connected.

[0069] Training a deep neural network involves two phases: a forward phase, during which the weights of each node are fixed and the inputs are propagated through the network, and a backward phase, during which error values ​​are propagated backwards through the network to update the weight values.

[0070] Computational nodes 1032 in one or more computational (hidden) layers 1030 perform non-linear transformations on the input data 1012 that generate a feature space. Classes or categories are easier to separate in the feature space than in the original data space.

[0071] The embodiments described herein may be implemented entirely in hardware, entirely in software, or contain both hardware and software elements, in a preferred embodiment, the invention is implemented in software, which includes but is not limited to firmware, resident software, microcode, etc.

[0072] An embodiment may include a computer program product accessible from a computer usable or computer readable medium that provides program code for use by or in connection with a computer or any instruction execution system. A computer usable or computer readable medium may include any device that stores, conveys, propagates, or transfers a program for use by or in connection with an instruction execution system, device, or apparatus. The medium may be a magnetic, optical, electronic, electromagnetic, infrared, or semiconductor system (or device or apparatus), or a propagation medium. The medium may include computer readable media such as semiconductor or solid state memory, magnetic tape, removable computer diskettes, random access memory (RAM), read-only memory (ROM), rigid magnetic disk, and optical disk.

[0073] Each computer program is stored in a machine-readable recording medium or device (e.g., a program memory or a magnetic disk) that can be read by a general-purpose or special-purpose programmable computer. The computer program is read from the recording medium or device by a computer that executes the procedures described herein for configuring and controlling operations of the computer. The system of the present invention also contemplates a computer-readable recording medium that includes a computer program configured to cause a computer to operate in a specific and predefined manner to perform the functions described herein.

[0074] A data processing system suitable for storing and / or executing program code may include at least one processor coupled directly or indirectly to memory elements via a system bus. The memory elements may include local memory, bulk memory devices, and cache memory for temporarily storing at least some program code used during the actual execution of the program code to reduce the number of times code is retrieved from bulk memory devices during execution of a process. Input / output or I / O devices (including but not limited to keyboards, displays, pointing devices, etc.) may be connected to the system either directly or through I / O controllers.

[0075] Network adapters may be coupled with the data processing system to enable the system to be coupled to other data processing systems or remote printers or memory devices through private or public networks. Modems, cable modems and Ethernet cards are just a few of the currently available types of network adapters.

[0076] As used herein, the term "hardware processor subsystem" or "hardware processor" may refer to a processor, memory, software, or combination thereof that cooperate to perform one or more specific tasks. In useful embodiments, the hardware processor subsystem may include one or more data processing elements (e.g., logic circuits, processing circuits, instruction execution units, etc.). The one or more data processing elements may include a central processing unit, a graphics processing unit, and / or individual processors or computing element-based controllers (e.g., logic gates, etc.). The hardware processor subsystem may include one or more on-board memories (e.g., caches, dedicated memory arrays, read-only memories, etc.). In any embodiment, the hardware processor subsystem may include one or more memories that may be on-board or off-board, or may be dedicated for use with the hardware processor subsystem (e.g., ROM, RAM, Basic Input / Output System (BIOS), etc.).

[0077] In some embodiments, the hardware processor subsystem may include and execute one or more software elements, which may include an operating system and / or one or more applications and / or specific code for achieving a particular result.

[0078] In other embodiments, the hardware processor subsystem may include dedicated circuitry to perform one or more electronic processing functions to achieve a specified result. Such circuitry may include one or more application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), and / or programmable logic arrays (PLAs).

[0079] These and other variations of the hardware processor subsystem are also contemplated by embodiments of the present invention.

[0080] Reference herein to "one embodiment" or "one embodiment," as well as other variations, of the invention means that a particular feature, configuration, characteristic, etc. described in connection with an embodiment is included in at least one embodiment of the invention. Thus, appearances of the phrases "in one embodiment" or "in an embodiment," as well as any other variations appearing in various places throughout this specification, are not necessarily all referring to the same embodiment. However, it should be understood that given the teachings of the invention provided herein, features of one or more embodiments may be combined.

[0081] For example, the use of any of "A / B," "A and / or B," and "at least one of A and B" should be understood to be intended to include the selection of only the first listed option (A), the selection of only the second listed option (B), or the selection of both options (A and B). By way of further example, in the case of "A, B, and / or C" and "at least one of A, B, and C," such language is intended to include the selection of only the first listed option (A), the selection of only the second listed option (B), the selection of only the third listed option (C), the selection of only the first and second listed options (A and B), the selection of only the first and third listed options (A and C), the selection of only the second and third listed options (B and C), or the selection of all three options (A, B, and C). The above examples may be expanded depending on the number of items listed.

[0082] The foregoing should be understood in all respects as illustrative and exemplary, and not restrictive, and the scope of the invention disclosed herein should be determined not from the detailed description, but from the claims which are to be interpreted in accordance with the broadest possible interpretation permitted by the Patent Law. It should be understood that the embodiments shown and described herein are merely illustrative of the principles of the invention, and that various modifications may be made by those skilled in the art without departing from the scope and spirit of the invention. Various other feature combinations may be implemented by those skilled in the art without departing from the scope and spirit of the invention. Although aspects of the invention have been described above with the fine detail and particularity required by the Patent Law, the scope of the claims which are sought to be protected by Letters Patent are set forth in the appended claims.

Claims

1. 1. A computer-implemented method for training a model, comprising: encoding 203 the training peptide sequences using the model of the encoder; generating 202 novel peptide sequences using the generator model; The computer-implemented method includes training (206) the encoder model, the generator model, and the discriminator model to generate novel peptides that the discriminator misinterprets as training peptide sequences that include learning projection vectors with respective cross-entropy losses for bound and unbound sequences.

2. 10. The computer-implemented method of claim 1, wherein the generator model outputs amino acid representations using multiple tempered softmax output units.

3. 2. The computer-implemented method of claim 1, wherein generating the novel peptide sequence comprises sampling a multivariate monovariate Gaussian distribution as input to the generator.

4. p corresponds to a training peptide sequence and q corresponds to a peptide sequence generated by the generator model; [0010] represents the embedding of the training peptide sequences and the generated peptide sequences; [0025] represents the embedding of the peptide sequence generated by the generator model, [0030] is the embedding function, [0045] is a trained and generated sequence, and P and Q are trained and generated distributions, respectively. The cross-entropy loss is [0050] 2. The computer-implemented method of claim 1, comprising:

5. The method of claim 1 , wherein the encoder model embeds peptide sequences from a training dataset into vectors during training.

6. The method of claim 5 , wherein training the encoder model comprises minimizing a maximum mean discrepancy regularization term of a kernel.

7. The method of claim 5 , wherein the training data set comprises binding and non-binding peptide sequences for the major histocompatibility complex.

8. 6. The method of claim 5, wherein the generator converts the combined class labels and sampled latent code vectors from the encoder into a peptide feature representation matrix, each column of the matrix corresponding to an amino acid.

9. The method of claim 1 , wherein training the encoder model, the generator model, and the discriminator model uses a loss function based on a Wasserstein metric.

10. 1. A system for training a model, comprising: A hardware processor (910); A memory (940) for storing a computer program; and when the computer program is executed by the hardware processor, the hardware processor encoding 203 the training peptide sequences using the model of the encoder; generating 202 novel peptide sequences using the generator model; The system performs training (206) of the encoder model, the generator model, and the discriminator model to cause the generator model to generate new peptides that are misinterpreted by a discriminator as training peptide sequences that include learning projection vectors with respective cross-entropy losses for bound and non-bound sequences.

11. 11. The system of claim 10, wherein the generator model outputs amino acid representations using multiple tempered softmax output units.

12. The system of claim 10 , wherein the computer program further comprises causing the hardware processor to sample a multivariate unitary gaussian distribution as an input to the generator.

13. p corresponds to a training peptide sequence and q corresponds to a peptide sequence generated by the generator model; [006] represents the embedding of the training peptide sequences and the generated peptide sequences; [0070] represents the embedding of the peptide sequence generated by the generator model, [0080] is the embedding function, [0097] is a trained and generated sequence, and P and Q are trained and generated distributions, respectively. The cross-entropy loss is [0089] The system of claim 10 , comprising:

14. The system of claim 10 , wherein the encoder model embeds peptide sequences from a training dataset into vectors during training.

15. 15. The system of claim 14, wherein the computer program further causes the hardware processor to minimize a maximum mean discrepancy regularization term of a kernel to train a model of the encoder.

16. 16. The system of claim 15, wherein the training data set comprises binding and non-binding peptide sequences for the major histocompatibility complex.

17. 11. The system of claim 10, wherein the generator converts the combined class labels and sampled latent code vectors from the encoder into a peptide feature representation matrix, each column of the matrix corresponding to an amino acid.

18. 11. The system of claim 10, wherein the computer program further causes the hardware processor to train the encoder model, the generator model, and the discriminator model using a loss function based on a Wasserstein metric.

Images