Peptide Mutation Policy for Targeted Immunotherapy

Abstract

A method and system for training a machine learning model includes embedding a state including a peptide sequence and a protein as a vector 304. An effect including a modification to an amino acid of the peptide sequence is predicted using a presentation score of the peptide sequence with the protein as a reward 306. A mutation policy model is trained using the state and the reward to generate modifications that increase the presentation score 308.

Description

[Technical field]

[0001] Related Application Information This application claims priority to U.S. Non-provisional Patent Application No. 17 / 711,658, filed April 1, 2022, and U.S. Provisional Patent Application No. 63 / 170,727, filed April 5, 2021, both of which are incorporated by reference in their entireties. [Background technology]

[0002] Technical Field The present invention relates to immunotherapy, and more particularly to modifying peptide sequences and predicting the binding affinity of modified peptide sequences.

[0003] 2. Description of Related Art Peptide-MHC (major histocompatibility complex) protein interactions are involved in cell-mediated immunity, regulation of immune responses, and transplant rejection. Although computational tools exist to predict the binding interaction score between an MHC protein and a given peptide, tools to generate new binding peptides with new specific properties from existing binding peptides are lacking. Summary of the Invention

[0004] A method for training the machine learning model includes embedding states, including peptide sequences and proteins, as vectors. Using the presentation score of the peptide sequence with the protein as a reward, an action, including a modification to an amino acid in the peptide sequence, is predicted. A mutation policy model is trained to generate modifications that increase the presentation score using the states and reward.

[0005] A method for developing a treatment includes training a peptide mutation policy model that generates modifications to input peptides based on a presentation score. Known peptides are sampled from a peptide library that targets a viral pathogen or a tumor. The known peptides are mutated using the peptide mutation policy to generate new peptides that have a presentation score by the MHC protein above a threshold. A treatment for a pathogen associated with the MHC protein using the new peptides is developed.

[0006] A system for training a machine learning model includes a hardware processor and a memory. The memory stores a computer program that, when executed by the hardware processor, causes the hardware processor to execute the steps of: embedding a state including a peptide sequence and a protein as a vector; predicting an action including a modification to an amino acid in the peptide sequence using a presentation score of the peptide sequence with the protein as a reward; and training a mutation policy model using the state and the reward to generate a modification that increases the presentation score.

[0007] These and other features and advantages will become apparent from the following detailed description of illustrative embodiments thereof, which is to be read in connection with the accompanying drawings. [Brief description of the drawings]

[0008] The present disclosure provides details in the following description of preferred embodiments with reference to the following figures.

[0009] [Figure 1] FIG. 1 is a diagram of peptide binding to major histocompatibility complex (MHC) according to one embodiment of the present invention.

[0010] [Diagram 2] FIG. 1 is a block / flow diagram of a method for developing patient treatment using mutant peptides based on pathogen or tumor corresponding peptides according to one embodiment of the present invention.

[0011] [Diagram 3] FIG. 1 is a block / flow diagram of a method for training a peptide mutation policy model using reinforcement learning, according to one embodiment of the present invention.

[0012] [Figure 4] FIG. 1 is a block diagram of a computing device capable of training a peptide mutation policy model using reinforcement learning and using such a model to mutate a peptide, according to one embodiment of the present invention.

[0013] [Diagram 5] FIG. 1 is a diagram of a neural network architecture that can be used as part of a machine learning model, according to an embodiment of the present invention.

[0014] [Figure 6] FIG. 1 is a diagram of a deep neural network architecture that can be used as part of a machine learning model, according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015] Interactions of peptides with major histocompatibility complexes (MHC) play a role in cell-mediated immunity, control of immune responses, and transplant rejection. Predicting peptide-protein binding can guide the discovery and design of peptides that may be used in vaccines and other medicines. Given a library of known peptides, a mutation policy can be used to generate new peptide sequences. The resulting mutant peptides may differ within a threshold number of amino acids from the peptide library. If the library of peptides is derived from a specific pathogen, such as a virus or tumor sample, the mutant peptides can be used to target a specific pathogen or tumor. This can make it possible, for example, to identify a specific cancer in an individual and target that cancer.

[0016] Thus, given a particular genome (e.g., sequenced from a virus or tumor cell), peptide sequences can be extracted to create a library of peptides that uniquely identify the pathogen. This library can then be targeted to generate peptides that bind to MHC present on the cell surface, thus eliciting an immune response to kill the pathogen or tumor cell.

[0017] To this end, a deep neural network can be trained with a training data set to predict from the MHC allele sequence and the peptide sequence a peptide presentation score, which can be, for example, a combination of the peptide-MHC binding affinity and the antigen processing score.

[0018] Based on the trained peptide presentation model, deep reinforcement learning may be used to generate peptides with high presentation scores. A mutation policy can bring the generated peptides closer to the provided target peptide library. The pre-trained presentation score prediction model may be used to define a reward function starting from random peptides. The deep reinforcement learning system may be trained to learn a good peptide mutation policy by transforming the given random peptides into peptides with high presentation scores.

[0019] A batch of peptides can be randomly sampled from a target library and the peptides in the target library can be mutated using a mutation policy network. The mutation process can be stopped when the mutated peptide reaches a threshold number of amino acid differences from the starting peptide, and the mutated peptide can be output. The policy network can be fine-tuned for the target library using similarity constraints.

[0020] Each peptide in the target library may generate multiple mutant peptides that satisfy similarity constraints. The output mutant peptides are ranked, and the top-ranked mutant peptides may be used as drug candidates to target pathogens or tumors for immunotherapy.

[0021] When applying a reinforcement learning system to this process, the "state" may be interpreted as a given MHC allele sequence and peptide sequence, and the "action" may be interpreted as an edit to the peptide sequence, such an edit may replace a current amino acid at a determined position in the peptide sequence with a new amino acid.

[0022] The amino acid sequence may be embedded using convolutional and fully connected layers of a neural network model to generate allelic representations. A bidirectional long short-term memory (LSTM) layer may further process the amino acid embeddings to obtain peptide representations. A deep policy network may then learn the conditional probabilities of various actions given the state. At each time step, if the peptide presentation score of the mutant peptide based on the action increases above a threshold, a positive reward value may be assigned, otherwise a negative reward value may be assigned.

[0023] Referring now to Figure 1, a diagram of peptide-MHC protein binding is shown. A peptide 102 is shown bound to an MHC protein 104, with the complementary two-dimensional interface of the diagram suggesting the complementary shape of their three-dimensional structures. The MHC protein 104 may be attached to a cell surface 106.

[0024] MHC is a region on a DNA strand that codes for cell surface proteins used by the immune system. MHC molecules are used by the immune system to help white blood cells interact with other cells. For example, MHC proteins influence the compatibility of organs for transplantation and are also important in the production of vaccines.

[0025] On the other hand, peptides are parts of proteins. When a pathogen presents a peptide that is recognized by an MHC protein, the immune system responds to destroy the pathogen. Thus, by finding peptide structures that bind to MHC proteins, an immune response can be deliberately triggered without introducing the pathogen itself into the body. In particular, given existing peptides that bind well to MHC proteins 104, novel peptides 102 may be automatically identified according to desired properties or attributes.

[0026] Referring now to Figure 2, a method for treating a disease is shown. Block 202 trains a peptide scoring model that accepts as input a peptide p and an MHC protein m and generates an output score r(p,m) that represents the binding affinity between the peptide p and the protein m, in particular the probability that the peptide p will be presented on the cell surface by the protein m. In some cases, the scoring model is a ready-made model and may be pre-trained. In some cases, the presentation score may be a composite score of an antigen processing predictor and a binding affinity predictor, the former predicting the probability that the peptide will be delivered by a transporter associated with the antigen processing protein complex to the endoplasmic reticulum where it can bind to the MHC protein.

[0027] Block 204 trains a mutation policy network that guides how the peptide sequence is modified. As described in more detail below, the policy network guides a reinforcement learning system that takes the peptide and the MHC protein as input and outputs peptide modifications or "mutations." The policy network selects mutations with the goal of improving the presentation score of the mutant peptide against the MHC protein.

[0028] Block 206 samples a library of peptides associated with the pathogen of interest. In some cases, this sampling may be done randomly. In some cases, all peptides in the library may be evaluated. Block 208 then mutates the sampled peptides according to a mutation policy network to generate new mutant peptides that differ from the sampled peptides, e.g., by at most a predetermined number of amino acids. Block 210 ranks these mutant peptides according to their presentation scores, with better binding corresponding to a higher rank.

[0029] After identifying a mutant peptide that binds well to a pathogen's MHC protein, block 212 generates a therapeutic based on the peptide. In block 214, the developed therapeutic is used to treat a patient, for example, by administering a drug containing the identified peptide that binds to the pathogen's MHC protein and prompts the patient's immune system to target the pathogen.

[0030] In this framework, a peptide consists of amino acids p = <o1,o2, ... , o l >, where o is one of the set of natural amino acids and l is the length of the sequence, e.g., between 8 and 15. A reinforcement learning agent explores peptide mutation environments for generating high presentation peptides. Thus, given a pair of inputs (p,m), the reinforcement learning agent explores and exploits peptide mutation environments by iteratively mutating peptides and observing the resulting presentation scores. This allows the agent to learn a mutation policy π(·) to iteratively mutate amino acids of a given peptide to generate high presentation scores. Thus, a peptide mutation environment and a mutation policy network are determined.

[0031] In the peptide mutation environment, a reinforcement learning agent performs peptide mutations by trial and error and gradually refines the mutation policy by adjusting the parameters of the mutation policy network. During learning, the reinforcement learning agent continues to mutate peptides and determines their presentation scores as reward signals. The rewards help reinforce the agent's mutation behavior, and mutation behaviors that produce high presentation scores are encouraged.

[0032] The mutation environment includes a state space, an action space, and a reward function. The state includes the current mutant peptide and MHC protein. The actions and rewards respectively represent the possible mutation actions of the reinforcement learning agent, which result in a new presentation score for the mutant peptide.

[0033] The state of the environment is s for a pair (p,m) at time t. t The MHC protein can be represented as a pseudosequence of, for example, 34 amino acids, each of which may contact the bound peptide at a distance of, for example, 4.0 Å. For a peptide and MHC protein of length l, the state s t is a tuple s t =(E p ,E m ), where E p and E m are the coding matrices for peptides and MHC proteins, respectively. The state s0 may be initialized by sampling peptide sequences from the library and using MHC class I proteins. During training, appropriate peptide sequences and MHC proteins can be used. The final state s T The terminal state s can be defined as the state with the maximum time step T or the state with a presentation score greater than a given threshold σ. T Once this is reached, mutation of the peptide can be stopped.

[0034] To optimize a peptide by replacing one amino acid with another, we can define a multi-discrete workspace. At a time t, we define a peptide p tGiven, the action of the reinforcement learning agent is to find the amino acid to be replaced, o i The goal of the study is to determine the position of p and predict the type of the new amino acid at that position. The reward function guides the optimization of the reinforcement learning agent, and only the terminal states can receive rewards from the peptide mutation environment. The final reward is the peptide p T is the final state s T When r(p T ,m).

[0035] To learn the mutation policy at block 204, the reinforcement learning agent learns to mutate the amino acids of the input peptide sequence, one amino acid at each step, with the goal of maximizing the presentation score of the mutated peptide. Both peptides and MHC proteins may be encoded into a distributed embedding space, and the mapping between the embedding space and the mutation policy may be learned by gradient descent optimization.

[0036] Several coding methods may be used to represent amino acids in peptide sequences and MHC proteins. Each amino acid is represented by a coding vector from a block substitution matrix (BLOSUM).

number

number

number

number

number

number

number

[0037] Each amino acid o in the peptide sequence p i For example, a one-layer bidirectional LSTM

number

number

number

number

number

number

number

number

number

number

number

number

number

[0038] To embed MHC proteins into a continuous latent vector, we use the encoding matrix E m is a vector

number

number

number

number

[0039] At each time step t, the peptide sequence p t is the latent embedding

number

number

number

number

number

number

number

number

number

number

[0040] The objective function for learning a mutation policy can be defined as follows:

number

number

number

number

number

number

number

[0041] Value function V(s t ) uses a multilayer perceptron to embed MHCs.

number

number

number

number

number

number

[0042] To stabilize training and improve performance, we use the existing data to find expert policies. ept For each MHC protein m with sufficient bound peptide data, we can derive the amino acid distribution of peptides of length l. <p1(o|m),p2(o|m),...,p l (o|m)> may be determined. Given a peptide p, the position of I may be chosen as follows:

number

number

number

[0043] After determining the position, distribute the amino acids

number

[0044] The expert policy can be used to pre-train the policy network. The objective function for pre-training can be minimizing the cross-entropy loss:

number

number

number

[0045] To increase the diversity of the generated peptides, a non-deterministic policy can be used to generate a variety of actions, which increases the exploration of a large state space and thereby finds a variety of good actions.

[0046] To facilitate exploration, entropy regularization can be included in the objective function. To explicitly implement learning of diverse actions in the policy, a diversity-enhancing experience buffer can be used to store trajectories that may result in eligible peptides. At each iteration, visited state-action pairs of eligible peptide mutation trajectories can be added to the buffer. State-action pairs with low action frequency can be kept, and frequent action pairs can be removed so that the buffer is not dominated by frequent actions. State-action pairs with low action frequency can be sampled collectively from the buffer.

[0047] Then, to encourage the policy network to reproduce infrequent actions that are likely to induce high rewards, we use a cross-entropy loss L defined on batches of state-action pairs with infrequent actions. B can be included in the final objective function.

number

[0048] 3, additional details regarding training the policy model in block 204 are shown. Block 302 encodes amino acids, e.g., using a mixture of different encodings. Block 304 embeds peptide sequences (e.g., from a library) using a bidirectional LSTM and embeds MHC proteins into continuous latent vectors using a flattened encoding matrix.

[0049] Block 306 uses the current mutation policy to predict the reward of a mutation action. The action can be selected as described above, and the reward can be calculated based on the connection strength indicated by the pre-trained presentation score model. Block 308 can train the mutation policy based on the reward, such that the mutation policy indicates the mutation action that tends to result in the highest reward.

[0050] 4, an exemplary computing device 400 is shown, in accordance with an embodiment of the present invention. The computing device 400 is configured to perform classifier enhancement.

[0051] Computing device 400 may be embodied as any type of computing or computer device capable of performing the functions described herein, such as, but not limited to, a computer, a server, a rack-based server, a blade server, a workstation, a desktop computer, a laptop computer, a notebook computer, a tablet computer, a mobile computing device, a wearable computing device, a network appliance, a web appliance, a distributed computing system, a processor-based system, and / or a user electronic device. Additionally or alternatively, computing device 400 may be embodied as one or more compute sleds, memory sleds, or other racks, sleds, computing chassis, or other components of a physically separated computing device.

[0052] As shown in FIG. 4, computing device 400 illustratively includes a processor 410, an input / output subsystem 420, a memory 430, a data storage device 440, and a communication subsystem 450, and / or other components and devices typically found in a server or similar computing device. Of course, computing device 400 may, in other embodiments, include other or additional components such as typically found in a server computer (e.g., various input / output devices). Furthermore, in some embodiments, one or more of the illustrative components may be incorporated in or otherwise form part of another component. For example, memory 430, or portions thereof, may be incorporated in processor 410 in some embodiments.

[0053] The processor 410 may be embodied as any type of processor capable of performing the functions described herein, and may be embodied as a single processor, multiple processors, a central processing unit (CPU), a graphics processing unit (GPU), a single or multi-core processor, a digital signal processor, a microcontroller, or other processor or processing / control circuitry.

[0054] Memory 430 may be embodied as any type of volatile or non-volatile memory or data storage capable of performing the functions described herein. In operation, memory 430 may store various data and software used during operation of computing device 400, such as an operating system, applications, programs, libraries, and drivers. Memory 430 is communicatively coupled to processor 410 via I / O subsystem 420 and may be embodied as circuits and / or components to facilitate input / output operations with processor 410, memory 430, and other components of computing device 400. For example, I / O subsystem 420 may be embodied as or may otherwise include a memory controller hub, an input / output control hub, a platform controller hub, integrated control circuits, firmware devices, communication links (e.g., point-to-point links, bus links, wires, cables, light guides, printed circuit board traces, etc.), and / or other components and subsystems to facilitate input / output operations. In some embodiments, the I / O subsystem 420 may form part of a system on a chip (SOC) and be integrated with the processor 410, memory 430, and other components of the computing device 400 on a single integrated circuit chip.

[0055] The data storage device 440 can be embodied as any type of device or device configured for short-term or long-term storage of data, such as, for example, a memory device and circuit, a memory card, a hard disk drive, a solid-state drive, or other data storage device. The data storage device 440 can store program code 440A for training a mutation policy model and program code 440B for mutating a peptide sequence according to the mutation policy model. The communication subsystem 450 of the computing device 400 can be embodied as any network interface controller or other communication circuit, device, or collection thereof that can enable communication between the computing device 400 and other remote devices over a network. The communication subsystem 450 can be configured to achieve such communication using any one or more communication technologies (e.g., wired or wireless communication) and associated protocols (e.g., Ethernet, InfiniBand, Bluetooth, Wi-Fi, WiMAX, etc.).

[0056] As shown, computing device 400 may also include one or more peripheral devices 460. Peripheral devices 460 may include any number of additional input / output devices, interface devices, and / or other peripheral devices. For example, in some embodiments, peripheral devices 460 may include a display, a touch screen, graphics circuitry, a keyboard, a mouse, a speaker system, a microphone, a network interface, and / or other input / output devices, interface devices, and / or peripheral devices.

[0057] Of course, computing device 400 may include other elements (not shown) or omit certain elements, as would be readily appreciated by one of ordinary skill in the art. For example, various other sensors, input devices, and / or output devices may be included in computing device 400 depending on the particular implementation of the same, as would be readily appreciated by one of ordinary skill in the art. For example, various types of wireless and / or wired input and / or output devices may be used. Additionally, additional processors, controllers, memories, and the like may be utilized in a variety of configurations. These and other variations of processing system 400 will be readily contemplated by one of ordinary skill in the art in view of the teachings of the present invention provided herein.

[0058] 5 and 6, exemplary neural network architectures are shown that can be used to implement portions of the present model. A neural network is a generalized system whose capabilities and accuracy improve with exposure to additional empirical data. A neural network learns by exposure to empirical data. During training, the neural network stores and adjusts a number of weights that are applied to the input empirical data. By applying the adjusted weights to the data, the neural network can identify that the data belongs to a particular predefined class from a set of classes and output the probability that the input data belongs to each class.

[0059] Empirical data (also called training data) from a set of examples is formatted as a string of values ​​and fed to the input of the neural network. Each example is associated with a known outcome or output. Each column is represented as an (x,y) pair, where x represents the input data and y represents the known output. The input data can be of various data types and contain multiple distinct values. The network can have one input node for each value that makes up the input data of the examples, and separate weights can be applied to each input value. The input data can be formatted, for example, as a vector, array, or string, depending on the architecture of the neural network being built and trained.

[0060] Neural networks "learn" by comparing neural network outputs generated from input data with known values ​​of examples and adjusting stored weights to minimize the difference between the output values ​​and the known values. Adjustments can be made to the stored weights through backpropagation, where the influence of the weights on the output values ​​is determined by calculating the mathematical gradient and adjusting the weights in a way that shifts the output to the minimum difference. This optimization, called gradient descent, is a non-limiting example of how training is done. A subset of examples with known values ​​that were not used in training can be used to test and validate the accuracy of the neural network.

[0061] In operation, the trained neural network can be used on new data that was not previously used for training or validation, through generalization. The adjusted neural network weights can be applied to the new data, and the weights estimate the function developed from the training examples. The parameters of the estimated function, captured by the weights, are based on statistical inference.

[0062] In a layered neural network, the nodes are arranged in layers. An exemplary simple neural network has an input layer 520 of source nodes 522 and a single computational layer 530 with one or more computational nodes 532 that also function as output nodes, with a single computational node 532 for each possible category into which an input example may be classified. The input layer 520 may have a number of source nodes 522 equal to the number of data values ​​512 in the input data 510. The data values ​​512 in the input data 510 may be represented as column vectors. Each computational node 532 in the computational layer 530 generates a linear combination of weighted values ​​from the input data 510 fed to the input node 520 and applies a differentiable nonlinear activation function to the sum. An exemplary simple neural network may perform classification on linearly separable examples (e.g., patterns).

[0063] A deep neural network, such as a multi-layer perceptron, may have an input layer 520 of source nodes 522, one or more computational layers 530 with one or more computational nodes 532, and an output layer 540 with one output node 542 for each category into which an input example may be classified. The input layer 520 may have a number of source nodes 522 equal to the number of data values ​​512 in the input data 510. The computational nodes 532 in the computational layer 530 are also referred to as hidden layers because they are between the source nodes 522 and the output nodes 542 and are not directly observed. Each node 532, 542 in the computational layer generates a weighted linear combination of values ​​from the values ​​output from the nodes in the previous layer and applies a nonlinear activation function that is differentiable over the range of the linear combination. The weights applied to the values ​​from each previous node may be, for example, w1, w2, ... w n-i ,w n The output layer provides the network's overall response to the input data. Deep neural networks may be fully connected, where each node in a computational layer is connected to every node in the previous layer, or the connections between layers may have other configurations. If links between nodes are missing, the network is said to be partially connected.

[0064] Training a deep neural network involves two phases: a forward phase, in which the weights of each node are fixed and the input is propagated through the network, and a backward phase, in which error values ​​are propagated back through the network and the weight values ​​are updated.

[0065] Computational nodes 532 in one or more computational (hidden) layers 530 perform nonlinear transformations on the input data 512 that generate a feature space. Classes or categories may be more easily separated in the feature space than in the original data space.

[0066] The embodiments described herein may be entirely hardware, entirely software, or contain both hardware and software elements, in a preferred embodiment, the invention is implemented in software, which includes but is not limited to firmware, resident software, microcode, etc.

[0067] The embodiments may include a computer program product accessible from a computer usable or computer readable medium providing program code for use by or in connection with a computer or any instruction execution system. A computer usable or computer readable medium may include any apparatus that stores, communicates, propagates, or transports a program for use by or in connection with an instruction execution system, apparatus, or device. The medium may be a magnetic, optical, electronic, electromagnetic, infrared, or semiconductor system (or apparatus or device), or a propagation medium. The medium may include a computer readable storage medium such as a semiconductor or solid state memory, a magnetic tape, a removable computer diskette, a random access memory (RAM), a read-only memory (ROM), a rigid magnetic disk, and an optical disk.

[0068] Each computer program may be tangibly stored on a machine-readable storage medium or device (e.g., a program memory or magnetic disk) readable by a general-purpose or special-purpose programmable computer to configure and control the operation of the computer when the storage medium or device is read by the computer to perform the procedures described herein. The system of the present invention may also be considered to be embodied in a computer-readable storage medium configured with a computer program, where the configured storage medium causes the computer to operate in a particular, predetermined manner to perform the functions described herein.

[0069] A data processing system suitable for storing and / or executing program code may include at least one processor coupled directly or indirectly to memory elements via a system bus. The memory elements may include local memory employed during actual execution of the program code, bulk storage, and cache memory that provides temporary storage of at least some program code to reduce the number of times the code is retrieved from bulk storage during execution. Input / output or I / O devices (including but not limited to keyboards, displays, pointing devices, etc.) may be coupled to the system either directly or through intervening I / O controllers.

[0070] Network adapters may also be coupled to the system to enable the data processing system to become coupled to other data processing systems or remote printers or storage devices through intervening private or public networks. Modems, cable modems, and Ethernet cards are just a few of the types of network adapters currently available.

[0071] As employed herein, the term "hardware processor subsystem" or "hardware processor" may refer to a processor, memory, software, or combination thereof that cooperate to perform one or more specific tasks. In useful embodiments, the hardware processor subsystem may include one or more data processing elements (e.g., logic circuits, processing circuits, instruction execution devices, etc.). The one or more data processing elements may be included in a central processing unit, an image processing unit, and / or a separate processor or computing element-based controller (e.g., logic gates, etc.). The hardware processor subsystem may include one or more on-board memories (e.g., caches, dedicated memory arrays, read-only memories, etc.). In some embodiments, the hardware processor subsystem may include one or more memories that may be on-board or off-board, or may be dedicated for use by the hardware processor subsystem (e.g., ROM, RAM, basic input / output system (BIOS), etc.).

[0072] In one embodiment, the hardware processor subsystem may include and execute one or more software elements, which may include an operating system and / or one or more applications and / or specific code to achieve a particular result.

[0073] In other embodiments, the hardware processor subsystem may include dedicated, dedicated circuitry to perform one or more electronic processing functions to achieve a specified result. Such circuitry may include one or more application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), and / or programmable logic arrays (PLAs).

[0074] These and other variations of hardware processor subsystems are also contemplated in accordance with embodiments of the present invention.

[0075] In the specification, references to "one embodiment" or "one embodiment" of the invention, as well as other variations, mean that the particular features, structures, characteristics, etc. described in connection with the embodiment are included in at least one embodiment of the invention. Thus, the appearances of the phrase "in one embodiment" or "in one embodiment" in various places in this specification, as well as any other variations, do not necessarily all refer to the same embodiment. However, it will be understood that features of one or more embodiments can be combined in view of the teachings of the invention provided herein.

[0076] For example, in the case of "A / B," the use of any of the following " / ," "and / or," "at least one of," such as "A and / or B," "at least one of A and B," will be understood to be intended to encompass the selection of only the first listed alternative (A), or the selection of only the second listed alternative (B), or the selection of both alternatives (A and B). As a further example, in the case of "A, B, and / or C" and "at least one of A, B, and C," such language is intended to encompass the selection of only the first listed alternative (A), or the selection of only the second listed alternative (B), or the selection of only the third listed alternative (C), or the selection of only the first and second listed alternatives (A and B), the selection of only the first and third listed alternatives (A and C), the selection of only the second and third listed alternatives (B and C), or the selection of all three alternatives (A and B and C). This can be expanded as many times as there are listed items.

[0077] The foregoing is understood in all respects to be illustrative and illustrative, but not restrictive, and the scope of the invention disclosed herein is to be determined not from the detailed description, but from the claims which are interpreted in accordance with the full breadth permitted by the patent laws. It will be understood that the embodiments shown and described herein are merely exemplary of the invention, and that those skilled in the art could make various modifications without departing from the scope and spirit of the invention. Those skilled in the art could make various other feature combinations without departing from the scope and spirit of the invention. Having thus described aspects of the invention with the detail and particularity required by the patent laws, what is desired to be claimed and protected by Letters Patent is as set forth in the appended claims.

Claims

1. 1. A computer-implemented method for training a machine learning model, comprising: Embedding 304 the state including the peptide sequence and the protein as a vector; predicting (306) an effect comprising a modification to an amino acid in said peptide sequence using a score of the peptide sequence represented by the protein as a reward; training 308 a mutation policy model using the state and the reward to generate modifications that increase the presentation score; training a scoring model to generate the presentation scores; The presentation score represents a combination of peptide-protein binding affinity and antigen processing score.

2. The method of claim 1 , wherein training the mutation policy model comprises minimizing a loss function that includes a clipping term, a reward term, and an exploration term.

3. The method of claim 1 , wherein the state embedding is performed using a bidirectional long short-term memory (LSTM) neural network.

4. The method of claim 1 , wherein the protein is a major histocompatibility complex (MHC) protein.

5. 1. A system for training a machine learning model, comprising: A hardware processor (410); A memory (440) for storing a computer program; The computer program, when executed by the hardware processor, causes the hardware processor to: embedding 304 the state including the peptide sequence and the protein as a vector; predicting (306) an effect, including a modification, on an amino acid in said peptide sequence using a score of the peptide sequence presented by the protein as a reward; training 308 a mutation policy model using the state and the reward to generate modifications that increase the presentation score; and training a scoring model to generate the presentation scores. A system in which the presentation score represents a combination of peptide-protein binding affinity and antigen processing score.

6. 6. The system of claim 5, wherein the computer program causes the hardware processor to execute a procedure to train the mutation policy model by minimizing a loss function comprising a clipping term, a reward term, and an exploration term.

7. 6. The system of claim 5, wherein the computer program causes the hardware processor to execute a procedure for embedding the states using a bidirectional long short-term memory (LSTM) neural network.

8. The system of claim 5 , wherein the protein is a major histocompatibility complex (MHC) protein.

Images