T cell receptor structure optimization method based on reinforcement learning
By optimizing the CDR region of TCRs through reinforcement learning and Monte Carlo tree search, the problems of low convergence efficiency and weak specificity in existing TCR design methods are solved, achieving efficient and accurate TCR structure generation and supporting personalized immunotherapy.
Patent Information
- Application Number
- CN202511356698.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-09-22
- Publication Date
- 2025-12-23
AI Technical Summary
Existing TCR design methods have low convergence efficiency, lack goal orientation, and generate TCRs with weak specificity, making it difficult to achieve targeted optimization at the structural level.
A reinforcement learning-based approach is adopted, combining Monte Carlo tree search and geometric space constraints. By constructing a structural fragment library of TCRs, the CDR1, CDR2, CDR3 and FR regions of TCRs are optimized. The optimization process is driven by a reward function to generate highly specific TCR structures.
It significantly improves the speed and accuracy of TCR design, generates TCR structures that are highly matched to specific antigens, has good traceability of pathways and interpretability of the design process, and supports personalized TCR therapy.
Smart Images
Figure CN121191584A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of artificial intelligence and biomedical technology, specifically relating to a method for optimizing the structure of T-cell receptors based on reinforcement learning. Background Technology
[0002] T cell receptors (TCRs) play a central role in the body's immune response, recognizing antigen peptides and major histocompatibility complexes (pMHCs) to mediate T cell activation and targeted killing. With the development of tumor immunotherapy, TCRs, as specific antigen recognition molecules, have become important targets for developing personalized treatment strategies, and TCR-T therapy has gradually become one of the key technologies for precision immune intervention. The structural specificity of TCRs determines their functional performance, with the most critical recognition fragment being the third complementarity-determining region (CDR3). This highly variable structure directly inserts into the pMHC binding interface, forming a crucial contact with the antigen peptide. In recent years, designing TCRs around antigens, especially constructing highly specific and structurally stable CDR3 structures, has become one of the core tasks of current TCR therapy research. Existing TCR design methods mainly focus on sequence-based generation or homology modeling of existing structures. Some deep learning models attempt to use antigens or pMHCs as input to generate potential CDR3 sequences through sequence-to-sequence algorithms; however, these methods ignore the geometric complementarity of the spatial conformation between TCRs and antigens, making it difficult to guarantee that the generated TCRs have true binding capacity at the structural level. While structural modeling methods such as co-source modeling and loop splicing take into account structural constraints to some extent, they generally suffer from problems such as large search space, low convergence efficiency, and lack of goal orientation. In particular, in antigen-specific scenarios, they cannot achieve targeted structural optimization, resulting in weak TCR specificity.
[0003] While structure prediction tools such as AlphaFold have made progress in static structure modeling, they were not built for design optimization tasks and lack iterative feedback mechanisms and generative capabilities, making it difficult to support the need for active structure optimization searches. Therefore, there is an urgent need for a method that can combine artificial intelligence technology to design TCR structures specifically for antigens. Summary of the Invention
[0004] The purpose of this invention is to address the problems of low convergence efficiency, lack of target guidance, and weak specificity of generated TCRs in existing methods, and to propose a reinforcement learning-based method for optimizing T cell receptor structure.
[0005] The technical solution adopted by this invention to solve the above-mentioned technical problems is: a method for optimizing the structure of T-cell receptors based on reinforcement learning, the method specifically including the following steps:
[0006] Step S1: Download each pdb file from a publicly available database, and then obtain the TCR-pMHC complex structure data based on the downloaded pdb files;
[0007] Based on the obtained TCR-pMHC complex structure data, a TCR structure fragment library was constructed, and the TCR structure fragment library was used as a splicing candidate pool. And obtain the geometric features of each structural fragment in the structural fragment library;
[0008] Step S2: Define reinforcement learning parameters, which include state space, action space, reward function, and policy network;
[0009] Step S3: Starting from the backbone anchor point of the TCR sequence to be optimized, the CDR1, CDR2, CDR3 and FR regions of the TCR sequence to be optimized are optimized using the Monte Carlo tree search method and reinforcement learning method to generate the optimized T cell receptor sequence.
[0010] Furthermore, the construction of a TCR structure fragment library based on the obtained TCR-pMHC complex structure data specifically involves:
[0011] Step S11: For any TCR-pMHC complex structure, extract the CDR1, CDR2, CDR3 and FR regions from the Alpha and Beta chains of TCR.
[0012] Step S12: Using a sliding window of length L, slide it across each extracted region. When the sliding window is at each position, the residues within the window are taken as a structural fragment, and the sequence information and position information of the structural fragment are recorded.
[0013] Step S13: After processing each obtained TCR-pMHC complex structure according to steps S11 and S12, a TCR structure fragment library is constructed.
[0014] Furthermore, obtaining the geometric features of each structural fragment in the structural fragment library specifically involves:
[0015] For any given structural fragment, the main chain dihedral angle, side chain bond angle, bond length, and other parameters of each amino acid residue within the fragment are extracted based on the fragment's sequence and position information. The three-dimensional coordinates of the atom will determine the first atom within the structural segment. amino acid residues The dihedral angle of the main chain is denoted as and , will the amino acid residues The side chain bond angle is denoted as , , and , will the amino acid residues With the amino acid residues The bond length is denoted as , Indicates the relationship between the current structural segment and the first... amino acid residues Adjacent amino acid residues, Indicates the first amino acid residues of The three-dimensional coordinates of the atom, and then according to , , , , , , and Calculate the first amino acid residues Local curvature and normal vector , the first in the structural fragment amino acid residues A vector composed of geometric descriptors is used as a graph. The Middle Features of each node :
[0016]
[0017] The bonds between every two adjacent amino acid residues within the structural fragment are respectively represented as diagrams. If one of the edges is in the middle, then the first edge is... amino acid residues With the amino acid residues Features of the edges between for:
[0018]
[0019] in, Indicates the first amino acid residues of Atom and the first amino acid residues of Euclidean distance between atoms , Indicates the first amino acid residues of The three-dimensional coordinates of an atom Describing the 2-norm, Indicates the first amino acid residues of Atom to the amino acid residues of The direction vector of the line connecting the atoms;
[0020] Representing structural segments as structural diagrams ,in, This represents the set of nodes corresponding to a structural segment. Represents the set of edges corresponding to a structural segment;
[0021] Using ProteinMPNN encoder For the structural diagram Embedding extraction is performed to obtain the geometric feature representation of the structural fragments. :
[0022]
[0023] in, Geometric feature representation of structural segments;
[0024] Similarly, separate candidate pools for splicing are obtained. Geometric feature representation of each structural segment.
[0025] Furthermore, the specific process of step S3 is as follows:
[0026] Step S31: Initialize the number of search layers in the Monte Carlo tree used to generate the CDR3 region. The number of search layers used to generate the CDR1 region is The number of search layers used to generate the CDR2 region is The number of search layers used to generate the FR region is Using a splicing candidate pool The CDR3 region in the TCR sequence to be optimized is randomly initialized to obtain the randomly initialized TCR sequence;
[0027] The optimization order for the TCR sequence to be optimized is set as follows: optimize the CDR3 region, CDR1 region, CDR2 region and FR region in sequence;
[0028] The geometric features of the pMHC corresponding to the TCR sequence to be optimized are used as environmental information for reinforcement learning. ;
[0029] Step S32: Use the randomly initialized TCR sequence as the root node. ;
[0030] Step S33: Initialize the search layer number of the CDR3 region to 1;
[0031] Step S34: Using the skeleton anchor point of the randomly initialized TCR sequence as the starting point, the node of the next level in the Monte Carlo tree... The stored information includes nodes Geometric features and nodes corresponding to TCR sequences Number of visits and nodes Cumulative rewards;
[0032] For any node in the previous layer, the policy network selects several actions based on the node's state, that is, from the splicing candidate pool according to the geometric features of the TCR sequence corresponding to that node. Several structural segments are selected from the data, and then the selected structural segments are used to replace the corresponding parts in the TCR sequence of the node. Each replaced TCR sequence is used as the TCR sequence corresponding to each child node of the node.
[0033] Similarly, each node in the previous layer is processed separately;
[0034] Step S35: Determine whether the search layer of the CDR3 region has been reached. ;
[0035] If the search layer of the CDR3 region reaches If so, proceed to step S36;
[0036] If the search depth of the CDR3 region has not reached the required level Then increment the search layer number of the CDR3 region by 1 and return to step S34.
[0037] Step S36: Generate the CDR1 region using the methods from steps S33 to S35 until the CDR1 region is generated. Then, generate the CDR2 region using the methods from steps S33 to S35 until the CDR2 region is generated. Then, generate the FR region using the methods from steps S33 to S35 until the entire Monte Carlo tree expansion process is completed. Then, continue to execute step S37.
[0038] Step S37: Filter the TCR sequences corresponding to each leaf node on the Monte Carlo tree when expansion stops, and use the filtered TCR sequences as the optimized TCR sequences.
[0039] Furthermore, the node The number of corresponding next-level nodes does not exceed 5.
[0040] Furthermore, in step S34, after replacing the corresponding part in the TCR sequence with the selected structural fragment, it is necessary to calculate the reward function value for each replaced TCR sequence.
[0041] If the reward function value corresponding to a replaced TCR sequence is less than or equal to 0, then the child node corresponding to the replaced TCR sequence is taken as a leaf node, and the leaf node will not continue to split.
[0042] If the reward function value corresponding to the replaced TCR sequence is greater than 0, then backtrack from the child node corresponding to the replaced TCR sequence to the root node, that is, update the cumulative reward and access count of the nodes on the backtracking path according to the reward function value.
[0043] Furthermore, the reward function value is calculated as follows:
[0044]
[0045] in, This represents the reward function value. , , , and All are weighting coefficients. This indicates the normalized value of the affinity reward. This represents the geometrically constrained reward normalized value. This represents the normalized value of the closed-loop constraint reward. This represents the normalized value of the structural stability reward. This represents the normalized value of the sequence diversity reward.
[0046] Furthermore, the step of updating the cumulative reward and visit count of nodes on the backtracking path based on the reward function value specifically involves:
[0047] The cumulative reward value of each node on the backtracking path increases, and the increase is equal to the reward function value corresponding to the backtracking starting node.
[0048] The number of visits to each node on the backtracking path is increased by 1.
[0049] Furthermore, the specific process of step S37 is as follows:
[0050] Step S371: Perform preliminary screening on the TCR sequences corresponding to each leaf node on the Monte Carlo tree when expansion stops, and preliminarily screen out the TCR sequences that meet the screening conditions.
[0051] Step S372: Then, use ProteinMPNN to optimize each of the initially screened TCR sequences, and then use AlphaFold2 or TCRBinder to obtain the affinity of each optimized TCR sequence to the antigen. Finally, take the optimized TCR sequence corresponding to the highest affinity as the final optimized TCR sequence.
[0052] Furthermore, in step S371, TCR sequences that meet the screening criteria are initially selected. The screening criteria are:
[0053] For any leaf node, if the average cumulative reward of all nodes on the path from the leaf node to the root node is greater than 1.5, the average number of visits to all nodes is greater than 10, the CDR3 conformation in the TCR sequence corresponding to the leaf node forms a closed loop with the TCR skeleton anchor point, and the Hamming distance between the TCR sequence corresponding to the leaf node and the initial TCR sequence to be optimized is greater than 2, then the leaf node meets the screening conditions; otherwise, the leaf node does not meet the screening conditions.
[0054] The beneficial effects of this invention are:
[0055] This invention, by introducing Monte Carlo tree search and geometric space constraints, efficiently explores and rapidly locates the optimal CDR3 conformation in a complex structural fragment space, significantly improving the speed and accuracy of TCR design. Compared to traditional sequence-based generation methods, this invention not only generates TCR structures highly matched to specific antigens, thus improving the specificity of the generated TCRs, but also possesses good path traceability and interpretability of the design process. Furthermore, the reinforcement learning algorithm uses the geometric features of the antigen peptide-major histocompatibility complex as the environment, thus exhibiting strong goal-oriented guidance for TCR generation. This invention provides a novel, efficient, and controllable solution for personalized TCR design in immunotherapy. Attached Figure Description
[0056] Figure 1 This is a flowchart of a T-cell receptor structure optimization method based on reinforcement learning according to the present invention;
[0057] Figure 2 This is a schematic diagram of the TCR-pMHC complex structure data processing provided by the present invention;
[0058] Figure 3 This is a schematic diagram of reinforcement learning provided by the present invention;
[0059] Figure 4 This is a flowchart of the Monte Carlo tree search method provided by the present invention. Detailed Implementation
[0060] Specific implementation method one: Combining Figure 1This embodiment describes a method for optimizing the structure of T-cell receptors based on reinforcement learning. The method specifically includes the following steps:
[0061] Step S1: Download each pdb file with a complete three-dimensional crystal structure from publicly available databases (including but not limited to STCRDab and TCR3d), and then obtain the TCR-pMHC complex structure data based on the downloaded pdb files;
[0062] It should be noted that after downloading the pdb file, files with duplicate pdb IDs need to be deleted. When multiple TCR-pMHC complex structures exist in a pdb file, these TCR-pMHC complex structures should be separated and saved as a single pdb file.
[0063] Based on the obtained TCR-pMHC complex structure data, a TCR structure fragment library was constructed, and the TCR structure fragment library was used as a splicing candidate pool. And obtain the geometric features of each structural fragment in the structural fragment library;
[0064] Step S2: Define reinforcement learning parameters, which include state space, action space, reward function, and policy network;
[0065] The state space is the geometric feature of the TCR sequence, utilizing a concatenated candidate pool. After replacing the TCR sequence with structural fragments, the splicing candidate pool is used. The geometric features of the structural fragments contained therein can be used to obtain the geometric features of the replaced TCR sequence, thus obtaining a new state; the action space is the structural fragments that the reinforcement learning algorithm can select for splicing at each step, i.e., the splicing candidate pool. The various structural segments in the network; the policy network uses a pre-trained ProteinMPNN encoder;
[0066] Step S3: Starting from the backbone anchor point of the TCR sequence to be optimized, the CDR1, CDR2, CDR3 and FR regions of the TCR sequence to be optimized are optimized using the Monte Carlo tree search method and reinforcement learning method to generate the optimized T cell receptor sequence.
[0067] This invention proposes a reinforcement learning-based TCR structure design method. Using the Monte Carlo Tree Search (MCTS) algorithm as a foundation, it constructs key regions such as CDR3 of the TCR through recursive assembly of structural fragments, and combines geometric constraints and reward mechanisms to achieve antigen-specific structure generation. The advantage of this method lies in its efficient search strategy driven by reinforcement learning, which can rapidly explore potential high-quality structural solutions in complex protein conformational spaces. The reward function comprehensively evaluates and provides feedback on the binding performance of the TCR to the antigen, interface matching, and spatial adaptability, quickly locating the optimal CDR3 conformation. This method naturally supports integration with antigen conditions, enabling the precise generation of highly specific and structurally reliable TCR conformations under specific pMHC backgrounds. This not only provides a new pathway for the rational structural design of TCR-T cell therapy but also provides important technical support for tumor neoantigen recognition and personalized immune engineering.
[0068] Specific Implementation Method Two: This implementation method differs from Specific Implementation Method One in that the construction of a TCR structure fragment library based on the obtained TCR-pMHC complex structure data specifically involves:
[0069] Step S11: For any TCR-pMHC complex structure, the TCR-pMHC complex structure includes a TCRα chain, a TCR Beta chain, an antigenic peptide, and an MHC structure; in the pdb file, chain A is the main structural region of the MHC, and chain B is the MHC... The structure is as follows: C chain is the antigenic peptide structure, D chain is the alpha chain of TCR, and E chain is the beta chain of TCR. CDR1, CDR2, CDR3, and FR regions were extracted from the alpha and beta chains of TCR, and the start and end residues of each region were determined using the annotation tool IMGT.
[0070] Step S12: Using a sliding window with a window length of L (L is 3 in this invention), slide the window across each extracted region. When the sliding window is at each position, the residues within the window are taken as a structural fragment. Record the sequence information and position information of the structural fragments (the sequence information and position information include the sequence of residues in the structural fragment, the three-dimensional coordinates of each residue in the structural fragment, the type of residues in the structural fragment, the chain number (Alpha chain or Beta chain) where the structural fragment is located, and the position index information of each residue in the chain).
[0071] Taking the CDR3 region as an example, the sliding window step size is set to 1. The first to the Lth residues in the CDR3 region are treated as one structural segment, the second to the (L+1)th residues in the CDR3 region are treated as one structural segment, and so on, until the end of the window reaches the last residue of the CDR3 region.
[0072] Step S13: After processing each obtained TCR-pMHC complex structure according to steps S11 and S12, a non-redundant TCR structure fragment library covering different sources and different region lengths is constructed.
[0073] The other steps and parameters are the same as in Specific Implementation Method 1.
[0074] Specific implementation method three: Combining Figure 2 This embodiment is described below. The difference between this embodiment and specific embodiments one or two is that the step of obtaining the geometric features of each structural fragment in the structural fragment library specifically involves:
[0075] For any given structural fragment, the main chain dihedral angle, side chain bond angle, bond length, and other parameters of each amino acid residue within the fragment are extracted based on the fragment's sequence and position information. The three-dimensional coordinates of the atom will determine the first atom within the structural segment. amino acid residues The dihedral angle of the main chain is denoted as and , will the amino acid residues The side chain bond angle is denoted as , , and , will the amino acid residues With the amino acid residues The bond length is denoted as , Indicates the relationship between the current structural segment and the first... amino acid residues Adjacent amino acid residues, Indicates the first amino acid residues of The three-dimensional coordinates of an atom And then according to , , , , , , and Calculate the first amino acid residues Local curvature and normal vector , the first in the structural fragment amino acid residues A vector composed of geometric descriptors is used as a graph. The Middle Features of each node :
[0076]
[0077] The bonds between every two adjacent amino acid residues within the structural fragment are respectively represented as diagrams. If one of the edges is in the middle, then the first edge is... amino acid residues With the amino acid residues Features of the edges between for:
[0078]
[0079] in, Indicates the first amino acid residues of Atom and the first amino acid residues of Euclidean distance between atoms , Indicates the first amino acid residues of The three-dimensional coordinates of an atom Describing the 2-norm, Indicates the first amino acid residues of Atom to the amino acid residues of The direction vector of the line connecting the atoms;
[0080] Representing structural segments as structural diagrams ,in, This represents the set of nodes corresponding to a structural segment. Represents the set of edges corresponding to structural segments, where the graph All edges in the equation are directed edges pointing from the previous residue to the next residue;
[0081] ProteinMPNN encoder pre-trained with protein structure For the structural diagram Embedding extraction is performed to obtain the geometric feature representation of the structural fragments. :
[0082]
[0083] in, Geometric feature representation of structural segments;
[0084] Similarly, separate candidate pools for splicing are obtained. Geometric feature representation of each structural segment.
[0085] Other steps and parameters are the same as in specific implementation method one or two.
[0086] Specific implementation method four: Combination Figure 3 and Figure 4 This embodiment is described below. The difference between this embodiment and one of the specific embodiments one to three is that the specific process of step S3 is as follows:
[0087] Step S31: Initialize the number of search layers in the Monte Carlo tree used to generate the CDR3 region. The number of search layers used to generate the CDR1 region is The number of search layers used to generate the CDR2 region is The number of search layers used to generate the FR region is Using a splicing candidate pool Random initialization of the CDR3 region in the TCR sequence to be optimized (i.e., using the splicing candidate pool) The structural fragment corresponding to the CDR3 region in the TCR sequence to be optimized is randomly replaced to obtain the randomly initialized TCR sequence.
[0088] The optimization order for the TCR sequence to be optimized is set as follows: optimize the CDR3 region, CDR1 region, CDR2 region and FR region in sequence;
[0089] The geometric features of the pMHC corresponding to the TCR sequence to be optimized (in this invention, pMHC does not need to be segmented; the entire pMHC is treated as a structural segment for geometric feature extraction, and the geometric feature extraction method is the same as described above) are used as the environmental information for reinforcement learning. ;
[0090] Step S32: Use the randomly initialized TCR sequence as the root node. ;
[0091] Step S33: Initialize the search layer number of the CDR3 region to 1;
[0092] Step S34: Using the skeleton anchor point of the randomly initialized TCR sequence as the starting point, the node of the next level in the Monte Carlo tree... The stored information includes nodes Geometric features and nodes corresponding to TCR sequences Number of visits and nodes The cumulative rewards; among them, the number of visits and the cumulative rewards are updated gradually as the search process progresses;
[0093] It should be noted that: the backbone anchors of the randomly initialized TCR sequence are the upstream and downstream anchor residues that recognize the CDR3 region, including N-terminal and C-terminal anchors. The N-terminal anchor is the last residue of FR3, usually a conserved cysteine or aromatic amino acid; the C-terminal anchor is the first residue of FR4, usually glycine or proline, which serve as start and end restrictions in structure generation; for the first layer search, the node in the previous layer is the root node. .
[0094] For any node in the previous layer, the policy network selects several actions based on the node's state, that is, from the splicing candidate pool according to the geometric features of the TCR sequence corresponding to that node. Several structural segments are selected (the selected structural segments should belong to the CDR3 region. When selecting structural segments, the policy network outputs the probability distribution of each action. Here, the structural segments corresponding to the highest probabilities are selected). The selected structural segments are then used to replace the corresponding parts of the TCR sequence of the node. Each replaced TCR sequence is used as the TCR sequence of each child node of the node.
[0095] Similarly, each node in the previous layer is processed separately;
[0096] It should be noted that: due to the splicing candidate pool Each structural segment in the sequence has a corresponding position in the CDR3 region. Therefore, after selecting a structural segment, the corresponding part in the TCR sequence of the parent node can be replaced by the selected structural segment to obtain a child node of the parent node.
[0097] Step S35: Determine whether the search layer of the CDR3 region has been reached. ;
[0098] If the search layer of the CDR3 region reaches If so, proceed to step S36;
[0099] If the search depth of the CDR3 region has not reached the required level Then increment the search layer number of the CDR3 region by 1 and return to step S34.
[0100] Step S36: Generate the CDR1 region using the methods from steps S33 to S35 until the CDR1 region is generated. Then, generate the CDR2 region using the methods from steps S33 to S35 until the CDR2 region is generated. Then, generate the FR region using the methods from steps S33 to S35 until the entire Monte Carlo tree expansion process is completed. Then, continue to execute step S37.
[0101] Step S37: Filter the TCR sequences corresponding to each leaf node on the Monte Carlo tree when expansion stops, and use the filtered TCR sequences as the optimized TCR sequences.
[0102] The other steps and parameters are the same as those in one of the specific implementation methods one to three.
[0103] The TCR sequence to be optimized in this invention contains sequences with Alpha and Beta V regions. , Three-dimensional structure and anchor point coordinates CDR1 and CDR2 are based on conserved motifs in the V region of genes.
[0104] Specific Implementation Method Five: This implementation method differs from Specific Implementation Methods One to Four in that the node... The number of corresponding next-level nodes does not exceed 5.
[0105] The other steps and parameters are the same as those in one of the specific implementation methods one to four.
[0106] That is, the present invention limits the number of child nodes of each parent node to no more than 5.
[0107] Specific Implementation Method Six: This implementation method differs from Specific Implementation Methods One to Five in that, in step S34, after replacing the corresponding part in the TCR sequence with the selected structural fragment, it is necessary to calculate the reward function value based on each replaced TCR sequence.
[0108] If the reward function value corresponding to a replaced TCR sequence is less than or equal to 0, then the child node corresponding to the replaced TCR sequence is taken as a leaf node, and the leaf node will not continue to split.
[0109] If the reward function value corresponding to the replaced TCR sequence is greater than 0, then backtrack from the child node corresponding to the replaced TCR sequence to the root node, that is, update the cumulative reward and access count of the nodes on the backtracking path according to the reward function value.
[0110] The other steps and parameters are the same as those in one of the specific implementation methods one to five.
[0111] This invention introduces Monte Carlo tree search and geometric space constraints to efficiently explore and quickly locate the optimal CDR3 conformation in complex structural fragment space, significantly improving the speed and accuracy of TCR design.
[0112] Specific Implementation Method Seven: This implementation method differs from Specific Implementation Methods One through Six in that the method for calculating the reward function value is as follows:
[0113]
[0114] in, This represents the reward function value. , , , and All are weighting coefficients. This indicates the normalized value of the affinity reward. This represents the geometrically constrained reward normalized value. This represents the normalized value of the closed-loop constraint reward. This represents the normalized value of the structural stability reward. This represents the normalized value of the sequence diversity reward.
[0115] The other steps and parameters are the same as those in one of the specific implementation methods one to six.
[0116] The calculation process for each reward is explained below:
[0117] (1) Rosetta is used to calculate the affinity reward.
[0118] (2) Geometric constraint rewards are calculated according to the TCR geometric constraint rules;
[0119] The TCR geometric constraint rules include: the atomic distance between the splicing endpoints of structural segments is maintained at a certain level. And the key angle is limited to 100 degrees. ~120 Dihedrals formed by splicing adjacent segments and Changes not exceeding And it falls within the allowable range of protein secondary structure; TCR and pMHC form a mixture smaller than... The binding interface does not form steric hindrance; the generated CDR3 fragment length ranges from 12 to 16, and the distance to the contact residues with pMHC should be less than [missing information]. ;
[0120] If all the above TCR geometric constraint rules are satisfied, the geometric constraint reward is 1; otherwise, the geometric constraint reward is 0.
[0121] (3) Closed-loop constraint reward
[0122] TCR anchor point closed-loop constraints are used to ensure that the generated CDR3 is structurally and physically reasonable. Structures that fail to close the loop are given a negative score in the reward phase (the closed-loop constraint reward is -1); structures that successfully close the loop are given a positive score in the reward phase (the closed-loop constraint reward is 1).
[0123] (4) The structural stability reward is calculated using Rosetta Relax.
[0124] (5) The sequence diversity reward is calculated using Hamming distance.
[0125] After calculating each reward, the reward values need to be normalized, and then a weighted sum of the normalized reward values is performed to obtain the reinforcement learning reward. The weight parameters in this invention are used to balance the contribution of each reward item. In this invention, the weight parameters are set... , , , , .
[0126] Specific Implementation Method Eight: This implementation method differs from Specific Implementation Methods One to Seven in that the step of updating the cumulative reward and visit count of nodes on the backtracking path based on the reward function value is specifically as follows:
[0127] The cumulative reward value of each node on the backtracking path increases, and the increase is equal to the reward function value corresponding to the backtracking starting node.
[0128] The number of visits to each node on the backtracking path is increased by 1.
[0129] The other steps and parameters are the same as those in any of the specific implementation methods one to seven.
[0130] Specific Implementation Method Nine: This implementation method differs from Specific Implementation Methods One to Eight in that the specific process of step S37 is as follows:
[0131] Step S371: Perform preliminary screening on the TCR sequences corresponding to each leaf node on the Monte Carlo tree when expansion stops, and preliminarily screen out the TCR sequences that meet the screening conditions.
[0132] Step S372: Then, use ProteinMPNN to optimize each of the initially screened TCR sequences, and then use AlphaFold2 or TCRBinder to obtain the affinity of each optimized TCR sequence to the antigen. Finally, take the optimized TCR sequence corresponding to the highest affinity as the final optimized TCR sequence.
[0133] The other steps and parameters are the same as those in one of the specific implementation methods one to eight.
[0134] Specific Implementation Method Ten: This implementation method differs from Specific Implementation Methods One to Nine in that, in step S371, TCR sequences that meet the screening criteria are initially selected. The screening criteria are:
[0135] For any leaf node, if the average cumulative reward of all nodes on the path from the leaf node to the root node is greater than 1.5, the average number of visits to all nodes is greater than 10, the CDR3 conformation in the TCR sequence corresponding to the leaf node forms a closed loop with the TCR skeleton anchor point, and the Hamming distance between the TCR sequence corresponding to the leaf node and the initial TCR sequence to be optimized is greater than 2, then the leaf node meets the screening conditions; otherwise, the leaf node does not meet the screening conditions.
[0136] The other steps and parameters are the same as those in any of the specific implementation methods one to nine.
[0137] The above examples of this invention are merely illustrative of the computational model and process of this invention, and are not intended to limit the implementation of this invention. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is impossible to exhaustively list all possible implementations here. Any obvious variations or modifications derived from the technical solutions of this invention are still within the scope of protection of this invention.
Claims
1. A method for optimizing the structure of T-cell receptors based on reinforcement learning, characterized in that, The method specifically includes the following steps: Step S1: Download each pdb file from a publicly available database, and then obtain the TCR-pMHC complex structure data based on the downloaded pdb files; Based on the obtained TCR-pMHC complex structure data, a TCR structure fragment library was constructed, and the TCR structure fragment library was used as a splicing candidate pool. And obtain the geometric features of each structural fragment in the structural fragment library; Step S2: Define reinforcement learning parameters, which include state space, action space, reward function, and policy network; Step S3: Starting from the backbone anchor point of the TCR sequence to be optimized, the CDR1, CDR2, CDR3 and FR regions of the TCR sequence to be optimized are optimized using the Monte Carlo tree search method and reinforcement learning method to generate the optimized T cell receptor sequence.
2. The method for optimizing T-cell receptor structure based on reinforcement learning according to claim 1, characterized in that, The construction of a TCR structure fragment library based on the obtained TCR-pMHC complex structure data specifically involves: Step S11: For any TCR-pMHC complex structure, extract the CDR1, CDR2, CDR3 and FR regions from the Alpha and Beta chains of TCR. Step S12: Using a sliding window of length L, slide it across each extracted region. When the sliding window is at each position, the residues within the window are taken as a structural fragment, and the sequence information and position information of the structural fragment are recorded. Step S13: After processing each obtained TCR-pMHC complex structure according to steps S11 and S12, a TCR structure fragment library is constructed.
3. The method for optimizing T-cell receptor structure based on reinforcement learning according to claim 2, characterized in that, The specific steps for obtaining the geometric features of each structural fragment in the structural fragment library are as follows: For any given structural fragment, the main chain dihedral angle, side chain bond angle, bond length, and other parameters of each amino acid residue within the fragment are extracted based on the fragment's sequence and position information. The three-dimensional coordinates of the atom will determine the first atom within the structural segment. amino acid residues The dihedral angle of the main chain is denoted as and , will the amino acid residues The side chain bond angle is denoted as , , and , will the amino acid residues With the amino acid residues The bond length is denoted as , Indicates the relationship between the current structural segment and the first... amino acid residues Adjacent amino acid residues, Indicates the first amino acid residues of The three-dimensional coordinates of the atom, and then according to , , , , , , and Calculate the first amino acid residues Local curvature and normal vector , the first in the structural fragment amino acid residues A vector composed of geometric descriptors is used as a graph. The Middle Features of each node : The bonds between every two adjacent amino acid residues within the structural fragment are respectively represented as diagrams. If one of the edges is in the middle, then the first edge is... amino acid residues With the amino acid residues Features of the edges between for: in, Indicates the first amino acid residues of Atom and the first amino acid residues of Euclidean distance between atoms , Indicates the first amino acid residues of The three-dimensional coordinates of an atom Describes the 2-norm. Indicates the first amino acid residues of Atom to the amino acid residues of The direction vector of the line connecting the atoms; Represent structural segments as structural diagrams ,in, This represents the set of nodes corresponding to a structural segment. Represents the set of edges corresponding to a structural segment; Using ProteinMPNN encoder For the structural diagram Embedding extraction is performed to obtain the geometric feature representation of the structural fragments. : in, Geometric feature representation of structural segments; Similarly, separate candidate pools for splicing are obtained. Geometric feature representation of each structural segment.
4. The method for optimizing T-cell receptor structure based on reinforcement learning according to claim 3, characterized in that, The specific process of step S3 is as follows: Step S31: Initialize the number of search layers in the Monte Carlo tree used to generate the CDR3 region. The number of search layers used to generate the CDR1 region is The number of search layers used to generate the CDR2 region is The number of search layers used to generate the FR region is Using a splicing candidate pool The CDR3 region in the TCR sequence to be optimized is randomly initialized to obtain the randomly initialized TCR sequence; The optimization order for the TCR sequence to be optimized is set as follows: optimize the CDR3 region, CDR1 region, CDR2 region and FR region in sequence; The geometric features of the pMHC corresponding to the TCR sequence to be optimized are used as environmental information for reinforcement learning. ; Step S32: Use the randomly initialized TCR sequence as the root node. ; Step S33: Initialize the search layer number of the CDR3 region to 1; Step S34: Using the skeleton anchor point of the randomly initialized TCR sequence as the starting point, the node of the next level in the Monte Carlo tree... The stored information includes nodes Geometric features and nodes corresponding to TCR sequences Number of visits and nodes Cumulative rewards; For any node in the previous layer, the policy network selects several actions based on the node's state, that is, from the splicing candidate pool according to the geometric features of the TCR sequence corresponding to that node. Several structural segments are selected from the data, and then the selected structural segments are used to replace the corresponding parts in the TCR sequence of the node. Each replaced TCR sequence is used as the TCR sequence corresponding to each child node of the node. Similarly, each node in the previous layer is processed separately; Step S35: Determine whether the search layer of the CDR3 region has been reached. ; If the search layer of the CDR3 region reaches If so, proceed to step S36; If the search depth of the CDR3 region has not reached the required level Then increment the search layer number of region CDR3 by 1 and return to step S34. Step S36: Generate the CDR1 region using the methods from steps S33 to S35 until the CDR1 region is generated. Then, generate the CDR2 region using the methods from steps S33 to S35 until the CDR2 region is generated. Then, generate the FR region using the methods from steps S33 to S35 until the entire Monte Carlo tree expansion process is completed. Then, continue to execute step S37. Step S37: Filter the TCR sequences corresponding to each leaf node on the Monte Carlo tree when expansion stops, and use the filtered TCR sequences as the optimized TCR sequences.
5. The method for optimizing T-cell receptor structure based on reinforcement learning according to claim 4, characterized in that, The node The number of corresponding next-level nodes does not exceed 5.
6. The method for optimizing T-cell receptor structure based on reinforcement learning according to claim 5, characterized in that, In step S34, after replacing the corresponding part in the TCR sequence with the selected structural fragment, the reward function value needs to be calculated for each replaced TCR sequence. If the reward function value corresponding to a replaced TCR sequence is less than or equal to 0, then the child node corresponding to the replaced TCR sequence is taken as a leaf node, and the leaf node will not continue to split. If the reward function value corresponding to the replaced TCR sequence is greater than 0, then backtrack from the child node corresponding to the replaced TCR sequence to the root node, that is, update the cumulative reward and access count of the nodes on the backtracking path according to the reward function value.
7. The method for optimizing T-cell receptor structure based on reinforcement learning according to claim 6, characterized in that, The method for calculating the reward function value is as follows: in, This represents the reward function value. , , , and All are weighting coefficients. This indicates the normalized value of the affinity reward. This represents the geometrically constrained reward normalized value. This represents the normalized value of the closed-loop constraint reward. This represents the normalized value of the structural stability reward. This represents the normalized value of the sequence diversity reward.
8. The method for optimizing T-cell receptor structure based on reinforcement learning according to claim 7, characterized in that, The step of updating the cumulative reward and visit count of nodes on the backtracking path based on the reward function value is as follows: The cumulative reward value of each node on the backtracking path increases, and the increase is equal to the reward function value corresponding to the backtracking starting node. The number of visits to each node on the backtracking path is increased by 1.
9. The method for optimizing T-cell receptor structure based on reinforcement learning according to claim 8, characterized in that, The specific process of step S37 is as follows: Step S371: Perform preliminary screening on the TCR sequences corresponding to each leaf node on the Monte Carlo tree when expansion stops, and preliminarily screen out the TCR sequences that meet the screening conditions. Step S372: Then, use ProteinMPNN to optimize each of the initially screened TCR sequences, and then use AlphaFold2 or TCRBinder to obtain the affinity of each optimized TCR sequence to the antigen. Finally, take the optimized TCR sequence corresponding to the highest affinity as the final optimized TCR sequence.
10. The method for optimizing T-cell receptor structure based on reinforcement learning according to claim 9, characterized in that, In step S371, TCR sequences that meet the screening criteria are initially selected. The screening criteria are as follows: For any leaf node, if the average cumulative reward of all nodes on the path from the leaf node to the root node is greater than 1.5, the average number of visits to all nodes is greater than 10, the CDR3 conformation in the TCR sequence corresponding to the leaf node forms a closed loop with the TCR skeleton anchor point, and the Hamming distance between the TCR sequence corresponding to the leaf node and the initial TCR sequence to be optimized is greater than 2, then the leaf node meets the screening conditions; otherwise, the leaf node does not meet the screening conditions.