A reinforcement learning-based method for nanobody generation

Abstract

This invention discloses a nanobody generation method based on reinforcement learning, belonging to the field of computer simulation technology. It aims to address the problem of significantly reduced generation accuracy in existing nanobody generation models when facing "low-similarity antigens." The method includes: adding a LoRA adapter to a pre-trained nanobody generation model to generate an initial model; inputting the antigen conformation and antibody template, generating candidate antibody samples based on the initial model, and calculating scores using a Boltz-2 folding model and PyRosetta software; adding noise to the 3D coordinates of the selected samples, using the encoder output during generation as constraints, calculating the denoised 3D coordinates using an RL model and an Old model, calculating the loss function, and updating the RL adapter; when the average reward value in each round reaches a preset condition, outputting RL-LoRA weights to generate antibodies. By aligning the model with specific low-similarity antigens through reinforcement learning, the generation efficiency and accuracy of the model for novel antigens are improved even without known antibody references.

Description

Technical Field

[0001] This invention relates to the field of computer simulation technology, and more specifically to a method for generating nanobodies based on reinforcement learning. Background Technology

[0002] Antibodies are an important component of human immunology. The human body contains a large number of antibodies with different complementarity-determining regions (CDRs), and these different CDRs influence the antigens that antibodies can bind to. The CDRs of antibodies are distributed along one heavy chain and one light chain. Nanobodies are antibodies unique to the camel family. Unlike conventional antibodies, the CDRs of nanobodies are all distributed along a single heavy chain. Nanobodies have several advantages over traditional antibodies: higher penetration, higher stability, and stronger binding performance. Obtaining high-affinity nanobodies using traditional methods requires expensive experiments. Obtaining nanobodies with binding properties through computer analysis (based on deep learning-based nanobody generation models) can significantly reduce the upfront costs of nanobody drug development.

[0003] Current nanobody generation models (such as CN121641161A) can be divided into three generations: the first generation model (such as DiffAb) can generate complementarity-determining regions based on a fixed framework and binding sites; the second generation model (such as RFAantibody) first generates the backbone and binding sites, and then generates binding residues through the ProteinMPNN tool; the third generation model (such as BoltzGen) can directly generate nanobodies with affinity in a single step.

[0004] Currently, there is no method to align nanobody models to low-similarity antigens. The closest approach is fine-tuning on the antigen to be predicted, but this method requires antibody data and cannot be applied to low-similarity antigens for which no known antigens exist; current fine-tuning methods are highly dependent on known data of the "antigen-antibody complex". When dealing with novel antigens, the model cannot perform effective supervised learning due to the lack of ground truth. Summary of the Invention

[0005] This invention addresses the problem that existing nanobody generation models suffer from a significant drop in generation accuracy when faced with "low-similarity antigens." It proposes a reinforcement learning-based nanobody generation method that aligns the model with specific low-similarity antigens through reinforcement learning, thereby improving the model's generation efficiency and accuracy for novel antigens in the absence of known antibody references.

[0006] To achieve the above objectives, the present invention adopts the following technical solution.

[0007] A reinforcement learning-based method for generating nanobodies includes the following steps: S1, Add a LoRA adapter to the pre-trained nanobody generation model to generate an initialization model; S2, input antigen conformation and antibody template, generate candidate antibody samples based on initialization model, and calculate scores through Boltz-2 folding model and PyRosetta software; S3, add noise to the 3D coordinates of the selected samples, use the encoder output at the time of generation as a constraint, calculate the denoised 3D coordinates through the RL model and the Old model, calculate the loss function, and update the RL adapter; S4, when the average reward value in each round reaches the preset condition, output the RL-LoRA weight to generate the antibody.

[0008] In this technical solution, two LoRA adapters are first added to the pre-trained nanobody generation model. Then, candidate antibody samples are generated based on the input antigen conformation and antibody template. Samples that meet the criteria are selected by calculating scores. Noise is added to the 3D coordinates of the selected samples, and reinforcement learning training steps are performed to update the weights of the PL adapters. Subsequently, through parameter synchronization and iterative optimization, the final RL-LoRA weights are obtained through iterative iteration, and finally, the antibody is generated. This improves the model's performance on low-similarity antigens even without known antibody references.

[0009] The present invention is further configured such that step S1 includes: Two sets of LoRA adapters, including RL adapters and Old adapters, are embedded in parallel in the diffusion control layer and diffusion model of the pre-trained nanobody generation model.

[0010] The present invention is further configured such that step S2 includes: S21, input the target antigen conformation and multiple antibody templates; S22, Generate candidate antibody samples in batches based on the frozen Ref model and the Old adapter; S23, call the Boltz-2 folding model to calculate the confidence score, and use the PyRosetta dG_separated energy function to calculate the affinity energy score, and perform quality assessment and screening on the generated samples.

[0011] In this technical solution, after inputting the target antigen conformation and multiple antibody templates, candidate antibodies can be generated in batches using an initialization model.

[0012] The present invention is further configured such that step S3 includes: S31, randomly set the sampling diffusion step number t, and analyze the original coordinates of the selected samples. Noise was added during the process. ; S32 uses the RL model and the Old model for denoising prediction to obtain the positive predicted coordinates. Compare with negative to predict coordinates ; S33, Calculation and The mean squared error between the two values ​​is used to calculate the AWM loss function, which is then used to update the weights of the RL adapter.

[0013] In this technical solution, the 3D coordinates of the selected samples are noise-added, and reinforcement learning training steps are performed.

[0014] The present invention is further configured such that step S4 includes: The parameters of the Old adapter are updated according to the exponential moving average formula, so that it slowly follows the evolution of the RL adapter; Each round regenerates samples and trains until the average reward growth rate of two consecutive rounds is less than a preset threshold or the maximum number of training rounds is reached. Finally, the trained RL-LoRA weights are output to generate antibodies.

[0015] In this technical solution, parameter synchronization and iterative optimization ultimately yield accurate RL-LoRA weights.

[0016] The present invention is further configured such that step S23 includes: Calculate the affinity ranking and performance ranking of the sample within the batch, and take the maximum of the two. ; Each sample Convert to a linearly normalized reward value between 0 and 1 .

[0017] The present invention is further configured such that the target antigen conformation includes a PDB file and a chain identifier.

[0018] The present invention is further configured such that: the RL model is a Ref network plus an RL adapter, and the Old model is a Ref network plus an Old adapter.

[0019] The present invention is further configured such that: when generating antibodies, the RL-LoRA weights are directly loaded into the pre-trained nanobody generation model to output nanobodies optimized for the antigen.

[0020] The present invention is further configured such that the pre-trained nanobody generation model is the BoltzGen model.

[0021] The present invention provides a reinforcement learning-based method for generating nanobodies, which can bring the following beneficial effects: Current nanobody generation models have low accuracy on low-similarity antigens, thus requiring the generation of a large number of samples. To address this issue, reinforcement learning is used to align the model with specific low-similarity antigens, thereby improving the model's generation accuracy, reducing the number of samples required, and lowering computational costs. Attached Figure Description

[0022] Figure 1 This is a schematic flowchart of a reinforcement learning-based nanobody generation method according to the present invention.

[0023] Figure 2 This is a schematic diagram of a reinforcement learning-based nanobody generation method according to the present invention. Detailed Implementation

[0024] Example 1:

[0025] To address the technical problem of significantly reduced accuracy in existing nanobody generation models when faced with "low-similarity antigens," this embodiment proposes a reinforcement learning-based nanobody generation method, referencing... Figure 1 and Figure 2 It mainly includes the following steps S1 to S4.

[0026] Step S1: Add the LoRA adapter to the pre-trained nanobody generation model to generate the initialization model.

[0027] In this embodiment, the pre-trained nanobody generation model is the BoltzGen model.

[0028] For step S1, the process is as follows: two sets of LoRA adapters are embedded in parallel in the diffusion control layer and diffusion model of the pre-trained nanobody generation model. The LoRA adapters include RL adapters and Old adapters. The Ref network of the pre-trained nanobody generation model can freeze the weights of the original pre-trained model, provide a benchmark for policy constraints, and prevent the model from deviating too far from the original distribution.

[0029] More specifically, in this embodiment, two sets of LoRA adapters are embedded in parallel within the diffusion control layer and the noise prediction network (for the final layer and diffusion layer of the AlphaFold3 type encoder) of the main model using PEFT (parameter efficient fine-tuning).

[0030] For RL adapters ( It is a trainable layer; it uses LoRA default initialization ( The matrix is ​​initialized using a Gaussian distribution (standard deviation 1e-3). (Clear the matrix to zero) to ensure that the increment is 0 at the start of training. For the Old adapter ( ): It is a non-gradient update layer; the parameters are synchronized by the RL adapter through exponential moving average (EMA).

[0031] Step S2: Input the antigen conformation and antibody template, generate candidate antibody samples based on the initialization model, and calculate the scores using the Boltz-2 folding model and PyRosetta software.

[0032] Step S2 above mainly includes the following sub-steps.

[0033] Step S21: Input the target antigen conformation and multiple antibody templates; in this embodiment, the target antigen conformation includes a PDB file and a chain identifier.

[0034] Step S22: Generate candidate antibody samples in batches based on the frozen Ref model and the Old adapter.

[0035] In this embodiment, 1024 candidate antibodies are generated in batches using the BoltzGen model for a given antigen.

[0036] For the specific sampling strategy, 16 antibodies were generated from each of the 64 antibody templates. Atomic coordinates were saved. The complex is encoded for subsequent NFT training.

[0037] Step S23: The Boltz-2 folding model is called to calculate the confidence score, and the affinity energy score is calculated using the dG_separated energy function of PyRosetta. The generated samples are then evaluated and screened for quality.

[0038] More specifically, step S23 mainly includes confidence scoring and affinity scoring.

[0039] For confidence scoring ( Specifically, it calls the Boltz2 built-in confidence function to calculate ipAE (interface position deviation) and pLDDT.

[0040] For the affinity score ( Specifically, the interface binding energy is calculated using PyRosetta's dG_separated energy function.

[0041] Step S23 further includes: calculating the affinity ranking and performance ranking of the sample within the batch, and taking the maximum value of the two to obtain... ; for each sample Convert to a linearly normalized reward value between 0 and 1 .

[0042] More specifically, after completing the confidence and affinity scoring, the affinity ranking and execution ranking of the sample within the batch are calculated. The maximum value between the affinity ranking and the execution ranking is then taken to obtain the joint sequence number. .

[0043] Design Basis: Based on the "barrel principle"; if a nanobody fails in any one of its metrics (structure or energy), the entire nanobody will fail. The worst-performing metric is selected as the indicator, forcing the model to consider both performance aspects simultaneously.

[0044] Combined serial number Mapping to linearly normalized reward , It equals 1 minus the first intermediate term, where the first intermediate term is... Subtract 1 from the difference and then divide by Subtract 1 from the difference.

[0045] In this embodiment, The score is 1024. The reward for the smallest (optimal) number is 1, and the reward for the largest number is 0.

[0046] Step S3: Add noise to the 3D coordinates of the selected samples, use the encoder output at the time of generation as a constraint, calculate the denoised 3D coordinates through the RL model and the Old model, calculate the loss function, and update the RL adapter.

[0047] In this embodiment, 32 sets of samples are aggregated according to the template dimension, and the highest reward in each set is taken as the reward value of the template. The 16 sets of templates (a total of 512 samples) with the highest reward values ​​in the top 50% are retained for training.

[0048] Step S3 above includes the following sub-steps.

[0049] Step S31: Randomly set the sampling diffusion step number t, and calculate the original coordinates of the selected samples. Noise was added during the process. .

[0050] In more detail, noise was added to the 512 selected samples.

[0051] Noise addition processing is: random sampling diffusion steps ,right Noise addition was performed according to the diffusion schedule. Step S32: Perform denoising prediction using the RL model and the Old model to obtain the positive prediction coordinates. Compare with negative to predict coordinates The RL model is a Ref network with an RL adapter, and the Old model is a Ref network with an Old adapter.

[0052] For positive prediction coordinates It is based on the input at the current time t Obtained after processing by the RL model; for negative contrast prediction coordinates It is equal to twice the size of the Old model. The output at the current time t is then subtracted from the output of the RL model.

[0053] Step S33, Calculate and The mean squared error between the two values ​​is used to calculate the AWM loss function, which is then used to update the weights of the RL adapter.

[0054] The AWM loss function is equal to the forward predicted coordinates. Compare with negative to predict coordinates The mean square error of the coordinates is multiplied by the reward weight factor 2r-1, so that high reward samples produce positive imitation and low reward samples produce negative avoidance.

[0055] Step S4: When the average reward value in each round reaches the preset condition, output the RL-LoRA weight to generate the antibody.

[0056] Step S4 includes the following more specific process: updating the parameters of the Old adapter according to the exponential moving average formula so that it slowly follows the evolution of the RL adapter; regenerating samples and training in each round until the average reward growth rate of two consecutive rounds is less than a preset threshold or the maximum number of training rounds is reached, and finally outputting the trained RL-LoRA weights to generate antibodies.

[0057] More specifically, after calculating the loss for all 16 samples in a group, for Perform one gradient descent iteration (using the Adam optimizer with an update rate of 5e-4, plus a weight decay of 0.01); after all 512 samples have undergone 10 iterations of training with random noise, proceed as follows: Update the Old adapter; in this embodiment, It is 0.5.

[0058] The average reward value of each round converges (the growth rate is less than a preset threshold for two consecutive rounds, which is 0.5% in this embodiment) or reaches the preset 10 large rounds (Epoch). Each round contains 1024 sampled samples, and each sample is trained with random noise 10 times.

[0059] After training is complete, the RL-LoRA weights are output. When generating antibodies, the RL-LoRA weights are directly loaded into the pre-trained model, and the output according to the standard sampling procedure is the nanobody optimized for the antigen.

[0060] In this technical solution, two LoRA adapters are first added to the pre-trained nanobody generation model. Then, candidate antibody samples are generated based on the input antigen conformation and antibody template. Samples that meet the criteria are selected by calculating scores. Noise is added to the 3D coordinates of the selected samples, and reinforcement learning training steps are performed to update the weights of the PL adapters. Subsequently, through parameter synchronization and iterative optimization, the final RL-LoRA weights are obtained through iterative iteration, and finally, the antibody is generated. This improves the model's performance on low-similarity antigens even without known antibody references.

[0061] In this embodiment, for the LoRA adapter, the original weights W of the model are frozen, and two trainable low-rank weight matrices A and B are added. By using... This simulates training the model weights. The Ref network remains constant, providing the original knowledge distribution. KL divergence constraints are used to prevent the RL from deviating significantly from the model. The Old adapter is a slowly evolving "smooth version of itself," primarily calculating comparison terms and stabilizing training; it can generate training data and also serves as a training benchmark. The RL adapter acts as a "pathfinder" for real-time learning, accepting gradient updates.

[0062] The existing model (BoltzGen) achieves a success rate of up to 33% when processing antigens with existing antibody data, but the success rate drops sharply to about 11% when processing low-similarity antigens because the antigen features deviate from the distribution of the pre-training data. This invention aims to improve the efficiency and accuracy of the model in generating novel antigens by aligning the model with specific low-similarity antigens through reinforcement learning (RL) in the absence of known antibody references.

[0063] The above-described solution in this embodiment can bring the following technical effects: 1. Improved accuracy: Compared to the original BoltzGen, it improves performance on low-similarity antigens; 2. Cost reduction: By increasing the success rate of single sampling, the total number of samples required for experimental verification is reduced, thereby reducing computing power and consumable costs; 3. Multi-index optimization: Max-Rank logic effectively solves the problem of outlier solutions in generative models that are "high affinity but structurally broken".

[0064] Example 2:

[0065] Current nanobody generation models have low accuracy in generating antigens with low similarity, thus requiring the generation of a large number of samples. This invention can improve the generation accuracy of the model, thereby reducing the required number of samples and reducing computational power consumption, i.e., reducing costs. To this end, this embodiment proposes a nanobody generation method based on reinforcement learning, which mainly includes the following steps S1 to S4.

[0066] Step S1: Add the LoRA adapter to the pre-trained nanobody generation model to generate the initialization model.

[0067] In this embodiment, the pre-trained nanobody generation model is the BoltzGen model.

[0068] For step S1, the process is as follows: two sets of LoRA adapters are embedded in parallel in the diffusion control layer and diffusion model of the pre-trained nanobody generation model. The LoRA adapters include RL adapters and Old adapters. The Ref network of the pre-trained nanobody generation model can freeze the weights of the original pre-trained model, provide a benchmark for policy constraints, and prevent the model from deviating too far from the original distribution.

[0069] More specifically, in this embodiment, two sets of LoRA adapters are embedded in parallel within the diffusion control layer and the noise prediction network (for the final layer and diffusion layer of the AlphaFold3 type encoder) of the main model using PEFT (parameter efficient fine-tuning).

[0070] For RL adapters ( It is a trainable layer; it uses LoRA default initialization ( The matrix is ​​initialized using a Gaussian distribution (standard deviation 1e-3). (Clear the matrix to zero) to ensure that the increment is 0 at the start of training. For the Old adapter ( ): It is a non-gradient update layer; the parameters are synchronized by the RL adapter through exponential moving average (EMA).

[0071] Step S2: Input the antigen conformation and antibody template, generate candidate antibody samples based on the initialization model, and calculate the scores using the Boltz-2 folding model and PyRosetta software.

[0072] Step S2 above mainly includes the following sub-steps.

[0073] Step S21: Input the target antigen conformation and multiple antibody templates; in this embodiment, the target antigen conformation includes a PDB file and a chain identifier.

[0074] Step S22: Generate candidate antibody samples in batches based on the frozen Ref model and the Old adapter.

[0075] In this embodiment, 1024 candidate antibodies are generated in batches using the BoltzGen model for a given antigen.

[0076] For the specific sampling strategy, 16 antibodies were generated from each of the 64 antibody templates. Atomic coordinates were saved. The complex is encoded for subsequent NFT training.

[0077] Step S23: The Boltz-2 folding model is called to calculate the confidence score, and the affinity energy score is calculated using the dG_separated energy function of PyRosetta. The generated samples are then evaluated and screened for quality.

[0078] More specifically, step S23 mainly includes confidence scoring and affinity scoring.

[0079] For confidence scoring ( Specifically, it calls the Boltz2 built-in confidence function to calculate ipAE (interface position deviation) and pLDDT.

[0080] For the affinity score ( Specifically, the binding energy is calculated using the dG_separated energy function of PyRosetta.

[0081] Step S23 further includes: calculating the affinity ranking and performance ranking of the sample within the batch, and taking the maximum value of the two to obtain... ; for each sample Convert to a linearly normalized reward value between 0 and 1 .

[0082] More specifically, after completing the confidence and affinity scoring, the affinity ranking and execution ranking of the sample within the batch are calculated. The maximum value between the affinity ranking and the execution ranking is then taken to obtain the joint sequence number. .

[0083] Design Basis: Based on the "barrel principle"; if a nanobody fails in any one of its metrics (structure or energy), the entire nanobody will fail. The worst-performing metric is selected as the indicator, forcing the model to consider both performance aspects simultaneously.

[0084] Combined serial number Mapping to linearly normalized reward , It equals 1 minus the first intermediate term, where the first intermediate term is... Subtract 1 from the difference and then divide by Subtract 1 from the difference.

[0085] In this embodiment, The score is 1024. The reward for the smallest (optimal) number is 1, and the reward for the largest number is 0.

[0086] Step S3: Add noise to the 3D coordinates of the selected samples, use the encoder output at the time of generation as a constraint, calculate the denoised 3D coordinates through the RL model and the Old model, calculate the loss function, and update the RL adapter.

[0087] In this embodiment, 32 sets of samples are aggregated according to the template dimension, and the highest reward in each set is taken as the reward value of the template. The 16 sets of templates (a total of 512 samples) with the highest reward values ​​in the top 50% are retained for training.

[0088] Step S3 above includes the following sub-steps.

[0089] Step S31: Randomly set the sampling diffusion step number t, and calculate the original coordinates of the selected samples. Noise was added during the process. .

[0090] In more detail, noise was added to the 512 selected samples.

[0091] Noise addition processing is: random sampling diffusion steps ,right Noise addition was performed according to the diffusion schedule. .

[0092] Step S32: Perform denoising prediction using the RL model and the Old model to obtain the positive prediction coordinates. Compare with negative to predict coordinates The RL model is a Ref network with an RL adapter, and the Old model is a Ref network with an Old adapter.

[0093] For positive prediction coordinates It is based on the input at the current time t Obtained after processing by the RL model; for negative contrast prediction coordinates It is equal to twice the size of the Old model. The output at the current time t is then subtracted from the output of the RL model.

[0094] Step S33, Calculate and The mean squared error between the two values ​​is used to calculate the AWM loss function, which is then used to update the weights of the RL adapter.

[0095] The AWM loss function is equal to the forward predicted coordinates. Compare with negative to predict coordinates The mean square error of the coordinates is multiplied by the reward weight factor 2r-1, so that high reward samples produce positive imitation and low reward samples produce negative avoidance.

[0096] Step S4: When the average reward value in each round reaches the preset condition, output the RL-LoRA weight to generate the antibody.

[0097] Step S4 includes the following more specific process: updating the parameters of the Old adapter according to the exponential moving average formula so that it slowly follows the evolution of the RL adapter; regenerating samples and training in each round until the average reward growth rate of two consecutive rounds is less than a preset threshold or the maximum number of training rounds is reached, and finally outputting the trained RL-LoRA weights to generate antibodies.

[0098] More specifically, after calculating the loss for all 16 samples in a group, for Perform one gradient descent iteration (using the Adam optimizer with an update rate of 5e-4, plus a weight decay of 0.01); after all 512 samples have undergone 10 iterations of training with random noise, proceed as follows: Update the Old adapter; in this embodiment, It is 0.5.

[0099] The average reward value of each round converges (the growth rate is less than a preset threshold for two consecutive rounds, which is 0.5% in this embodiment) or reaches the preset 10 large rounds (Epoch). Each round contains 1024 sampled samples, and each sample is trained with random noise 10 times.

[0100] After training is complete, the RL-LoRA weights are output. When generating antibodies, the RL-LoRA weights are directly loaded into the pre-trained model, and the output according to the standard sampling procedure is the nanobody optimized for the antigen.

[0101] for Figure 1 The initialization process (a) involves adding two LoRA adapters to the diffusion control layer and diffusion model of the model encoder. The overall process (b) involves generating a set of samples from the frozen Old model, training the RL adapter, and finally updating the Old adapter using exponential moving average (EMA). The generation step (c) involves generating samples using the antigen conformation and antibody template in the model with the Old adapter, and calculating scores using the Boltz-2 folding model and PyRosetta software. The training step (d) involves adding noise to the 3D coordinates of the samples, using the encoder output during generation as a constraint, calculating the denoised 3D coordinates using the RL model (original frozen model with RL adapter) and the Old model (original frozen model with Old adapter), comparing them with the original input to calculate the loss function, and updating the RL adapter.

[0102] Based on this, and using a reinforcement learning-based nanobody generation method, this embodiment also proposes a nanobody generation system, including an input module, a model generation module, and an output module. The input module is connected to the model generation module, and the model generation module is connected to the output module. The model generation module generates RL-LoRA weights by adding a LoRA adapter to a pre-trained nanobody generation model and going through the training iteration process of steps S2 to S4 of the reinforcement learning-based nanobody generation method described in Embodiment 1. When generating antibodies, the RL-LoRA weights are directly loaded into the pre-trained nanobody generation model, and the output according to the standard sampling process is the nanobody optimized for the antigen.

Claims

1. A method for generating nanobodies based on reinforcement learning, characterized in that, Includes the following steps: S1, Add a LoRA adapter to the pre-trained nanobody generation model to generate an initialization model; S2, input antigen conformation and antibody template, generate candidate antibody samples based on initialization model, and calculate scores through Boltz-2 folding model and PyRosetta software; S3, add noise to the 3D coordinates of the selected samples, use the encoder output at the time of generation as a constraint, calculate the denoised 3D coordinates through the RL model and the Old model, calculate the loss function, and update the RL adapter; S4, when the average reward value in each round reaches the preset condition, output the RL-LoRA weight to generate the antibody.

2. The method for generating nanobodies based on reinforcement learning according to claim 1, characterized in that, Step S1 includes: Two sets of LoRA adapters, including RL adapters and Old adapters, are embedded in parallel in the diffusion control layer and diffusion model of the pre-trained nanobody generation model.

3. The method for generating nanobodies based on reinforcement learning according to claim 1, characterized in that, Step S2 includes: S21, input the target antigen conformation and multiple antibody templates; S22, Generate candidate antibody samples in batches based on the frozen Ref model and the Old adapter; S23, call the Boltz-2 folding model to calculate the confidence score, and use the PyRosetta dG_separated energy function to calculate the affinity energy score, and perform quality assessment and screening on the generated samples.

4. A method for generating nanobodies based on reinforcement learning according to claim 1, 2, or 3, characterized in that, Step S3 includes: S31, randomly set the sampling diffusion step number t, and analyze the original coordinates of the selected samples. Noise was added during the process. ; S32 uses the RL model and the Old model for denoising prediction to obtain the positive predicted coordinates. Compare with negative to predict coordinates ; S33, Calculation and The mean squared error between the two values ​​is used to calculate the AWM loss function, which is then used to update the weights of the RL adapter.

5. A method for generating nanobodies based on reinforcement learning according to claim 1, 2, or 3, characterized in that, Step S4 includes: The parameters of the Old adapter are updated according to the exponential moving average formula, so that it slowly follows the evolution of the RL adapter; Each round regenerates samples and trains until the average reward growth rate of two consecutive rounds is less than a preset threshold or the maximum number of training rounds is reached. Finally, the trained RL-LoRA weights are output to generate antibodies.

6. The method for generating nanobodies based on reinforcement learning according to claim 3, characterized in that, Step S23 includes: Calculate the affinity ranking and performance ranking of the sample within the batch, and take the maximum of the two. ; Each sample Convert to a linearly normalized reward value between 0 and 1 .

7. The method for generating nanobodies based on reinforcement learning according to claim 3, characterized in that, The target antigen conformation includes a PDB file and a chain identifier.

8. The method for generating nanobodies based on reinforcement learning according to claim 4, characterized in that, The RL model is a Ref network with an RL adapter, and the Old model is a Ref network with an Old adapter.

9. The method for generating nanobodies based on reinforcement learning according to claim 5, characterized in that, When generating antibodies, the RL-LoRA weights are directly loaded into the pre-trained nanobody generation model to output nanobodies optimized for the antigen.

10. A method for generating nanobodies based on reinforcement learning according to claim 9, characterized in that, The pre-trained nanobody generation model is the BoltzGen model.

Images