A method and system for generating peptide sequences of controllable length based on masking mechanism

Abstract

This invention relates to a method and system for generating peptide sequences with controllable length based on a masking mechanism. The method includes: adding start characters, end characters, and padding characters to a peptide sequence dataset and inputting it into a protein language ESM2 model to extract the latent space embeddings of the peptide sequences; adding noise based on a diffusion model and training a denoiser using a masking mechanism to predict the un-noised latent space embeddings; adding noise to the latent space embeddings and training a guidance module using a masking mechanism to predict the category of the noisy latent space embeddings; calculating the noisy data and decoding the noisy data to obtain the peptide sequence. Because the denoiser uses a masking mechanism, combining the advantages of the protein language model and the diffusion model, it not only allows control over the length of the generated sequence but also enables the diffusion model to better fit the characteristics of the protein language model, thus allowing the diffusion model to generate latent space embeddings that are more consistent with the protein language model.

Description

Technical Field

[0001] This invention relates to the field of polypeptide sequence generation technology, and in particular to a method and system for generating polypeptide sequences of controllable length based on a masking mechanism. Background Technology

[0002] Therapeutic peptides, such as antimicrobial peptides, antifungal peptides, and antiviral peptides, are a unique class of drugs composed of short amino acid chains that hold immense potential for treating complex human diseases. These short peptides are characterized by their compact structure and high adaptability, and promise to revolutionize treatment interventions for diseases caused by bacteria, fungi, parasites, and viruses.

[0003] However, current engineering design models for these peptides are primarily based on high-throughput screening and rational design, aiming to improve in vivo stability, solubility, and strain specificity while reducing aggregation. While the flexibility of peptides is beneficial for clinical research, it complicates design because traditional structure-based methods often cannot adequately handle the dynamic and conformationally unstable properties of these molecules; furthermore, the combinatorial space for these peptides is vast, with only a small fraction of solutions meeting clinical needs. Therefore, this near-exhaustive screening approach can be both time-consuming and expensive.

[0004] Currently, there are various techniques for peptide sequence generation. For example, autoregressive methods depict peptide sequences as sentences composed of amino acid markers, allowing the problem to be solved by predicting the amino acid sequence using recurrent neural networks (RNNs); variational autoencoder (VAE) methods sample from a latent space learned through an encoder-decoder architecture to generate new peptide sequences, with therapeutic properties as either a conditional constraint or not; and generative adversarial network (GAN) methods train generators and discriminators using known data, which learn the data distribution in a competitive manner, enabling the generator to produce new peptides that closely resemble the real peptide distribution.

[0005] The length of a peptide sequence has a significant impact on the efficacy and physicochemical properties of peptides. Most existing peptide generation models cannot control the length of the peptide sequence they generate; the length of the generated sequence is entirely determined by the generation model, and the model cannot intervene in the length of the generated sequence from the beginning of the generation process. Summary of the Invention

[0006] In order to solve the above-mentioned technical problems, a method and system for generating peptide sequences with controllable length based on a masking mechanism is provided, which can control the length of the generated peptide sequences.

[0007] A method for generating polypeptide sequences of controllable length based on a masking mechanism, the method comprising:

[0008] Peptide sequences are collected from a peptide database, and after data processing, the peptide sequences are divided into various peptide sequence datasets.

[0009] The polypeptide sequence dataset is encoded using a vectorized text tool, and start characters, end characters, and padding characters are added. The polypeptide sequence dataset with added characters is then input into the ESM2 model to extract the latent space embeddings of the polypeptide sequences.

[0010] The latent space embedding is denoised, and a denoiser is trained based on the denoised latent space embedding combined with a masking mechanism. The denoiser is used to predict the undenoised latent space embedding, and the mean square error loss between the predicted undenoised latent space embedding and the actual undenoised latent space embedding is calculated.

[0011] The latent space embedding is noise-added, and a guiding module is trained based on the noise-added latent space embedding combined with a masking mechanism. The class of the noisy latent space embedding is predicted through the guiding module, and the cross-entropy loss between the predicted class and the actual class is calculated.

[0012] Based on the mean squared error loss and the cross-entropy loss, noise-free data is obtained, and the noise-free data is decoded to obtain the peptide sequence.

[0013] In one embodiment, peptide sequences are collected from a peptide database, and after data processing, the peptide sequences are divided into various peptide sequence datasets, including:

[0014] Collect peptide sequences labeled with antibacterial, antifungal, and antiviral functions from peptide databases;

[0015] Duplicate peptide sequences were removed from the collected peptide sequences, and the remaining peptide sequences were classified to obtain individual peptide sequence datasets.

[0016] The individual polypeptide sequence datasets are: antimicrobial peptide dataset, antifungal peptide dataset, antiviral peptide dataset, and polypeptide mixed dataset.

[0017] In one embodiment, the polypeptide sequence dataset is encoded using a vectorized text tool, and start characters, end characters, and padding characters are added, including:

[0018] The polypeptide sequence dataset is encoded using a vectorized text tool, and the formatted index vector and attention mask vector are output.

[0019] A start character is added at the beginning of each polypeptide sequence dataset, an end character is added at the end, and padding characters are added at the end when the polypeptide sequence is less than the target number of characters.

[0020] The method further includes:

[0021] The index vector and attention mask vector are input into the ESM2 model to obtain the latent space embedding of the polypeptide sequence.

[0022] In one embodiment, the latent space embedding is denoised, a denoiser is trained based on the denoised latent space embedding combined with a masking mechanism, and the denoiser is used to predict the undenoised latent space embedding, including:

[0023] Gaussian noise is gradually added to the latent space embedding until the latent space embedding is completely transformed into all Gaussian noise, thus completing the noise addition process and obtaining the noisy latent space embedding.

[0024] The noisy latent space embedding, time step, and attention mask vector are input into the denoiser. The denoiser uses LayerNorm to normalize the noisy latent space embedding and transforms the attention mask vector into a tensor.

[0025] Based on the normalized latent space embedding and the tensor prediction unnoised latent space embedding.

[0026] In one embodiment, Gaussian noise is progressively added to the latent space embedding until the latent space embedding becomes completely Gaussian noise, including:

[0027] Based on the latent space embedding, the latent space embedding distribution is determined, the intensity data of the added noise is determined, and the corresponding mean and variance are obtained.

[0028] Based on the latent space embedding, the latent space embedding distribution, and the intensity data, Gaussian noise is gradually increased by the mean and variance control.

[0029] In one embodiment, the denoiser uses LayerNorm to normalize the noisy latent space embedding, transforming the attention mask vector into a tensor, including:

[0030] Based on the denoiser, the time step is encoded into a continuous variable using cosine position coding. After encoding, it is input into a multilayer perceptron to obtain the processed time step.

[0031] LayerNorm is used to normalize the noisy latent space embedding to obtain normalized noisy data;

[0032] The processed time step is then input into a multilayer perceptron to output the target time step;

[0033] The target time step and the normalized noisy data are scaled and shifted, and the output tensor is obtained by combining the attentionmask vector with the ESM2 model.

[0034] In one embodiment, the method further includes;

[0035] Determine the number of times to repeat the operation;

[0036] The output tensor is used as the noisy data, and scaling and shifting operations are repeatedly performed until the number of repetitions is reached.

[0037] In one embodiment, predicting the noisy latent space embedding via the denoiser includes:

[0038] The output tensor is added to the normalized noisy data, and after being normalized using LayerNorm, it is input into a multilayer perceptron to output the predicted noisy latent space embedding.

[0039] In one embodiment, the method further includes:

[0040] The noisy latent space embedding and the corresponding time step are processed by the guidance module to obtain the probability of the category to which the noisy latent space embedding belongs;

[0041] The gradient of the classifier in the guiding template is calculated, and the logarithm of the gradient is taken. Based on the logarithm, the mean and variance are modified, and after reparameterization, a noise-free latent space embedding is obtained.

[0042] A peptide sequence generation system with controllable length based on a masking mechanism, the system comprising:

[0043] The dataset collection module is used to collect polypeptide sequences from a polypeptide database and, after data processing, divide the polypeptide sequences into various polypeptide sequence datasets.

[0044] The extraction module is used to encode the polypeptide sequence dataset using a vectorized text tool and add start characters, end characters, and padding characters; the polypeptide sequence dataset with added characters is then input into the ESM2 model to extract the latent space embeddings of the polypeptide sequences.

[0045] The noise-added training denoiser module is used to add noise to the latent space embedding, train the denoiser based on the noise-added latent space embedding combined with the masking mechanism, predict the unnoised latent space embedding through the denoiser, and calculate the mean square error loss between the predicted unnoised latent space embedding and the actual unnoised latent space embedding.

[0046] A noisy training guide template is used to add noise to the latent space embedding. The guide module is trained based on the noisy latent space embedding combined with a masking mechanism. The guide module is used to predict the category of the noisy latent space embedding and calculate the cross-entropy loss between the predicted category and the actual category.

[0047] The peptide sequence sampling module is used to obtain noise-free data based on the mean square error loss and the cross-entropy loss, and to decode the noise-free data to obtain the peptide sequence.

[0048] The above-mentioned method and system for generating polypeptide sequences with controllable length based on masking mechanism combines the advantages of protein language model and diffusion model by using masking mechanism in the denoiser. Masking mechanism is also used in protein language model, which not only controls the length of the generated sequence, but also makes diffusion model fit the characteristics of protein language model well, so that diffusion model generates latent space embeddings that are more in line with protein language model. Attached Figure Description

[0049] Figure 1 This is an application environment diagram of a peptide sequence generation method with controllable length based on a masking mechanism in one embodiment;

[0050] Figure 2 This is a flowchart illustrating a method for generating polypeptide sequences of controllable length based on a masking mechanism in one embodiment.

[0051] Figure 3 This is a schematic diagram of the latent space embedding extraction process of the protein sequence in the ESM2 model in one embodiment;

[0052] Figure 4 This is a schematic diagram of the denoiser architecture in one embodiment;

[0053] Figure 5 for Figure 4 A schematic diagram of the noise reduction block architecture in the structure;

[0054] Figure 6 This is a schematic diagram illustrating the general process of adding and removing noise in a diffusion model in one embodiment;

[0055] Figure 7 This is a schematic diagram of the process of the guidance module in the sampling phase in one embodiment;

[0056] Figure 8 This is a schematic diagram of the peptide sequence sampling process in one embodiment;

[0057] Figure 9 This is a structural block diagram of a peptide sequence generation system with controllable length based on a masking mechanism in one embodiment;

[0058] Figure 10This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0059] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0060] The method for generating polypeptide sequences with controllable length based on a masking mechanism provided in this application can be applied to, for example... Figure 1 The application environment shown. For example... Figure 1 As shown, the application environment includes computer device 110. Computer device 110 can collect peptide sequences from a peptide database, process the peptide sequences, and divide them into various peptide sequence datasets. Computer device 110 can encode the peptide sequence datasets using a vectorized text tool, adding start characters, end characters, and padding characters. The peptide sequence datasets with added characters are input into the protein language ESM2 model to extract the latent space embeddings of the peptide sequences. Computer device 110 can add noise to the latent space embeddings based on a diffusion model, train a denoiser based on the noisy latent space embeddings combined with a masking mechanism, and predict the unnoisy latent space embeddings using the denoiser, calculating the mean squared error loss between the predicted and actual unnoisy latent space embeddings. Computer device 110 can add noise to the latent space embeddings, train a guidance module based on the noisy latent space embeddings combined with a masking mechanism, and predict the category of the noisy latent space embeddings using the guidance module, calculating the cross-entropy loss between the predicted and actual categories. Computer device 110 can obtain noisy data based on the mean squared error loss and cross-entropy loss, and decode the noisy data to obtain the peptide sequences. The computer equipment 110 may include, but is not limited to, various personal computers, laptops, smartphones, robots, drones, tablets, and portable wearable devices.

[0061] In one embodiment, such as Figure 2 As shown, a method for generating peptide sequences of controllable length based on a masking mechanism is provided, including the following steps:

[0062] Step 202: Collect peptide sequences from the peptide database, and divide the peptide sequences into various peptide sequence datasets after data processing.

[0063] The peptide database may include CAMPR4, ADP, DBAASP, dbAMP2, DRAMP, SATPDB, LAMP2, and GRAMPA.

[0064] In one embodiment, a method for generating peptide sequences with controllable length based on a masking mechanism may further include a process of collecting peptide sequences and dividing them into datasets. The specific process includes: collecting peptide sequences labeled with antibacterial, antifungal, and antiviral functions from a peptide database; deleting duplicate peptide sequences from the collected peptide sequences and classifying the remaining peptide sequences to obtain various peptide sequence datasets; wherein, each peptide sequence dataset is an antibacterial peptide dataset, an antifungal peptide dataset, an antiviral peptide dataset, and a mixed peptide dataset.

[0065] Specifically, the computer device can collect peptide sequences labeled with antibacterial, antifungal, and antiviral functions from peptide databases CAMPR4, ADP, DBAASP, dbAMP2, DRAMP, SATPDB, LAMP2, and GRAMPA, and remove duplicate sequences. Then, it divides the sequences into three categories: antimicrobial peptides, antifungal peptides, and antiviral peptides, constructing four peptide datasets. The first dataset contains a mixture of the three peptides and is used to train the denoiser. The second, third, and fourth datasets contain antimicrobial peptides, antifungal peptides, and antiviral peptides, respectively, and are used to train the bootstrapping module. The peptide sequences in each dataset are composed of twenty common amino acids, represented by the letters A, C, D, E, F, G, H, I, K, L, M, N, P, Q, R, S, T, V, W, and Y.

[0066] Step 204: Encode the polypeptide sequence dataset using a vectorized text tool and add start characters, end characters, and padding characters; input the polypeptide sequence dataset with added characters into the protein language ESM2 model to extract the latent space embeddings of the polypeptide sequences.

[0067] Computer devices can use the protein language model ESM2 to encode polypeptide sequence datasets to obtain the corresponding latent space embeddings.

[0068] In one embodiment, a method for generating peptide sequences with controllable length based on a masking mechanism may further include a process for extracting the latent space embedding of the peptide sequences. This process includes: encoding the peptide sequence dataset using a vectorized text tool to output the formatted index vector and attention mask vector; adding a start character at the beginning position and an end character at the end of each peptide sequence dataset, and adding padding characters at the end when the peptide sequence is shorter than the target number of characters; and inputting the index vector and attention mask vector into the ESM2 model to obtain the latent space embedding of the peptide sequences. The process of extracting the latent space embedding of the peptide sequences is as follows: Figure 3 As shown.

[0069] Specifically, computer devices can encode peptide sequence datasets using the vectorized text tool Tokenizer, adding a CLS start character at the beginning and an EOS end character at the end, padding the end with PAD characters when the peptide sequence is less than 50 characters.

[0070] After encoding, the output format encoding includes input_ids, which represents the index of each amino acid in the sequence in the vocabulary; attention mask, which indicates whether the current position corresponds to a valid character, is 1 if valid and 0 otherwise; after encoding, a 52-dimensional index vector and a 52-dimensional attention mask vector can be obtained; the obtained index vector and attention mask are input into the pre-trained protein language model ESM2 model, and finally a 52*320 tensor is obtained, which is the latent space embedding of the polypeptide sequence.

[0071] Step 206: Add noise to the latent space embedding based on the diffusion model, train a denoiser based on the noisy latent space embedding combined with a masking mechanism, predict the noisy latent space embedding through the denoiser, and calculate the mean square error loss between the predicted noisy latent space embedding and the actual noisy latent space embedding.

[0072] The obtained latent space embedding is noise-added, and the corresponding mask is used to train the denoiser. The block diagram of the denoiser is shown below. Figure 4 As shown, the block diagram of the noise reduction block in the noise reduction unit is as follows. Figure 5 As shown, the denoiser directly predicts the undenoised latent space embedding, and the optimization objective is the mean squared error loss between the predicted undenoised latent space embedding and the true undenoised latent space embedding.

[0073] In this embodiment, the computer device can add noise to the obtained ESM2 latent space embedding, the generative model learns the noise addition process and performs denoising, and uses the backpropagation algorithm to update the model parameters until the training termination condition is met, thus obtaining a trained generative model.

[0074] In one embodiment, a method for generating controllable-length peptide sequences based on a masking mechanism may further include adding noise to the obtained latent space embedding of the peptide sequence and training a denoiser using a corresponding mask. Specifically, this process includes: progressively adding Gaussian noise to the latent space embedding until it becomes entirely Gaussian noise, thus completing the noise addition process and obtaining the noisy latent space embedding; inputting the noisy latent space embedding, the time step, and the attention mask vector into the denoiser; normalizing the noisy latent space embedding using LayerNorm through the denoiser, transforming the attention mask vector into a tensor; and predicting the unnoisy latent space embedding based on the normalized latent space embedding and the tensor.

[0075] Specifically, in one embodiment, the noise addition process may include: determining the latent space embedding distribution based on the latent space embedding, determining the intensity data of the noise to be added, and obtaining the corresponding mean and variance; and gradually increasing Gaussian noise based on the latent space embedding, the latent space embedding distribution, and the intensity data, controlled by the mean and variance. The general process for adding and removing noise in the diffusion model is as follows: Figure 6 As shown.

[0076] The noise addition process, also known as the diffusion process, refers to the gradual addition of Gaussian noise to the original data until the original data is completely transformed into Gaussian noise. Here, x0 represents the ESM2 latent space embedding, and q(x0) represents the ESM2 latent space embedding distribution; the diffusion process uses q(x0)... t |x t-1 This represents the process of gradually adding noise, which is determined by the mean μ and variance σ. 2 If controlled, the Markov chain of the diffusion process is represented as: The diffusion process is represented as:

[0077] One important feature of the diffusion model is that its state at any time t can be directly obtained from x0: in The intensity of the added noise is indicated by the following formula:

[0078] The denoising process is also known as the reverse diffusion process, denoted as p(x t-1 |x t First, a Gaussian noise sample is randomly taken at time T. A denoising model is then used to progressively remove the noise, ultimately yielding data x0 that matches the original true data distribution. The reverse diffusion process can be represented as: Where μ θ and Σ θ This represents the mean and variance of the predictions.

[0079] In one embodiment, the process of using a denoiser may include: based on the denoiser, encoding the time step into a continuous variable using cosine position coding, and then inputting the encoded data into a multilayer perceptron to obtain the processed time step; using LayerNorm to normalize the noisy latent space embedding to obtain normalized noisy data; inputting the processed time step into another multilayer perceptron to output the target time step; scaling and shifting the target time step and the normalized noisy data, and then using the attention mask vector based on the ESM2 model to obtain the output tensor.

[0080] The denoising process follows the DDPM method, first dividing the noisy data x t The time step t and the attention mask vector are input into the denoiser to obtain the original predicted data. Then based on the raw data for prediction To calculate the mean and variance at time t-1: Where β t =1-α t ,and Therefore, it can be used. To deduce β t After calculating the mean and variance at time t-1, x is obtained using the formula. t-1 :

[0081] The denoiser receives three parameters: the noisy data x t The time step is t, and the attention mask vector is denoted as PE. In this embodiment, cosine position encoding can be used to encode the time step t as a continuous variable, and after encoding, it is input into a multilayer perceptron. The cosine position encoding formula is: PE (pos,2i) =sin(pos / 10000) 2i / 640 ); PE (pos,2i+1) =cos(pos / 10000) 2i / 640 ); where PE (pos,2i) This represents the value of the i-th dimension at the pos-th position in the position coding matrix; i represents the value of the i-th dimension in the position coding matrix.

[0082] Next, LayerNorm can be used to normalize the noisy data, and then the attention mask vector can be transformed into a tensor of (1,52,52).

[0083] In one embodiment, a method for generating a peptide sequence of controllable length based on a masking mechanism may further include a process of repeated scaling and shifting, specifically including: determining the number of repetitions; treating the output tensor as noisy data and repeatedly performing scaling and shifting operations until the number of repetitions is reached.

[0084] The encoded time step, processed by the multilayer perceptron, is then input into another multilayer perceptron. The output is scaled and shifted with the normalized noisy data, and then processed with the deformed attention mask tensor into the EMS2AttentionLayer. The output is a 52*320 tensor, which is then used to replace the aforementioned noisy data. This process is repeated a specified number of times.

[0085] In one embodiment, a method for generating a peptide sequence of controllable length based on a masking mechanism may further include: adding the output tensor to the normalized noisy data, normalizing it using LayerNorm, and then inputting it into a multilayer perceptron to output the predicted noisy latent space embedding.

[0086] The obtained tensor is added to the normalized noisy data and normalized using LayerNorm. Finally, it is fed into a multilayer perceptron, and the output is the predicted noise-free original data. The model is based on noisy data x t Directly predict the raw data Loss selection prediction raw data The L2 loss between the actual original data x0 and the real original data is expressed by the formula: Where L(θ) represents the optimization objective. M represents the denoiser, and M represents the attentionmask tensor. This represents the square of the L2 norm.

[0087] Step 208: Add noise to the latent space embedding, train the guiding module based on the noisy latent space embedding and the masking mechanism, and predict the category of the noisy latent space embedding through the guiding module, and calculate the cross-entropy loss between the predicted category and the actual category.

[0088] The obtained latent space embeddings are noisy and combined with the corresponding mask to guide the module. The guiding module predicts the class of the noisy latent space embedding, and the optimization objective is the cross-entropy loss between the predicted class and the true class.

[0089] Specifically, in one embodiment, a method for generating a peptide sequence of controllable length based on a masking mechanism may further include: processing the noisy latent space embedding and the corresponding time step through a guiding module to obtain the probability of the noisy latent space embedding belonging to a category; calculating the gradient of the classifier in the guiding template, taking the logarithm of the gradient, modifying the mean and variance based on the logarithm, and reparameterizing to obtain a noise-free latent space embedding.

[0090] The guidance module includes a guidance classifier and a mean calculation section. The guidance classifier is used to classify the ESM2 latent space embeddings of noisy peptide sequences. The mean calculation section uses the gradient of the guidance classifier to modify the mean at time t-1, thereby guiding the denoising process in the desired direction. The process of the guidance module in the sampling phase is as follows: Figure 7 As shown.

[0091] The model structure of the classifier is basically the same as that of the denoiser, but it needs to be modified. Specifically, the input of the last multilayer perceptron is (52*320, 640); the output of the last multilayer perceptron is a 2-dimensional vector, and the probability of the two classes is calculated using the Softmax activation function.

[0092] During the denoising process, after calculating the mean and variance at time t-1, the formula used to update the mean at time t-1 is as follows: Where 's' controls the guiding strength. Indicates x t Find the derivative, G φ G represents the guiding classifier. φ (y|x t ) represents x obtained by the guiding classifier t Let y be the probability of category y.

[0093] Step 210: Obtain noise-free data based on mean square error loss and cross-entropy loss, and decode the noise-free data to obtain the peptide sequence.

[0094] Specifically, such as Figure 8 As shown, when sampling peptide sequences, one type of peptide can be selected from antimicrobial peptides, antifungal peptides, and antiviral peptides, and the length of the sequence to be generated can be specified. These two targets are input into the generation model, which includes a noise acquisition module, a denoising module, a guidance module, and a decoding module. Specifically, the noise acquisition module samples noise points from a Gaussian distribution, which serve as the initial input to the denoising module; the denoising module inputs the time step t of the current noisy data and the noisy data x... t The data is processed to obtain the predicted, noise-free raw data. Then according to To calculate the mean and variance at time t-1; the guidance module is used to set the time step t of the current noisy data and the noisy data x. t After processing, we obtain x. t The probability of the class is calculated, then the gradient of the classifier is obtained based on the probability of the specified peptide type, and the logarithm is taken. This result is then used to modify the mean at time t-1. Finally, the modified mean and variance are used for reparameterization to obtain x. t-1 Repeat the above steps to obtain x0; the decoding module is used to decode the denoised x0 to obtain a new polypeptide sequence.

[0095] This application combines the advantages of protein language models and diffusion models. The denoising mechanism uses a masking mechanism, which is also used in protein language models. This not only controls the length of the generated sequences but also allows the diffusion model to better fit the characteristics of protein language models, thus enabling the diffusion model to generate latent embeddings that are more consistent with the latent space of protein language models. Furthermore, the training time for the guided classifier is significantly shorter than that for the diffusion model. The training time required to guide the diffusion model to generate a model of a specified type of peptide using the guided module is much shorter than the time required to train multiple generative models individually, saving time and costs.

[0096] It should be understood that although the steps in the flowchart above are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowchart above may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0097] In one embodiment, such as Figure 9 As shown, a peptide sequence generation system with controllable length based on a masking mechanism is provided, including: a dataset collection module 910, an extraction module 920, a noise-adding training denoising module 930, a noise-adding training guiding template 940, and a peptide sequence sampling module 950, wherein:

[0098] The dataset collection module 910 is used to collect polypeptide sequences from the polypeptide database and divide the polypeptide sequences into various polypeptide sequence datasets after data processing.

[0099] The extraction module 920 is used to encode the polypeptide sequence dataset using a vectorized text tool and add start characters, end characters, and padding characters; the polypeptide sequence dataset with added characters is then input into the ESM2 model to extract the latent space embeddings of the polypeptide sequences.

[0100] The noise-adding training denoiser module 930 is used to add noise to the latent space embedding based on the diffusion model, train the denoiser based on the noise-adding latent space embedding combined with the masking mechanism, predict the unnoised latent space embedding through the denoiser, and calculate the mean square error loss between the predicted unnoised latent space embedding and the actual unnoised latent space embedding.

[0101] Noisy training guide template 940 is used to add noise to the latent space embedding. The guide module is trained based on the noisy latent space embedding combined with the masking mechanism. The guide module predicts the category of the noisy latent space embedding and calculates the cross-entropy loss between the predicted category and the actual category.

[0102] The peptide sequence sampling module 950 is used to obtain noise-free data based on mean square error loss and cross-entropy loss, and then decode the noise-free data to obtain the peptide sequence.

[0103] In one embodiment, the dataset collection module 910 is further configured to collect peptide sequences labeled with antibacterial, antifungal, and antiviral functions from a peptide database; delete duplicate peptide sequences in the collected peptide sequences, and classify the remaining peptide sequences to obtain various peptide sequence datasets; wherein, the various peptide sequence datasets are respectively an antibacterial peptide dataset, an antifungal peptide dataset, an antiviral peptide dataset, and a peptide mixed dataset.

[0104] In one embodiment, the extraction module 920 is further configured to encode the polypeptide sequence dataset using a vectorized text tool, outputting the format encoding of the index vector and attention mask vector; add a start character at the beginning position and an end character at the end position of each polypeptide sequence dataset, and add padding characters at the end when the polypeptide sequence is less than the target number of characters; input the index vector and attention mask vector into the ESM2 model to obtain the latent space embedding of the polypeptide sequence.

[0105] In one embodiment, the noise-adding training denoiser module 930 is further configured to progressively add Gaussian noise to the latent space embedding until the latent space embedding is completely transformed into full Gaussian noise, thus completing the noise-adding process and obtaining the noise-adding latent space embedding; the noise-adding latent space embedding, time step, and attention mask vector are input into the denoiser, and the denoiser uses LayerNorm to normalize the noise-adding latent space embedding, transforming the attention mask vector into a tensor; the unnoised latent space embedding is predicted based on the normalized latent space embedding and the tensor.

[0106] In one embodiment, the noise-training denoiser module 930 is further configured to determine the latent space embedding distribution based on the latent space embedding, determine the intensity data of the added noise, and obtain the corresponding mean and variance; and gradually increase Gaussian noise based on the latent space embedding, the latent space embedding distribution, and the intensity data, controlled by the mean and variance.

[0107] In one embodiment, the noise-added training denoiser module 930 is further configured to encode the time step into a continuous variable using cosine position coding based on the denoiser, and input the encoded data into a multilayer perceptron to obtain the processed time step; use LayerNorm to normalize the latent space embedding after noise addition to obtain normalized noisy data; input the processed time step into another multilayer perceptron to output the target time step; and perform scaling and shifting operations on the target time step and the normalized noisy data, and obtain the output tensor based on the ESM2 model with the attention mask vector.

[0108] In one embodiment, the noise-adding training denoiser module 930 is further configured to determine the number of repetitions; and to repeatedly perform scaling and shifting operations on the output tensor as noisy data until the number of repetitions is reached.

[0109] In one embodiment, the noise-added training denoiser module 930 is further configured to add the output tensor to the normalized noisy data, normalize it using LayerNorm, and then input it into the multilayer perceptron to output the predicted noisy latent space embedding.

[0110] In one embodiment, the noisy training guidance template 940 is further used to process the noisy latent space embedding and the corresponding time step through the guidance module to obtain the probability of the noisy latent space embedding to the category; calculate the gradient of the classifier in the guidance template, take the logarithm of the gradient, modify the mean and variance based on the logarithm, and perform reparameterization to obtain the noise-free latent space embedding.

[0111] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 10 As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage medium. The network interface is used to communicate with external terminals via a network connection. When executed by the processor, the computer program implements a method for generating polypeptide sequences with controllable length based on a masking mechanism. The display screen can be a liquid crystal display (LCD) or an e-ink display. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0112] Those skilled in the art will understand that Figure 10The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0113] In one embodiment, a computer device is provided, including a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the steps of a method for generating peptide sequences of controllable length based on a masking mechanism.

[0114] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of a method for generating polypeptide sequences of controllable length based on a masking mechanism.

[0115] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0116] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0117] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method for generating polypeptide sequences of controllable length based on a masking mechanism, characterized in that, The method includes: Peptide sequences are collected from a peptide database, and after data processing, the peptide sequences are divided into various peptide sequence datasets. The polypeptide sequence dataset is encoded using a vectorized text tool, and start characters, end characters, and padding characters are added. The polypeptide sequence dataset with added characters is then input into the protein language ESM2 model to extract the latent space embeddings of the polypeptide sequences. The latent space embedding is denoised based on the diffusion model. A denoiser is trained based on the denoised latent space embedding combined with a masking mechanism. The denoiser is used to predict the undenoised latent space embedding. The mean square error loss between the predicted undenoised latent space embedding and the actual undenoised latent space embedding is calculated. The latent space embedding is noise-added, and a guiding module is trained based on the noise-added latent space embedding combined with a masking mechanism. The class of the noisy latent space embedding is predicted through the guiding module, and the cross-entropy loss between the predicted class and the actual class is calculated. Based on the mean squared error loss and the cross-entropy loss, noise-free data is obtained, and the noise-free data is decoded to obtain the peptide sequence.

2. The method for generating polypeptide sequences with controllable length based on a masking mechanism according to claim 1, characterized in that, Peptide sequences were collected from a peptide database, and after data processing, the peptide sequences were divided into various peptide sequence datasets, including: Collect peptide sequences labeled with antibacterial, antifungal, and antiviral functions from peptide databases; Duplicate peptide sequences were removed from the collected peptide sequences, and the remaining peptide sequences were classified to obtain individual peptide sequence datasets. The individual polypeptide sequence datasets are: antimicrobial peptide dataset, antifungal peptide dataset, antiviral peptide dataset, and polypeptide mixed dataset.

3. The method for generating polypeptide sequences of controllable length based on a masking mechanism according to claim 1, characterized in that, The polypeptide sequence dataset was encoded using a vectorized text tool, and start characters, end characters, and padding characters were added, including: The polypeptide sequence dataset is encoded using a vectorized text tool, and the formatted index vector and attentionmask vector are output. A start character is added at the beginning of each polypeptide sequence dataset, an end character is added at the end, and padding characters are added at the end when the polypeptide sequence is less than the target number of characters. The method further includes: The index vector and attention mask vector are input into the ESM2 model to obtain the latent space embedding of the polypeptide sequence.

4. The method for generating polypeptide sequences of controllable length based on a masking mechanism according to claim 3, characterized in that, The latent space embedding is denoised based on a diffusion model. A denoiser is trained based on the denoised latent space embedding combined with a masking mechanism. The denoiser is then used to predict the undenoised latent space embedding, including: Gaussian noise is gradually added to the latent space embedding until the latent space embedding is completely transformed into all Gaussian noise, thus completing the noise addition process and obtaining the noisy latent space embedding. The noisy latent space embedding, time step, and attention mask vector are input into the denoiser. The denoiser uses LayerNorm to normalize the noisy latent space embedding and transforms the attention mask vector into a tensor. Based on the normalized latent space embedding and the tensor prediction unnoised latent space embedding.

5. The method for generating polypeptide sequences of controllable length based on a masking mechanism according to claim 4, characterized in that, Gradually increase Gaussian noise into the latent space embedding until the latent space embedding becomes completely Gaussian noise, including: Based on the latent space embedding, the latent space embedding distribution is determined, the intensity data of the added noise is determined, and the corresponding mean and variance are obtained. Based on the latent space embedding, the latent space embedding distribution, and the intensity data, Gaussian noise is gradually increased by the mean and variance control.

6. The method for generating polypeptide sequences with controllable length based on a masking mechanism according to claim 4, characterized in that, The denoiser uses LayerNorm to normalize the noisy latent space embedding, transforming the attention mask vector into a tensor, including: Based on the denoiser, the time step is encoded into a continuous variable using cosine position coding. After encoding, it is input into a multilayer perceptron to obtain the processed time step. LayerNorm is used to normalize the noisy latent space embedding to obtain normalized noisy data; The processed time step is then input into a multilayer perceptron to output the target time step; The target time step and the normalized noisy data are scaled and shifted, and the output tensor is obtained by combining the attentionmask vector with the ESM2 model.

7. The method for generating polypeptide sequences of controllable length based on a masking mechanism according to claim 6, characterized in that, The method further includes; Determine the number of times to repeat the operation; The output tensor is used as the noisy data, and scaling and shifting operations are repeatedly performed until the number of repetitions is reached.

8. The method for generating polypeptide sequences of controllable length based on a masking mechanism according to claim 6, characterized in that, Predicting the noisy latent space embedding using the denoiser includes: The output tensor is added to the normalized noisy data, and after being normalized using LayerNorm, it is input into a multilayer perceptron to output the predicted noisy latent space embedding.

9. A peptide sequence generation system with controllable length based on a masking mechanism, characterized in that, The system includes: The dataset collection module is used to collect polypeptide sequences from a polypeptide database and, after data processing, divide the polypeptide sequences into various polypeptide sequence datasets. The extraction module is used to encode the polypeptide sequence dataset using a vectorized text tool and add start characters, end characters, and padding characters; the polypeptide sequence dataset with added characters is then input into the ESM2 model to extract the latent space embeddings of the polypeptide sequences. The noise-added training denoiser module is used to add noise to the latent space embedding based on the diffusion model, train the denoiser based on the noise-added latent space embedding combined with the masking mechanism, predict the unnoised latent space embedding through the denoiser, and calculate the mean square error loss between the predicted unnoised latent space embedding and the actual unnoised latent space embedding. A noisy training guide template is used to add noise to the latent space embedding. The guide module is trained based on the noisy latent space embedding combined with a masking mechanism. The guide module is used to predict the category of the noisy latent space embedding and calculate the cross-entropy loss between the predicted category and the actual category. The peptide sequence sampling module is used to obtain noise-free data based on the mean square error loss and the cross-entropy loss, and to decode the noise-free data to obtain the peptide sequence.

Images