Mask mechanism-based length-controllable polypeptide sequence generation method and system

Abstract

The invention relates to a length-controllable polypeptide sequence generation method and system based on a mask mechanism. The method comprises the following steps: adding a start character, an end character and a filling character into a polypeptide sequence data set, inputting the data set into a protein language ESM2 model, and extracting polypeptide sequence hidden space embedding; noise adding is carried out based on a diffusion model, and a de-noising device is trained in combination with a mask mechanism to predict non-noise-added submerged space embedding; a noise adding and mask mechanism training guide module is carried out on the submerged space embedding, and the category of the noise-containing submerged space embedding is predicted; and calculating to obtain noise-free data, and decoding the noise-free data to obtain a polypeptide sequence. Due to the fact that a mask mechanism is used in the de-noising device, the advantages of the protein language model and the diffusion model are combined, the length of the generated sequence can be controlled, the diffusion model can well conform to the characteristics of the protein language model, and therefore the diffusion model can generate submerged space embedding better conforming to the protein language model.

Description

Technical Field

[0001] The present invention relates to the technical field of polypeptide sequence generation, and in particular to a method and system for generating a polypeptide sequence with controllable length based on a mask mechanism. Background Art

[0002] Therapeutic peptides such as antimicrobial peptides, antifungal peptides and antiviral peptides are a unique class of drugs composed of short amino acid chains that have great potential in treating complex human diseases. These short peptides are characterized by compact structure and strong adaptability, and are expected to revolutionize the therapeutic intervention of diseases caused by bacteria, fungi, parasites and viruses.

[0003] However, the current engineering paradigm for these peptides is based on high-throughput screening and rational design, aiming to improve in vivo stability, solubility, and strain specificity while reducing aggregation. The flexibility of peptides, while beneficial for clinical research, complicates design because traditional structure-based approaches often do not handle the dynamic, conformationally unstable nature of these molecules well; and the combinatorial space of these peptides is vast, with only a small fraction of solutions meeting clinical needs. Therefore, this near-exhaustive screening approach can be time-consuming and expensive.

[0004] There are currently a variety of techniques for peptide sequence generation. For example, the autoregression-based method depicts the peptide sequence as a sentence composed of amino acid labels, so that the problem can be solved by predicting the amino acid arrangement through a recurrent neural network (RNN); the variational autoencoder (VAE)-based method samples from the latent space learned through the encoder-decoder architecture to generate new peptide sequences, and uses therapeutic properties as conditional constraints or not; the generative adversarial network (GAN)-based method uses known data to train the generator and discriminator, which use a competitive approach to learn the data distribution, so that the generator generates new peptides that are close to the real peptide distribution.

[0005] The length of the polypeptide sequence has an important influence on the effect and physicochemical properties of the polypeptide. Most of the existing polypeptide generation models cannot control the length of the polypeptide sequence they generate. The length of the generated sequence is completely determined by the generation model, and it is not possible to intervene in the length of the generated sequence from the beginning of generation. Summary of the invention

[0006] Based on this, in order to solve the above technical problems, a method and system for generating a polypeptide sequence with controllable length based on a mask mechanism are provided, which can control the length of the generated polypeptide sequence.

[0007] A method for generating a polypeptide sequence with controllable length based on a mask mechanism, the method comprising:

[0008] Collecting polypeptide sequences from a polypeptide database, and dividing the polypeptide sequences into various polypeptide sequence data sets after data processing;

[0009] Encode the polypeptide sequence dataset by using a vectorized text tool, and add a start character, an end character, and a padding character; input the polypeptide sequence dataset after adding characters into the ESM2 model to extract the polypeptide sequence latent space embedding;

[0010] Noising the latent space embedding, training a denoiser based on the noisy latent space embedding in combination with a masking mechanism, predicting an unnoised latent space embedding through the denoiser, and calculating a mean square error loss between the predicted unnoised latent space embedding and the actual unnoised latent space embedding;

[0011] Noising the latent space embedding, training a guidance module based on the noisy latent space embedding combined with a mask mechanism, predicting the category of the noisy latent space embedding through the guidance module, and calculating the cross entropy loss between the predicted category and the actual category;

[0012] Noise-free data is obtained based on the mean square error loss and the cross entropy loss, and the noise-free data is decoded to obtain a polypeptide sequence.

[0013] In one embodiment, polypeptide sequences are collected from a polypeptide database, and the polypeptide sequences are processed and divided into various polypeptide sequence data sets, including:

[0014] Collect peptide sequences marked as antibacterial, antifungal, and antiviral functions from the peptide database;

[0015] Deleting repeated polypeptide sequences from the collected polypeptide sequences, and classifying the remaining polypeptide sequences into categories to obtain various polypeptide sequence data sets;

[0016] Among them, the various peptide sequence data sets are antibacterial peptide data set, antifungal peptide data set, antiviral peptide data set, and peptide mixed data set.

[0017] In one embodiment, the polypeptide sequence dataset is encoded by a vectorized text tool, and a start symbol, an end symbol, and a padding character are added, including:

[0018] Encode the peptide sequence dataset using a vectorized text tool, and output the format encoding of the index vector and the attention mask vector;

[0019] Adding a start symbol at the start position of each of the polypeptide sequence data sets, adding an end symbol at the end position, and adding a padding character at the end when the polypeptide sequence is less than the target character;

[0020] The method further comprises:

[0021] The index vector and the attention mask vector are input into the ESM2 model to obtain the latent space embedding of the polypeptide sequence.

[0022] In one embodiment, the latent space embedding is denoised, a denoiser is trained based on the noisy latent space embedding in combination with a masking mechanism, and the denoiser is used to predict the unnoised latent space embedding, including:

[0023] Gradually adding Gaussian noise to the latent space embedding until the latent space embedding is completely transformed into full Gaussian noise, completing the noise adding process and obtaining the latent space embedding after the noise adding;

[0024] Input the latent space embedding after adding noise, time step, and attention mask vector into a denoiser, and use LayerNorm to normalize the latent space embedding after adding noise, and convert the attention mask vector into a tensor;

[0025] The unnoised latent space embedding is predicted based on the normalized latent space embedding and the tensor.

[0026] In one embodiment, gradually adding Gaussian noise to the latent space embedding until the latent space embedding is completely transformed into full Gaussian noise, comprising:

[0027] Determining a latent space embedding distribution based on the latent space embedding, determining intensity data of adding noise, and obtaining a corresponding mean and variance;

[0028] Based on the latent space embedding, the latent space embedding distribution, and the intensity data, Gaussian noise is gradually added controlled by the mean and variance.

[0029] In one embodiment, the denoiser uses LayerNorm to normalize the latent space embedding after adding noise, and converts the attention mask vector into a tensor, including:

[0030] Based on the denoiser, the time step is encoded into a continuous variable using cosine position encoding, and after encoding, it is input into a multi-layer perceptron to obtain the processed time step;

[0031] Using LayerNorm to normalize the latent space embedding after adding noise to obtain normalized noisy data;

[0032] The processed time step is input into a multi-layer perceptron again, and the target time step is output;

[0033] The target time step and the normalized noisy data are scaled and shifted, and the output tensor is obtained based on the ESM2 model together with the attention mask vector.

[0034] In one embodiment, the method further comprises:

[0035] Determine the number of repetitions;

[0036] The output tensor is used as the noisy data, and scaling and shifting operations are repeatedly performed until the number of repeated operations is reached.

[0037] In one embodiment, predicting an unnoised latent space embedding by the denoiser comprises:

[0038] The output tensor is added to the normalized noisy data, normalized using LayerNorm, and then input into a multi-layer perceptron to output the predicted unnoised latent space embedding.

[0039] In one embodiment, the method further comprises:

[0040] Processing the noisy latent space embedding and the corresponding time step by the guiding module to obtain the probability of the category to which the noisy latent space embedding belongs;

[0041] The gradient of the classifier in the guide template is calculated, and the logarithm of the gradient is taken. The mean and variance are modified based on the logarithm, and a noise-free latent space embedding is obtained after reparameterization.

[0042] A system for generating polypeptide sequences with controllable length based on a mask mechanism, the system comprising:

[0043] A data set collection module is used to collect polypeptide sequences from a polypeptide database, and divide the polypeptide sequences into various polypeptide sequence data sets after data processing;

[0044] An extraction module is used to encode the polypeptide sequence dataset by using a vectorized text tool, and add a start character, an end character, and a padding character; the polypeptide sequence dataset after the characters are added is input into the ESM2 model to extract the polypeptide sequence latent space embedding;

[0045] A noise training denoiser module is used to denoise the latent space embedding, train a denoiser based on the noisy latent space embedding in combination with a masking mechanism, predict the unnoised latent space embedding through the denoiser, and calculate the mean square error loss between the predicted unnoised latent space embedding and the actual unnoised latent space embedding;

[0046] A noisy training guidance template is used to add noise to the latent space embedding, train a guidance module based on the noisy latent space embedding combined with a mask mechanism, and predict the category of the noisy latent space embedding through the guidance module, and calculate the cross entropy loss between the predicted category and the actual category;

[0047] The polypeptide sequence sampling module is used to obtain noise-free data based on the mean square error loss and the cross entropy loss, and decode the noise-free data to obtain a polypeptide sequence.

[0048] The above-mentioned method and system for generating polypeptide sequences with controllable length based on a mask mechanism, because the mask mechanism is used in the denoiser, combines the advantages of the protein language model and the diffusion model. The mask mechanism is also used in the protein language model, which can not only control the length of the generated sequence, but also make the diffusion model fit the characteristics of the protein language model well, thereby making the diffusion model generate a latent space embedding that is more in line with the protein language model. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 FIG. 1 is an application environment diagram of a method for generating a polypeptide sequence with controllable length based on a mask mechanism in one embodiment;

[0050] Figure 2 Schematic diagram of a process for generating a polypeptide sequence with controllable length based on a mask mechanism in one embodiment;

[0051] Figure 3 A schematic diagram of a latent space embedding extraction process of a protein sequence of an ESM2 model in one embodiment;

[0052] Figure 4 is a schematic diagram of the architecture of a denoiser in one embodiment;

[0053] Figure 5 for Figure 4 Schematic diagram of the architecture of the denoising block in the structure;

[0054] Figure 6 A schematic diagram of a general process of adding noise and removing noise of a diffusion model in one embodiment;

[0055] Figure 7 A schematic diagram of a process of a guiding module in a sampling phase in an embodiment;

[0056] Figure 8 A schematic diagram of a polypeptide sequence sampling process in one embodiment;

[0057] Fig. 9 It is a structural block diagram of a polypeptide sequence generation system with controllable length based on a mask mechanism in one embodiment;

[0058] Fig.10FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0060] The method for generating a polypeptide sequence with controllable length based on a mask mechanism provided in the embodiments of the present application can be applied to Figure 1 In the application environment shown. Figure 1 As shown, the application environment includes a computer device 110. The computer device 110 can collect polypeptide sequences from a polypeptide database, and divide the polypeptide sequences into various polypeptide sequence data sets after data processing; the computer device 110 can encode the polypeptide sequence data set through a vectorized text tool, and add start characters, end characters, and padding characters; the polypeptide sequence data set after adding characters is input into the protein language ESM2 model to extract the polypeptide sequence latent space embedding; the computer device 110 can add noise to the latent space embedding based on the diffusion model, train a denoiser based on the latent space embedding after adding noise in combination with a masking mechanism, and predict the unnoised latent space embedding through the denoiser, and calculate the mean square error loss between the predicted unnoised latent space embedding and the actual unnoised latent space embedding; the computer device 110 can add noise to the latent space embedding, train a guide module based on the latent space embedding after adding noise in combination with a masking mechanism, and predict the category of the noisy latent space embedding through the guide module, and calculate the cross entropy loss between the predicted category and the actual category; the computer device 110 can obtain noise-free data based on the mean square error loss and the cross entropy loss, and decode the noise-free data to obtain the polypeptide sequence. The computer device 110 may be, but is not limited to, various personal computers, laptop computers, smart phones, robots, unmanned aerial vehicles, tablet computers, portable wearable devices, and the like.

[0061] In one embodiment, Figure 2 As shown, a method for generating a polypeptide sequence with controllable length based on a mask mechanism is provided, comprising the following steps:

[0062] Step 202: collect polypeptide sequences from a polypeptide database, and divide the polypeptide sequences into various polypeptide sequence data sets after data processing.

[0063] Among them, peptide databases can include CAMPR4, ADP, DBAASP, dbAMP2, DRAMP, SATPDB, LAMP2 and GRAMPA.

[0064] In one embodiment, a method for generating a polypeptide sequence with controllable length based on a mask mechanism may also include a process of collecting polypeptide sequences and dividing the data set, and the specific process includes: collecting polypeptide sequences marked with antibacterial, antifungal, and antiviral functions from a polypeptide database; deleting repeated polypeptide sequences in the collected polypeptide sequences, and classifying the remaining polypeptide sequences into categories to obtain various polypeptide sequence data sets; wherein the various polypeptide sequence data sets are respectively an antibacterial peptide data set, an antifungal peptide data set, an antiviral peptide data set, and a polypeptide mixed data set.

[0065] Specifically, the computer device can collect polypeptide sequences marked as antibacterial, antifungal and antiviral functions from the polypeptide databases CAMPR4, ADP, DBAASP, dbAMP2, DRAMP, SATPDB, LAMP2 and GRAMPA and delete repeated sequences, and then divide them into three categories, namely antibacterial peptides, antifungal peptides and antiviral peptides, to construct four polypeptide data sets. The first one is a mixture of three polypeptides for training the denoiser; the second, third and fourth ones are antibacterial peptides, antifungal peptides and antiviral peptides data sets, respectively, for training the guide module; the polypeptide sequences in each data set are composed of twenty common amino acids, represented by the English letters A, C, D, E, F, G, H, I, K, L, M, N, P, Q, R, S, T, V, W and Y, respectively.

[0066] Step 204, encode the polypeptide sequence data set by using a vectorized text tool, and add start characters, end characters, and padding characters; input the polypeptide sequence data set after adding characters into the protein language ESM2 model to extract the polypeptide sequence latent space embedding.

[0067] The computer device can use the protein language model ESM2 to encode the polypeptide sequence dataset to obtain the corresponding latent space embedding.

[0068] In one embodiment, a method for generating a polypeptide sequence with controllable length based on a mask mechanism may also include a process for extracting a latent space embedding of a polypeptide sequence, the specific process including: encoding a polypeptide sequence dataset through a vectorized text tool, outputting a format encoding of an index vector and an attention mask vector; adding a start symbol to the start position of each polypeptide sequence dataset, adding an end symbol to the end position, and adding padding characters to the end when the polypeptide sequence is less than the target character; inputting the index vector and the attention mask vector into the ESM2 model to obtain a latent space embedding of the polypeptide sequence. The process of extracting the latent space embedding of the polypeptide sequence is as follows: Figure 3 shown.

[0069] Specifically, the computer device can encode the polypeptide sequence data set through the vectorized text tool Tokenizer and add a CLS start symbol at the start position, add an EOS end symbol at the end position, and fill the end with PAD characters when the polypeptide sequence is less than 50 characters.

[0070] After encoding, the output format encoding includes input_ids, which represents the index of each amino acid in the sequence in the vocabulary; attention mask indicates whether the current position corresponds to a valid character, which is 1 if valid and 0 if not; after encoding, a 52-dimensional index vector and a 52-dimensional attention mask vector can be obtained; the obtained index vector and attention mask are input into the pre-trained protein language model ESM2 model, and finally a 52*320 tensor is obtained, which is the latent space embedding of the polypeptide sequence.

[0071] Step 206, denoising the latent space embedding based on the diffusion model, training a denoiser based on the noisy latent space embedding combined with a masking mechanism, predicting the unnoised latent space embedding through the denoiser, and calculating the mean square error loss between the predicted unnoised latent space embedding and the actual unnoised latent space embedding.

[0072] The obtained latent space embedding is denoised and the denoiser is trained in combination with the corresponding mask. The structural block diagram of the denoiser is shown in Figure 4 As shown in the figure, the structural block diagram of the denoising block in the denoiser is as follows Figure 5 As shown in Figure 2, the denoiser directly predicts the unnoised latent space embedding, and the optimization objective is the mean square error loss between the predicted unnoised latent space embedding and the true unnoised latent space embedding.

[0073] In this embodiment, the computer device can add noise to the obtained ESM2 latent space embedding, the generative model learns its noise addition process and performs denoising, and uses the back propagation algorithm to update the model parameters until the training end conditions are met to terminate the training and obtain a trained generative model.

[0074] In one embodiment, a method for generating a polypeptide sequence with controllable length based on a mask mechanism is provided, and further, the process of adding noise to the obtained latent space embedding of the polypeptide sequence and training a denoiser for denoising in combination with a corresponding mask is provided. The specific process includes: gradually adding Gaussian noise to the latent space embedding until the latent space embedding is completely converted into full Gaussian noise, completing the denoising process, and obtaining the noisy latent space embedding; inputting the noisy latent space embedding, time step, and attention mask vector into the denoiser, and normalizing the noisy latent space embedding using LayerNorm through the denoiser, and converting the attention mask vector into a tensor; and predicting the unnoised latent space embedding based on the normalized latent space embedding and the tensor.

[0075] Specifically, in one embodiment, the process of performing noise addition processing may include: determining the latent space embedding distribution based on the latent space embedding, determining the intensity data of adding noise, and obtaining the corresponding mean and variance; based on the latent space embedding, the latent space embedding distribution, and the intensity data, gradually adding Gaussian noise by controlling the mean and variance. The general process of the noise addition and denoising process of the diffusion model is as follows Figure 6 shown.

[0076] The noise addition process is also called the diffusion process, which refers to gradually adding Gaussian noise to the original data until the original data is completely transformed into full Gaussian noise. Where x0 represents the ESM2 latent space embedding, q(x0) is used to represent the ESM2 latent space embedding distribution; the diffusion process uses q(x t |x t-1 ) represents the process of gradually adding noise, which is composed of mean μ and variance σ 2 Control, then the Markov chain of the diffusion process is expressed as: The diffusion process is expressed as:

[0077] Among them, an important feature of the diffusion model is that its state at any time t can be directly obtained from x0: in Indicates the intensity of added noise, and the calculation formula is:

[0078] The denoising process is also called the reverse diffusion process, expressed as p(x t-1 |x t ). First, a Gaussian noise is randomly sampled at time T, and the denoising model is used to gradually denoise it. Finally, the data x0 that is consistent with the original real data distribution is obtained. The reverse diffusion process can be expressed as: where μ θ and Σ θ represents the mean and variance of the prediction.

[0079] In one embodiment, the process of processing using a denoiser may include: based on the denoiser, using cosine position encoding to encode the time step into a continuous variable, and after encoding, inputting it into a multi-layer perceptron to obtain a processed time step; using LayerNorm to normalize the latent space embedding after noise addition to obtain normalized noisy data; inputting the processed time step into a multi-layer perceptron again to output a target time step; scaling and shifting the target time step and the normalized noisy data, and obtaining an output tensor based on the ESM2 model with the attention mask vector.

[0080] The denoising process follows the DDPM method. First, the noisy data x t , time step t and attention mask vector are input into the denoiser to obtain the predicted raw data Then according to the original data of the prediction To calculate the mean and variance at time t-1: where β t =1-α t ,and So you can use To infer β t After finding the mean and variance at time t-1, use the formula to calculate x t-1 :

[0081] The denoiser receives three parameters, namely the noisy data x t , time step t, attention mask vector. In this embodiment, cosine position encoding can be used to encode time step t into a continuous variable, and after encoding, it is input into a multi-layer perceptron. Among them, the cosine position encoding formula is: PE (pos,2i) =sin(pos / 10000 2i / 640 );PE (pos,2i+1) =cos(pos / 10000 2i / 640 );PE (pos,2i) Represents the value of the ith dimension at the posth position in the position encoding matrix; i represents the value of the ith dimension in the position encoding matrix.

[0082] Next, you can use LayerNorm to normalize the noisy data and then convert the attention mask vector into a (1,52,52) tensor.

[0083] In one embodiment, a method for generating a polypeptide sequence with controllable length based on a mask mechanism may also include a process of repeated scaling and displacement processing, and the specific process includes: determining the number of repeated operations; treating the output tensor as noisy data, and repeatedly performing scaling and displacement operations until the number of repeated operations is reached.

[0084] The encoded time step processed by the multi-layer perceptron is input into another multi-layer perceptron. The output is scaled and shifted with the normalized noisy data, and then input into EMS2AttentionLayer with the deformed attentionmask tensor for processing to output a 52*320 tensor. The output is then used to replace the above noisy data, and the operation is repeated for the specified number of times.

[0085] In one embodiment, a method for generating a polypeptide sequence with controllable length based on a mask mechanism may also include: adding the output tensor to the normalized noisy data, normalizing it using LayerNorm, and inputting it into a multi-layer perceptron, and outputting a predicted non-noised latent space embedding.

[0086] The obtained tensor is added to the normalized noisy data and normalized using LayerNorm, and finally enters a multi-layer perceptron, and the output is the predicted original data without noise. The model is based on the noisy data x t Directly predicting raw data Loss selection prediction raw data The l2 loss between the real original data x0 is expressed as: Where L(θ) represents the optimization objective, represents the denoiser, M represents the attentionmask tensor, represents the square of the 2-norm.

[0087] Step 208, add noise to the latent space embedding, train a guidance module based on the noisy latent space embedding combined with a mask mechanism, predict the category of the noisy latent space embedding through the guidance module, and calculate the cross entropy loss between the predicted category and the actual category.

[0088] The obtained latent space embedding is denoised and combined with the corresponding mask to guide the module. The guidance module predicts the category of the noisy latent space embedding, and the optimization target is the cross entropy loss between the predicted category and the true category.

[0089] Specifically, in one embodiment, a method for generating a polypeptide sequence with controllable length based on a mask mechanism may also include: processing the noisy latent space embedding and the corresponding time step through a guiding module to obtain the probability of the category to which the noisy latent space embedding belongs; calculating the gradient of the classifier in the guiding template, taking the logarithm of the gradient, modifying the mean and variance based on the logarithm, and obtaining a noise-free latent space embedding after re-parameterization.

[0090] The guidance module includes a guidance classifier and a mean calculation part; the guidance classifier is used to classify the ESM2 latent space embedding of the noisy peptide sequence; the mean calculation part uses the gradient of the guidance classifier to modify the mean at time t-1, so that the denoising process proceeds in the expected direction. The process of the guidance module in the sampling stage is as follows Figure 7 shown.

[0091] The model structure of the guided classifier is basically the same as that of the denoiser, and modifications need to be made. The specific modifications are: the input of the last multi-layer perceptron is (52*320,640); the output of the last multi-layer perceptron is a 2D vector, and the Softmax activation function is used to calculate the probabilities of the two categories.

[0092] In the denoising process, after calculating the mean and variance at time t-1, the calculation formula used to update the mean at time t-1 is: Where s is used to control the guiding strength, Indicates x t Find the derivative, G φ represents the guided classifier, G φ (y|x t ) represents the x obtained by guiding the classifier t is the probability of category y.

[0093] Step 210, obtaining noise-free data based on mean square error loss and cross entropy loss, and decoding the noise-free data to obtain a polypeptide sequence.

[0094] Specifically, Figure 8 As shown in the figure, when sampling peptide sequences, one type of peptide can be selected from the three types of peptides: antibacterial peptides, antifungal peptides, and antiviral peptides, and the length of the sequence to be generated can be specified. The two goals are input into the generation model, which includes a noise acquisition module, a denoising module, a guidance module, and a decoding module. Among them, the noise acquisition module samples noise points from the Gaussian distribution for the initial input of the denoising module; the denoising module takes the time step t of the current noisy data and the noisy data x t Processing is performed to obtain the predicted raw data without noise Then according to To calculate the mean and variance at time t-1; the guidance module is used to convert the current noisy data time step t and the noisy data x t Processing, get x t The probability of the category to which it belongs is then calculated based on the probability of the specified peptide type, and the gradient of the classifier is calculated and the logarithm is taken, and the result is used to modify the mean at time t-1; finally, the modified mean and variance are used to reparameterize to obtain x t-1 ; Repeat the above to obtain x0; the decoding module is used to decode the denoised x0 to obtain a new polypeptide sequence.

[0095] In this application, the advantages of the protein language model and the diffusion model are combined. A mask mechanism is used in the denoiser, which is also used in the protein language model. It can not only control the length of the generated sequence, but also make the diffusion model fit the characteristics of the protein language model well, so that the diffusion model generates a latent space embedding that is more in line with the protein language model. The training time of the guided classifier is much shorter than that of the diffusion model. The training time required to use the guidance module to guide the diffusion model to generate a model of a specified type of polypeptide is much shorter than the time required to train multiple generation models separately, which can save time costs.

[0096] It should be understood that, although the various steps in the above-mentioned flow chart are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the above-mentioned flow chart may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0097] In one embodiment, Fig. 9 As shown, a polypeptide sequence generation system with controllable length based on a mask mechanism is provided, comprising: a data set collection module 910, an extraction module 920, a noise training denoiser module 930, a noise training guide template 940 and a polypeptide sequence sampling module 950, wherein:

[0098] The data set collection module 910 is used to collect polypeptide sequences from the polypeptide database, and divide the polypeptide sequences into various polypeptide sequence data sets after data processing;

[0099] The extraction module 920 is used to encode the polypeptide sequence data set by using a vectorized text tool, and add a start character, an end character, and a padding character; the polypeptide sequence data set after the characters are added is input into the ESM2 model to extract the polypeptide sequence latent space embedding;

[0100] A noise training denoiser module 930 is used to denoise the latent space embedding based on a diffusion model, train a denoiser based on the noisy latent space embedding in combination with a masking mechanism, predict the unnoised latent space embedding through the denoiser, and calculate the mean square error loss between the predicted unnoised latent space embedding and the actual unnoised latent space embedding;

[0101] A noise training guidance template 940 is used to add noise to the latent space embedding, train a guidance module based on the latent space embedding after adding noise in combination with a mask mechanism, and predict the category of the noisy latent space embedding through the guidance module, and calculate the cross entropy loss between the predicted category and the actual category;

[0102] The polypeptide sequence sampling module 950 is used to obtain noise-free data based on mean square error loss and cross entropy loss, and decode the noise-free data to obtain a polypeptide sequence.

[0103] In one embodiment, the data set collection module 910 is also used to collect polypeptide sequences marked as antibacterial, antifungal, and antiviral functions from a polypeptide database; delete repeated polypeptide sequences in the collected polypeptide sequences, and classify the remaining polypeptide sequences to obtain various polypeptide sequence data sets; wherein the various polypeptide sequence data sets are respectively an antibacterial peptide data set, an antifungal peptide data set, an antiviral peptide data set, and a polypeptide mixed data set.

[0104] In one embodiment, the extraction module 920 is also used to encode the polypeptide sequence dataset through a vectorized text tool, and output the format encoding of the index vector and the attention mask vector; add a start symbol to the starting position of each polypeptide sequence dataset, add an end symbol to the end position, and add padding characters at the end when the polypeptide sequence is less than the target characters; input the index vector and the attention mask vector into the ESM2 model to obtain the polypeptide sequence latent space embedding.

[0105] In one embodiment, the noise training denoiser module 930 is also used to gradually add Gaussian noise to the latent space embedding until the latent space embedding is completely converted into full Gaussian noise, completing the noise adding process and obtaining the noisy latent space embedding; the noisy latent space embedding, time step and attention mask vector are input into the denoiser, and the noisy latent space embedding is normalized by the denoiser using LayerNorm to convert the attention mask vector into a tensor; the unnoised latent space embedding is predicted based on the normalized latent space embedding and the tensor.

[0106] In one embodiment, the noise training denoiser module 930 is also used to determine the latent space embedding distribution based on the latent space embedding, determine the intensity data of the added noise, and obtain the corresponding mean and variance; based on the latent space embedding, the latent space embedding distribution, and the intensity data, the Gaussian noise is gradually increased by controlling the mean and variance.

[0107] In one embodiment, the noise training denoiser module 930 is also used to encode the time step into a continuous variable based on the denoiser using cosine position coding, and after encoding, input it into a multi-layer perceptron to obtain a processed time step; use LayerNorm to normalize the latent space embedding after noise addition to obtain normalized noisy data; input the processed time step into a multi-layer perceptron again to output a target time step; scale and shift the target time step and the normalized noisy data, and obtain an output tensor based on the ESM2 model with the attention mask vector.

[0108] In one embodiment, the denoising training denoiser module 930 is further used to determine the number of repetition operations; the output tensor is treated as noisy data and the scaling and shifting operations are repeatedly performed until the number of repetition operations is reached.

[0109] In one embodiment, the noise training denoiser module 930 is further used to add the output tensor to the normalized noisy data, normalize it using LayerNorm and then input it into a multi-layer perceptron to output a predicted non-noised latent space embedding.

[0110] In one embodiment, the noisy training guidance template 940 is also used to process the noisy latent space embedding and the corresponding time step through the guidance module to obtain the probability of the category to which the noisy latent space embedding belongs; calculate the gradient of the classifier in the guidance template, take the logarithm of the gradient, modify the mean and variance based on the logarithm, and obtain the noise-free latent space embedding after re-parameterization.

[0111] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Fig.10 As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for generating a polypeptide sequence with controllable length based on a mask mechanism is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a key, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0112] Those skilled in the art will understand that Fig.10The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0113] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the steps of a method for generating a polypeptide sequence with controllable length based on a mask mechanism are implemented.

[0114] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of a method for generating a polypeptide sequence with controllable length based on a mask mechanism are implemented.

[0115] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0116] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0117] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A method for generating a polypeptide sequence with controllable length based on a mask mechanism, characterized in that: The method comprises: Collecting polypeptide sequences from a polypeptide database, and dividing the polypeptide sequences into various polypeptide sequence data sets after data processing; The polypeptide sequence dataset is encoded by a vectorized text tool, and a start character, an end character, and a padding character are added; the polypeptide sequence dataset after the characters are added is input into the protein language ESM2 model to extract the polypeptide sequence latent space embedding; Noising the latent space embedding based on a diffusion model, training a denoiser based on the noisy latent space embedding combined with a masking mechanism, predicting an unnoised latent space embedding through the denoiser, and calculating a mean square error loss between the predicted unnoised latent space embedding and the actual unnoised latent space embedding; Noising the latent space embedding, training a guidance module based on the noisy latent space embedding combined with a mask mechanism, predicting the category of the noisy latent space embedding through the guidance module, and calculating the cross entropy loss between the predicted category and the actual category; Noise-free data is obtained based on the mean square error loss and the cross entropy loss, and the noise-free data is decoded to obtain a polypeptide sequence.

2. The method for generating a polypeptide sequence with controllable length based on a mask mechanism according to claim 1, characterized in that: Collect polypeptide sequences from a polypeptide database, and divide the polypeptide sequences into various polypeptide sequence data sets after data processing, including: Collect peptide sequences marked as antibacterial, antifungal, and antiviral functions from the peptide database; Deleting repeated polypeptide sequences from the collected polypeptide sequences, and classifying the remaining polypeptide sequences into categories to obtain various polypeptide sequence data sets; Among them, the various peptide sequence data sets are antibacterial peptide data set, antifungal peptide data set, antiviral peptide data set, and peptide mixed data set.

3. The method for generating a polypeptide sequence with controllable length based on a mask mechanism according to claim 1, characterized in that: The peptide sequence dataset is encoded by a vectorized text tool, and start characters, end characters, and padding characters are added, including: Encode the polypeptide sequence dataset using a vectorized text tool, and output the format encoding of the index vector and the attention mask vector; Adding a start symbol at the start position of each of the polypeptide sequence data sets, adding an end symbol at the end position, and adding a padding character at the end when the polypeptide sequence is less than the target character; The method further comprises: The index vector and the attention mask vector are input into the ESM2 model to obtain the latent space embedding of the polypeptide sequence.

4. The method for generating a polypeptide sequence with controllable length based on a mask mechanism according to claim 3, characterized in that: Noising the latent space embedding based on a diffusion model, training a denoiser based on the noisy latent space embedding combined with a masking mechanism, and predicting the unnoised latent space embedding by the denoiser, including: Gradually adding Gaussian noise to the latent space embedding until the latent space embedding is completely transformed into full Gaussian noise, completing the noise adding process and obtaining the latent space embedding after the noise adding; Input the latent space embedding after adding noise, time step, and attention mask vector into a denoiser, and use LayerNorm to normalize the latent space embedding after adding noise, and convert the attention mask vector into a tensor; The unnoised latent space embedding is predicted based on the normalized latent space embedding and the tensor.

5. The method for generating a polypeptide sequence with controllable length based on a mask mechanism according to claim 4, characterized in that: Gradually adding Gaussian noise to the latent space embedding until the latent space embedding is completely transformed into full Gaussian noise, including: Determining a latent space embedding distribution based on the latent space embedding, determining intensity data of adding noise, and obtaining a corresponding mean and variance; Based on the latent space embedding, the latent space embedding distribution, and the intensity data, Gaussian noise is gradually added controlled by the mean and variance.

6. The method for generating a polypeptide sequence with controllable length based on a mask mechanism according to claim 4, characterized in that: The denoiser uses LayerNorm to normalize the latent space embedding after adding noise, and converts the attention mask vector into a tensor, including: Based on the denoiser, the time step is encoded into a continuous variable using cosine position encoding, and after encoding, it is input into a multi-layer perceptron to obtain the processed time step; Using LayerNorm to normalize the latent space embedding after adding noise to obtain normalized noisy data; The processed time step is input into a multi-layer perceptron again, and the target time step is output; The target time step and the normalized noisy data are scaled and shifted, and the output tensor is obtained based on the ESM2 model together with the attention mask vector.

7. The method for generating a polypeptide sequence with controllable length based on a mask mechanism according to claim 6, characterized in that: The method further comprises: Determine the number of repetitions; The output tensor is used as the noisy data, and scaling and shifting operations are repeatedly performed until the number of repeated operations is reached.

8. The method for generating a polypeptide sequence with controllable length based on a mask mechanism according to claim 6, characterized in that: Predicting an unnoised latent space embedding through the denoiser comprises: The output tensor is added to the normalized noisy data, normalized using LayerNorm, and then input into a multi-layer perceptron to output the predicted unnoised latent space embedding.

9. The method for generating a polypeptide sequence with controllable length based on a mask mechanism according to claim 1, characterized in that: The method further comprises: Processing the noisy latent space embedding and the corresponding time step by the guiding module to obtain the probability of the category to which the noisy latent space embedding belongs; The gradient of the classifier in the guide template is calculated, and the logarithm of the gradient is taken. The mean and variance are modified based on the logarithm, and a noise-free latent space embedding is obtained after reparameterization.

10. A system for generating polypeptide sequences with controllable length based on a mask mechanism, characterized in that: The system comprises: A data set collection module is used to collect polypeptide sequences from a polypeptide database, and divide the polypeptide sequences into various polypeptide sequence data sets after data processing; An extraction module is used to encode the polypeptide sequence dataset by using a vectorized text tool, and add a start character, an end character, and a padding character; the polypeptide sequence dataset after the characters are added is input into the ESM2 model to extract the polypeptide sequence latent space embedding; A noise training denoiser module is used to denoise the latent space embedding based on a diffusion model, train a denoiser based on the noisy latent space embedding in combination with a masking mechanism, predict the unnoised latent space embedding through the denoiser, and calculate the mean square error loss between the predicted unnoised latent space embedding and the actual unnoised latent space embedding; A noisy training guidance template is used to add noise to the latent space embedding, train a guidance module based on the noisy latent space embedding combined with a mask mechanism, and predict the category of the noisy latent space embedding through the guidance module, and calculate the cross entropy loss between the predicted category and the actual category; The polypeptide sequence sampling module is used to obtain noise-free data based on the mean square error loss and the cross entropy loss, and decode the noise-free data to obtain a polypeptide sequence.

Images