Enzyme sequence generation method based on diffusion model

Abstract

The invention discloses an enzyme sequence generation method based on a diffusion model, and belongs to the technical field of biological information. In order to realize that multi-modal data is constructed through one-hot coding and an ESM-1b model under a classification guidance condition, and then a deep learning model based on a Stable Diffusion model and combining a global attention and local attention mechanism denoising module is established. Meanwhile, it is verified that the method is superior to other models in the aspects of structural prediction confidence, structural similarity and sequence homology in terms of performance and results in a controllable protein sequence generation task.

Description

Technical Field

[0001] The invention relates to an enzyme sequence generation method based on a diffusion model, which can be used for generating specific enzymes under classification guidance conditions and belongs to the field of bioinformatics technology. Background Art

[0002] In the fields of biology and chemistry, designing enzymes with specific catalytic functions and structural stability is a crucial task. Generative models have demonstrated their great potential in reliable enzyme design. However, despite the success of Repecka et al. in designing biologically rational enzyme sequences in 2023, these enzyme design models are mainly limited to unconditional generation. In practical application scenarios, researchers usually need to screen out sequences that meet specific criteria from randomly generated enzymes, which is not only time-consuming but also inefficient. Therefore, the current unconditional protein generation technology is still a certain distance away from the needs of practical applications. Improving these technologies will help accelerate scientific discoveries and improve efficiency in the field of bioengineering. Therefore, the development of a classification-guided diffusion model for the generation of specific enzyme sequences, which combines biological species information and the generation ability of diffusion models to generate structurally stable enzyme sequences, is of great significance for the study of specific enzymes.

[0003] In 2024, Lin et al. proposed a classification-guided diffusion model TaxDiff, which gradually adds noise to the data through a forward diffusion process to generate noisy data, and denoises it through a module combined with an attention mechanism. Finally, the denoised data is converted into a new enzyme sequence through a decoder, achieving classification guidance for the enzyme sequence and accurate generation of structural stability. However, although the original TaxDiff model can effectively denoise and generate structurally stable enzyme sequences, it only uses a single encoding process when digitizing the enzyme sequence. However, relying on a single data source may lack an accurate description of the function or structure of certain enzymes, resulting in the generated enzyme sequence being less than ideal in function or structure. At the same time, the traditional diffusion model has the problem of slow convergence and low quality of the generated enzyme sequence. For the above reasons, it is necessary to study a method for generating higher-quality specific enzyme sequences. Summary of the invention

[0004] In order to solve the problem of generation quality in the existing enzyme sequence generation method, the present invention provides an enzyme sequence generation method based on a diffusion model, especially for specific enzyme generation under classification guidance, the method comprises the following steps: Step S1: The enzyme sequence is digitized using one-hot encoding and the ESM-1b protein language model to form multimodal data. The multimodal data formed contains different types of information, such as protein sequence, structural information, etc. This diverse information can provide a more comprehensive representation and help the model understand and generate the target enzyme sequence more accurately.

[0005] Step S2: Construct a Stable Diffusion model for generating enzyme sequences. Compared with the traditional diffusion model, the Stable Diffusion model has the advantages of faster convergence speed and higher generation quality. Noise is added to the data using the forward diffusion process of the Stable Diffusion model.

[0006] Step S3: After the noisy data is embedded by position, time step and classification label, these embedded information are combined as input, and the input data is denoised by a denoising module that combines global attention and local attention mechanisms.

[0007] Step S4: The denoised enzyme sequence is used as input into a decoder to generate a specific high-quality enzyme sequence.

[0008] Optionally, the step S1 includes: The enzyme sequence features are standardized to ensure that different features are on the same scale and improve the training stability of the model.

[0009] One-hot encoding method is used to extract features from enzyme sequences, converting sequences with fixed length into vector representations; Optionally, in addition to one-hot encoding, convolutional neural networks can be used to extract features from enzyme sequences to capture more complex local features.

[0010] Optionally, in addition to using the ESM-1b protein language model, language models such as BERT and ProtBERT may also be used for encoding.

[0011] Beneficial effects of the present invention: The enzyme sequence is digitized by computer encoding, public data sets are collected, and a multimodal data set for training is constructed through one-hot encoding and the ESM-1b protein language model. With the digitized enzyme sequence as input, the Stable Diffusion model learns to add noise to the data, and reduces the noise of the data by introducing a denoising module that adopts classification guidance and combines global attention and local attention mechanisms, and generates a specific enzyme sequence through a decoder. The application also provides the application of the above-mentioned enzyme sequence generation method based on the diffusion model in the fields of chemicals, pharmaceuticals, etc. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0013] Figure 1 It is a flow chart of generating enzyme sequence based on diffusion model provided in one embodiment of the present invention. DETAILED DESCRIPTION

[0014] In order to make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0015] PyTorch: is the python version of torch, an open source neural network framework by Facebook, specifically for GPU-accelerated deep neural network (DNN) programming. Torch is a classic tensor library (tensor, a tensor is a digital container in machine learning programs, essentially an array of various dimensions, usually the dimensions of a tensor are called axes, and the number of axes is called order) that operates on multi-dimensional matrix data. It is widely used in machine learning and other math-intensive applications. Unlike the static computational graph of Tensorflow, the computational graph of pytorch is dynamic and can be changed in real time according to computational needs.

[0016] Example: This example provides an enzyme sequence generation method based on a diffusion model, including a StableDiffusion model and a denoising module that combines global attention and local attention mechanisms, and combines multimodal data for training to generate a specific enzyme sequence, see Figure 1 , the method comprising: Step S1: One-hot encoding of the input enzyme sequence: In model training, the Uniref50 dataset from Uniprot was used, which is a protein database formed by clustering. Using this dataset for unconditional generative model training enables the model to learn a wide range of protein sequence features. At the same time, in order to verify the performance of the model under conditional generation, a specific enzyme sequence was selected for testing. Specifically, the amino acid sequence of a specific enzyme (such as KMNLTMAGLMYGVCLGPWHEE) was input into the model to observe whether the model can generate high-quality sequences related to the input conditions.

[0017] First, feature extraction is performed using the one-hot encoding method. For example, the amino acid sequence FMFYMPPPHYEGEGR of a specific enzyme is converted into a vector representation of length 20, where each amino acid corresponds to a vector with a position of 1 and the remaining positions of 0. We can get: [[0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0], [00 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0], [0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 00 1], [0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0], [0 0 0 0 0 0 0 0 0 0 0 0 1 00 0 0 0 0 0], [0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0],[0 0 0 0 0 0 0 0 0 00 1 0 0 0 0 0 0 0 0], [0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0], [0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1], [0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0], [0 00 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 00], [0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0], [0 0 0 0 0 0 0 0 0 0 0 0 0 10 0 0 0 0] ].

[0018] Step S2: Encoding of the input enzyme sequence: The amino acid sequence of a specific enzyme is simultaneously input into the ESM-1b model for encoding. The features formed by the ESM-1b language model can more effectively capture the structural characteristics of the enzyme.

[0019] Step S3: Construct a model for sequence feature learning, including: S3.1: Convert step S1 into a tensor recognizable by PyTorch; mainly convert the Numpy array into a tensor, then integrate the data obtained from S1 and S2 and input them into the Stable Diffusion model.

[0020] S3.2: In the diffusion process of the Stable Diffusion model, the model gradually adds noise to the input data. This process can be regarded as a Markov chain from the original data to the completely noisy data. Through multiple iterations, Gaussian noise is gradually added to the data. This process makes the data gradually become random and eventually close to Gaussian noise.

[0021] This process can be expressed as: gradually adding Gaussian noise to the input data to form a Markov chain, which can be specifically expressed as:

[0022] S3.3: In the denoising process, the model's task is to do the reverse operation, that is, to gradually denoise the completely noisy data and restore the original protein sequence data. First, the noisy data is embedded in position, time step, and classification identifier, and then input into the denoising block to gradually remove the noise in the data and restore clear protein sequence data.

[0023] This process can be expressed as:

[0024] In the denoising module, the global and local relationships between amino acids in the enzyme sequence are captured by using global attention and local attention mechanisms.

[0025] Step S4: The denoised enzyme sequence features are used as input into the decoder to ultimately generate a high-quality specific enzyme sequence.

[0026] In order to illustrate the superiority of the generative model constructed by the present invention, CAPR, LRAR, and Evodiff were selected as benchmark models for comparison, and natural protein structures in the PDB dataset and the AFDB dataset were selected for verification, and TM-score, RMSD, and Fident were used as evaluation indicators. Among them, TM-score is a novel indicator for measuring the similarity of two protein structures, and is particularly suitable for evaluating the overall similarity between the predicted structure and the reference structure. RMSD is the most commonly used indicator for measuring the similarity of two protein structures. The structural difference is evaluated by calculating the root mean square deviation between all atoms or Cα atoms after alignment, and Fident is a measure of the proportion of identical residues in two protein sequences or structural alignments, and is usually used to evaluate the quality of sequence similarity and structural alignment. The results are shown in Table 1. It can be seen from the table that the model performs well in the task of controllable generation of protein sequences, and outperforms other models in terms of structure prediction confidence, structural similarity, and sequence homology.

[0027]

Claims

1. A method for generating enzyme sequences based on a diffusion model, characterized in that: include: Step S1: Relying on Stable Diffusion and the denoising module that combines global attention and local attention mechanisms, during the diffusion process, the model gradually adds noise to the input data to form a Markov chain from the original data to the completely noisy data. After the noisy data is embedded in position, time step, and classification identifier, the data is denoised using a module that combines global attention and local attention mechanisms, and the global and local relationships between amino acids in the enzyme sequence are captured, ultimately generating a high-quality specific enzyme sequence; Step S2: using the Uniref50 public dataset, which is a protein database formed by clustering, to train the model and save the parameters after training; Step S3: Generate a specific enzyme sequence using the trained model, where the trained model is a diffusion model.

2. The method according to claim 1, characterized in that The step S2 comprises: Step S2.1: Digitize the protein sequences in the public dataset to form multimodal data; Step S2.2: The diffusion model for generating enzyme sequences is trained using the multimodal data formed after digital processing, and the loss value is calculated during the training process, and the model parameters after the training are saved.

3. The method according to claim 2, characterized in that The S2.1 includes: Step 1: Assume a protein sequence "VYKFN", convert it into one-hot encoding Valine (V): [1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], Tyrosine (Y): [0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], Lysine (K): [0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], Phenylalanine (F): [0, 0, 0, 1, Combining these one-hot encodings, we can get the one-hot encoding representation of the entire sequence; Step 2: Input the protein sequence into the ESM-1b protein language model to encode the sequence and capture the structural information of the protein; Step 3: Through steps 1 and 2, multimodal data about protein sequences are finally formed; Step 4: Use the custom network to calculate the multimodal data obtained in step 3 and output the final generated results; The custom network is built based on PyTorch, and includes a Stable Diffusion model, a denoising module that combines global attention and local attention mechanisms, and a decoder.

4. Application of the enzyme sequence generation method based on diffusion model according to any one of claims 1 to 3 in chemicals and medicines.

Images