A sequence diffusion method based on multiple sequence alignment and electronic device thereof

Abstract

The application discloses a sequence diffusion method based on multiple sequence alignment and an electronic device thereof. The application adopts two-stage training to split the purposes of training in different stages, which is more convenient for the interpretability of the model and the flexible training of the field sub-model. The application focuses on the feature extraction of MSA, extracts the hidden pair information in MSA by using a deeper attention layer, and can effectively capture the biological correlation between sequences. The application uses the sequence diffusion mode to realize the sequence generation guided by the evolutionary information, and can generate sequences with high correctness and diversity.

Description

Technical Field

[0001] This invention relates to the field of protein design, and more specifically to a sequence diffusion method based on multiple sequence alignment and its electronic device. Background Technology

[0002] In the field of bioinformatics, multiple sequence alignment (MSA) is an important tool for studying homologous biological sequences and their evolutionary relationships. MSA is used not only to identify conserved regions in similar sequences but also to help infer their common ancestor. With the development of high-throughput sequencing technology, the massive amounts of sequence data generated have provided abundant resources for biological research, but have also brought challenges to processing and analyzing this data.

[0003] In recent years, deep learning methods have shown great potential in sequence analysis. Deep learning can capture complex relationships between sequences by learning the implicit features in ever-growing datasets. Architectures such as convolutional neural networks and Transformers have been applied to tasks such as sequence classification, generation, and alignment. However, these methods still face difficulties in modeling long-range dependencies when dealing with complex biological sequences. Therefore, improving the accuracy of sequence analysis to generate high-quality and diverse sequences is a pressing technical problem that needs to be solved. Summary of the Invention

[0004] To address the shortcomings of existing technologies, one of the objectives of this invention is to propose a new method that aims to provide a way to deeply extract paired information from MSA and, based on this, perform noise addition and reverse denoising through a diffusion model to optimize the training method of sequence generation models.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a sequence diffusion training method based on multiple sequence alignment, comprising the following steps:

[0006] S1 Evolutionary Encoder: Input MSA and initial pairwise representation, the initial pairwise representation is presented in the form of a sequence distance matrix, the MSA is masked and embedded to obtain the MSA representation; the MSA representation and the initial pairwise representation are corrected by the pairwise update module and the MSA update module to obtain the updated MSA and pairwise representation;

[0007] S2 diffusion decoder: Randomly selects an initial sequence from the MSA, batch-adds noise to the initial sequence to obtain a noisy sequence, vectorizes the noisy sequence to obtain a single representation, fuses the single representation with the updated pairwise representation obtained in step S1, inputs the time step, denoises through the denoising module, outputs the updated single representation, and obtains the predicted sequence information based on the updated single representation and the MSA.

[0008] The denoising module, update module, and MSA update module are corrected by calculating the total loss function between the predicted sequence information and the real sequence information.

[0009] Repeat steps S1 and S2 until the preset training termination condition is met.

[0010] Preferably, in step S1, the input MSA needs to be sampled, and the sampling requirement is a coverage of 0.75 or higher; the character dimension of the sequence is 21, and the characters of the sequence include 20 standard amino acids and spacer characters.

[0011] Preferably, in step S1, the pairwise update module includes triangular update, triangular attention, and a transition layer; in step S1, the MSA update module includes row attention and a transition layer.

[0012] Preferably, in step S2, the denoising module includes an attention layer and a transition layer.

[0013] Preferably, step S1 is iterated repeatedly for 6 to 84 layers before step S2 is performed.

[0014] In step S2, the method for generating the single representation is to generate a single representation for each noisy sequence using an adaptive layer normalization diffusion transformer with 12 to 120 layers.

[0015] Preferably, step S1 is iterated repeatedly for 12 to 48 layers before step S2 is performed.

[0016] In step S2, the method for generating the single representation is to generate a single representation for each noisy sequence using an adaptive layer normalization diffusion transformer with 24 to 72 layers.

[0017] Preferably, in step S2, the single representation is presented in the form of a JAX computation graph, which is obtained by vectorizing the single representation through vmap.

[0018] To address the shortcomings of existing technologies, the second objective of this invention is to propose a new method that aims to provide a method for generating sequences with high probability of forward design by deeply extracting paired information from MSA, accurately capturing co-mutations between functional sites, and performing reverse denoising through a diffusion model.

[0019] To achieve the above objectives, the present invention provides the following technical solution:

[0020] A sequence diffusion generation method based on multiple sequence alignment is proposed. After training using the aforementioned sequence diffusion training method based on multiple sequence alignment,

[0021] If the original sequence is unknown, then in step S1, input an unmasked MSA, and in step S2, input a random Gaussian distribution as the initial sequence, and perform a reverse process to gradually denoise and generate a sequence.

[0022] If the original sequence is unknown, then in step S1, input an unmasked MSA, and in step S2, input a random Gaussian distribution as the initial sequence, and perform a reverse process to gradually denoise and generate a sequence.

[0023] If the protein motif is known in the original sequence, then in step S1, the MSA without masking is input, and in step S2, the protein motif is input as the initial sequence. The reverse process is then performed to gradually remove noise and complete the remaining part to generate the sequence.

[0024] If the original sequence is known, then in step S1, input the MSA without masking.

[0025] In step S2, the protein motif of the original sequence is input as the initial sequence, and the reverse process is performed to gradually remove noise and complete the remaining part to generate the sequence.

[0026] Alternatively, in step S2, the original sequence can be input as the initial sequence, and noise can be added first, followed by a reverse process to gradually denoise and generate a new sequence.

[0027] To address the shortcomings of existing technologies, a third objective of this invention is to propose an electronic device that provides a training system capable of deeply extracting paired information from MSA and, based on this, performing noise addition and reverse denoising through a diffusion model to optimize the sequence generation model. This system is an entity that stores code implementing the aforementioned training method.

[0028] To achieve the above objectives, the present invention provides the following technical solution:

[0029] An electronic device, comprising:

[0030] processor and

[0031] The memory stores executable code, which, when executed by the processor, causes the processor to execute the aforementioned sequence diffusion training method based on multiple sequence alignment.

[0032] To address the shortcomings of existing technologies, the fourth objective of this invention is to propose an electronic device that can deeply extract paired information from MSA, accurately capture co-mutations between functional sites, and, based on this, perform reverse denoising through a diffusion model to generate a sequence generation system with a high probability of positive design. This system is an entity that stores code implementing the above training method.

[0033] To achieve the above objectives, the present invention provides the following technical solution:

[0034] An electronic device, comprising:

[0035] processor and

[0036] The memory stores executable code, which, when executed by the processor, causes the processor to execute the aforementioned sequence diffusion generation method based on multiple sequence alignment.

[0037] Compared with existing technologies, the advantages of this invention are as follows: This invention employs a two-stage training approach, splitting the objectives of different training stages, which facilitates model interpretability and allows for flexible training of domain sub-models. This invention focuses on feature extraction from MSAs, using deeper attention layers to extract hidden pairwise information within MSAs, effectively capturing biological correlations between sequences. This invention utilizes sequence diffusion to achieve evolutionarily guided sequence generation, generating sequences with high accuracy and diversity. Attached Figure Description

[0038] Figure 1 A descent plot of the training loss function;

[0039] Figure 2 A visualization of paired representations;

[0040] Figure 3 The left image in the middle section shows the MSA representation learned by the model, while the right image shows the actual site frequency distribution.

[0041] Figure 4 A structure prediction graph of a sequence generated from a natural sequence as input;

[0042] Figure 5 Structural prediction map of another sequence of different lengths generated from the same input natural sequence;

[0043] Figure 6 A rendering of the sequence generated from an input protein motif;

[0044] Figure 7 The left and right figures show the distribution of sequences generated by the model, while the right figure shows the distribution of natural sequences.

[0045] Figure 8 This is an architecture diagram of the S1 evolutionary encoder;

[0046] Figure 9 This is a diagram of the architecture of the S2 diffusion encoder. Detailed Implementation

[0047] The present invention will be further described in detail below with reference to the embodiments.

[0048] MSA: Multi-Sequence Alignment, refers to the alignment of three or more biological sequences, such as protein sequences, which often conceals evolutionary relationships.

[0049] Outer Product Mean Fusion (EPS) calculates the outer product of each pair of residues in the MSA feature and averages them to obtain a new pairwise representation. This method is one of the key operations used for information fusion in protein structure prediction models.

[0050] Pair representation is used to describe the relationship between two residues in a sequence and is widely used in protein structure prediction tasks.

[0051] Single Representation: A representation method that globally encodes the entire sequence, providing holistic feature information, and is typically used for preliminary feature extraction in structure prediction.

[0052] Triangle Update optimizes paired representations by using additive and multiplicative update rules to conform to the geometric constraints of protein structure, thereby improving the ability to model relationships between residues.

[0053] Triangle Attention: This technique uses an attention mechanism to propagate information in paired representations, modeling long-range dependencies between residue pairs and improving the accuracy and consistency of protein structure prediction.

[0054] Transition Layer: An intermediate layer in a deep learning network, typically used for feature transformation, normalization, or non-linear activation to enhance the model's expressive power.

[0055] Row Attention: A type of attention mechanism that calculates attention weights row by row when comparing multiple sequences to extract important information along the row dimension. It is commonly used in MSA-related tasks.

[0056] Vmap: Vectorized Map, an automatic vectorization (batch processing) feature provided by JAX, which allows the same operation to be applied in parallel to multiple inputs, thereby improving computational efficiency.

[0057] JAX Computation Graph: An optimized execution graph built by JAX during computation, which can accelerate computation through JIT (Just-In-Time compilation) and supports automatic differentiation and hardware acceleration.

[0058] Example 1:

[0059] like Figure 8 and Figure 9 As shown, a sequence diffusion training method based on multiple sequence alignment includes the following steps:

[0060] The S1 evolutionary encoder takes the MSA (Sequence Distance Matrix) and initial paired representations as input, which are presented as sequence distance matrices. The MSA is masked and embedded to obtain the MSA representation. The MSA representation and the initial paired representation are then corrected by a paired update module and an MSA update module to obtain updated MSA and paired representations. In the first stage, the evolutionary encoder integrates the MSA into the paired representations layer by layer. The purpose of the MSA module is to learn paired representations from homologous sequences and decode individual sites and paired amino acids accordingly. Here, the input MSA needs to be sampled, masked, and embedded to obtain the MSA representation, increasing the model's robustness to MSA distributions. The paired representations are initialized with a sequence distance matrix. Subsequently, the MSA representations are progressively updated using the OuterProductMean, and then fed back into the MSA representations, achieving an interactive information flow from multi-track representations.

[0061] This invention uses the open-source OpenProteinSet dataset for model training. This data comes from multiple databases, including UniRef90, UniProt, MGnify, and UniClust30+BFD, and contains 140,000 unique PDB single strands and 270,000 multiple sequence alignments generated by UniClust30 after clustering and redundancy removal. Non-standard amino acids in the data were replaced with similar standard amino acids, and the data length was limited to 384 bytes or less. Additionally, we prepared family datasets for the model fine-tuning stage. For a specified family, we collected and filtered related datasets, then used BLAST to obtain homologous sequences for the family, removing redundant sequences and low-quality data. During this process, a sequence similarity clustering algorithm was used to remove duplicates and sequences with excessively high similarity, ensuring the uniqueness and representativeness of the final dataset.

[0062] This invention trains the model using a subsequent energy loss function and an MSA mask prediction task. The MSA mask prediction task uses MSA representations to indicate the predicted probability distribution after a linear layer. The paired coupling term and bias term of the energy term are parameterized from the paired and MSA representations learned by the model, respectively. Through the first stage of learning, the model acquires the ability to extract MSA information and calculate the score of the corresponding sequence distribution using the energy function, thereby achieving zero-shot fitness prediction capability.

[0063] The S2 diffusion decoder (Diffusion Transformer decoder) randomly selects an initial sequence from the MSA (Maximum Sequence Amplifier), batches of these sequences are denoised to obtain noisy sequences, and the noisy sequences are vectorized to obtain a single representation. This single representation is then fused with the updated pairwise representations obtained in step S1, and the time step is input. The denoising module then denoises the denoised sequences, outputting an updated single representation. The predicted sequence information is obtained based on the updated single representation and the MSA. In the second stage, the diffusion module uses the pairwise representations from the MSA to progressively analyze, denoise, and decode the single representations of the batches of representative noisy sequences.

[0064] The denoising module, update module, and MSA update module are corrected by calculating the total loss function between the predicted sequence information and the real sequence information.

[0065] Repeat steps S1 and S2 until the preset training termination condition is met.

[0066] Diffusion models are a type of generative method that constructs a process that gradually transforms data samples into random noise, then considers its inverse transformation, and repeatedly executes the inverse transformation to generate data samples. This process consists of a pair of Markov processes, where the forward process progressively perturbs the data x0 into a stationary distribution x. T The reverse denoising process, on the other hand, progressively denoises the data, bringing the samples closer to the original data distribution. Its goal is to train the denoising model to fit the data distribution, thus achieving the transformation process.

[0067] For the diffusion model DDPM used in this invention, forward noise addition is achieved through... Implementation, where α t and β t These are the parameters in the model that change with time t, and It follows a normal distribution. The reverse stepwise denoising, however, is achieved through... Implementation, where ∈ θ This is the noise calculated based on the model, σ t It is the standard deviation.

[0068] During training, we positively add noise to the data sample x0 based on the randomly sampled time term t to train the model's denoising ability. After the model training is complete, we first start from x... T ~∈ Sample an initial noisy sample, and then use the denoising process p(x) t-1 |x t The noise is gradually reduced to generate samples that conform to the data distribution.

[0069] The calculation of the total loss function between the predicted sequence information and the true sequence information specifically includes the following methods:

[0070] The total loss function for the predicted sequence information and the true sequence information is calculated as follows:

[0071]

[0072] The The sum of the predicted sequence information and the actual sequence information is the total loss function. For sequence cross-entropy loss, Let the energy loss function be... For MSA cross-entropy loss, α sequence α Energy and α msa α is the weighting coefficient. sequence =2, α Energy =1, α msa =1.

[0073] Sequence cross-entropy loss is used to evaluate the degree of sequence recovery at the residue level, the energy loss function represents the energy stability of the sequence system in the model, and MSA cross-entropy loss is used to evaluate the accuracy of MSA reconstruction.

[0074]

[0075] Specifically, for sequence cross-entropy loss, we calculate the cross-entropy loss between the sequence and the probability distribution obtained from a single representation. In the corresponding formula, L is the sequence length, c is the number of amino acids plus the spacer character, and p... sequence (x i c) represents the predicted probability distribution of the denoised sequence.

[0076] The model represents the true sequence distribution. The denoising loss is used to evaluate the model's performance in the noise removal process. By calculating the difference between the generated sequence and the target sequence in the denoising step, the model is guided to gradually learn how to recover the original sequence.

[0077]

[0078] This formula calculates the cross-entropy loss at the mask position in MSA to measure the difference between the probability distribution predicted by the model and the true label. In the formula, c represents the 20 amino acids plus the spacer character, and N... mask p is the total number of masks. msa (x i c) represents the MSA probability distribution of the predicted mask positions. This represents the true MSA distribution.

[0079] In MSA, each sequence has a 15% probability of being masked, and the replacement method is the same as the classic replacement method:

[0080] There is a 10% probability that it will be replaced with a random amino acid;

[0081] There is a 10% probability that it will be replaced with an amino acid from the MSA feature at that position;

[0082] There is a 10% probability that it will remain unchanged;

[0083] With a 70% probability, these masked locations are replaced with masked markers, which are the model's prediction targets.

[0084]

[0085] in

[0086] The above formula measures the Hamiltonian energy term on the MSA, composed of a single-unit energy term and a two-unit energy term, reflecting the influence of the interaction between single and two points. N represents the total number of MSAs, and L represents the sequence length. The single-unit energy term in the formula is the bias term b, derived from the predicted MSA probability distribution p. msa The formula is derived from the average probability distribution of MSA at each position, reflecting the characteristics of each position. The two-body energy term comes from the amino acid pair matrix W obtained through pairwise characterization, reflecting the interactions between different positions.

[0087] In the first stage, the loss function is: In the second stage, a generative model is added for training: The model's representational and generative capabilities are trained in two phases, respectively.

[0088] There are other methods in the existing technology for calculating the total loss function between predicted sequence information and true sequence information, which will not be elaborated here.

[0089] To avoid overfitting during training, early stopping and regularization techniques were employed. Furthermore, data augmentation methods were used to generate sequences, enriching the training dataset and improving the model's generalization ability.

[0090] Among them The added regularization uses the LH regularization method, with a coefficient γ of 0.01, and is defined as follows: in W is the amino acid pair matrix calculated for pairwise characterization, M is the sum of each row or column, and γ is a user-defined hyperparameter. This method is existing technology and will not be elaborated further here.

[0091] The final evaluation method of this invention includes evolutionary guidance generation for specific MSAs and conditional generation based on specific functional motifs, including the reversion rate of generated sequences, the co-evolutionary information of the generated sequences as a whole, the accuracy of the generated sequence structure prediction, the diversity of generated sequences, novelty, etc.

[0092] The datasets used in this invention for evaluation include some structures from the adenosine kinase dataset, protease dataset, CASP14 dataset, and PDB dataset.

[0093] Example 2:

[0094] The difference from Example 1 is that in step S1, the input MSA needs to be sampled, and the sampling requirement is a coverage of 0.75 or higher; the character dimension of the sequence is 21, and the characters of the sequence include 20 standard amino acids and spacer characters.

[0095] The sequence x0 in this module is sampled from the original MSA, but with stricter requirements, demanding a coverage of at least 0.75. The sequence has a character dimension of 21, including 20 standard amino acids and spacer characters. This allows the model to generate placeholder spacer characters during sequence generation, which can be modified to generate sequences of different lengths. After calculating the single representation, we obtain the sequence probability distribution through an additional linear layer. During training, we use this probability distribution and the true sequence probability distribution to calculate the loss function.

[0096] Example 3:

[0097] The difference from Example 2 is that in step S1, the pairwise update module includes triangular update, triangular attention, and a transition layer; in step S1, the MSA update module includes row attention and a transition layer. Step S1 iterates for 6 layers before proceeding to step S2.

[0098] In step S2, the method for generating the single representation is to generate a single representation for each noisy sequence using a 12-layer adaptive layer normalization diffusion transformer.

[0099] Example 4:

[0100] The difference from Example 3 is that step S1 is iterated 12 times before step S2 is performed;

[0101] In step S2, the single representation is generated for each noisy sequence using a 24-layer adaptive layer normalization diffusion transformer. Paired representations are corrected using triangular updates and triangular attention layers in Alphafold, and the MSA representation is updated using simplified bias row attention. This process is iterated over 12 layers to obtain refined paired information for subsequent modules. The transition layer, consisting of layer normalization, three linear layers, and a nonlinear activation function, primarily serves to transmit information. Existing technologies offer other methods for updating MSA and paired representations; for example, row attention can be used with simpler or more complex network architectures, and steps can be reduced, such as omitting triangular attention. The MSA and paired representation update method defined in this invention, while updating MSA and paired representations, also effectively considers spatial relative position information using triangular relationships, making the model more scientifically compliant with geometric constraints.

[0102] The number of iterations for steps S1 and S2 can be chosen to be other numbers. The limitation of the number of iterations for steps S1 and S2 in this invention can extract pairwise representations with good interpretability with a lower number of parameters.

[0103] Example 5:

[0104] The difference from Example 3 is that step S1 is iterated repeatedly for 48 layers before step S2 is performed;

[0105] In step S2, the method for generating the single representation is to generate a single representation for each noisy sequence using a 72-layer adaptive layer normalization diffusion transformer.

[0106] Example 6:

[0107] The difference from Example 3 is that step S1 is iterated repeatedly for 72 layers before step S2 is performed;

[0108] In step S2, the method for generating the single representation is to generate a single representation for each noisy sequence using a 96-layer adaptive layer normalization diffusion transformer.

[0109] Example 7:

[0110] The difference from Example 3 is that step S1 is iterated repeatedly for 84 layers before step S2 is performed;

[0111] In step S2, the method for generating the single representation is to generate a single representation for each noisy sequence using a diffusion transformer with 120 layers of adaptive layer normalization.

[0112] Example 8:

[0113] The difference from Embodiment 4 is that in step S2, the denoising module includes an attention layer and a transition layer. Other methods exist in the prior art for updating a single representation, such as changing the attention layer to a gated attention layer and the transition layer to an MOE or a gated transition layer. Due to limitations in sequence length and the similarity of MSA sequences, the need for a better architecture to handle ultra-long sequences is lower. Therefore, the denoising method defined in this invention can efficiently and accurately denoise noisy sequences.

[0114] Example 9:

[0115] The difference from Example 1 is that in step S2, the single representation is presented as a JAX computation graph, which is obtained by vectorizing the single representation using vmap. After one-hot encoding of the sequence sampled from MSA, the vectorization operation in the deep learning framework is fully utilized in the noise-adding part to process it in batches. The batches of noisy single representations are vectorized by vmap, and compiled into a single JAX computation graph, reducing the overhead of loops and speeding up the execution speed. This allows for efficient computation through parallelization and automatic batch processing. Each batch of data corresponds to only one pair of information from MSA, and the corresponding pair of representations are not batch processed to save GPU memory.

[0116] Example 10:

[0117] A sequence diffusion generation method based on multiple sequence alignment is provided. After training using any one of the sequence diffusion training methods based on multiple sequence alignment shown in Examples 1 to 9,

[0118] If the original sequence is unknown, then in step S1, input an unmasked MSA, and in step S2, input a random Gaussian distribution as the initial sequence, and perform a reverse process to gradually denoise and generate a sequence.

[0119] If the protein motif is known in the original sequence, then in step S1, the MSA without masking is input, and in step S2, the protein motif is input as the initial sequence. The reverse process is then performed to gradually remove noise and complete the remaining part to generate the sequence.

[0120] If the original sequence is known, then in step S1, input the MSA without masking.

[0121] In step S2, the protein motif of the original sequence is input as the initial sequence, and the reverse process is performed to gradually remove noise and complete the remaining part to generate the sequence.

[0122] Alternatively, in step S2, the original sequence can be input as the initial sequence, and noise can be added first, followed by a reverse process to gradually denoise and generate a new sequence.

[0123] During the generation process, we use the argmax of the obtained sequence probability distribution as the generated sequence.

[0124] Example 11:

[0125] An electronic device, comprising:

[0126] processor and

[0127] The memory stores executable code, which, when executed by the processor, causes the processor to execute a sequence diffusion training method based on multiple sequence alignment as described in any one of embodiments 1 to 9.

[0128] Example 12:

[0129] An electronic device, comprising:

[0130] processor and

[0131] The memory stores executable code, which, when executed by the processor, causes the processor to perform a sequence diffusion generation method based on multiple sequence alignment as described in Example 10.

[0132] By training and evaluating the model of this invention, a series of important experimental results were obtained. These results demonstrate the effectiveness and application potential of the model in biological sequence analysis.

[0133] like Figure 1 As shown, the first stage uses MSA data for initial training to generate a preliminary sequence distribution. Experiments show that after 10,000 training steps, the sequence distribution tends to stabilize, and the mutation and deletion probabilities reach reasonable levels. The second stage further optimizes the sequences through a generation task. Experiments show that guided generation has a significant effect on sequence optimization in the early stages, with a smaller impact in the later stages. For example, at 1,000 training steps, the sequence recovery rate is 0.7-0.8, while at 2,000 training steps, the recovery rate reaches over 0.95.

[0134] To gain a deeper understanding of the model's learning process and feature extraction capabilities, this invention visualizes the model's learned latent representations and observes the differences between the representations and the true conserved information and co-evolutionary information. Figure 2 and Figure 3 This displays the model's visualization of the input sequence (pdb_id:6p1e). Figure 2 This indicates that the pairwise representations of the model have successfully extracted the co-evolutionary contact information hidden in the MSA information; Figure 3This demonstrates the similarity between the MSA representation learned by the model and the actual site frequency distribution, which fully illustrates that the present invention decomposes and represents the conserved and co-evolutionary information in evolutionary information, and can effectively capture the biological correlation between sequences.

[0135] In terms of sequence generation, the model can achieve evolution-based sequence generation, and since the training data contains gap characters, it can generate sequences of different lengths. Figure 4 and Figure 5 The results show the predictive performance of generated sequence structures on two enzymes of different lengths. While generating predictable sequences, the two sequences have low similarity to the original natural sequences, at 0.43 and 0.5 respectively. Furthermore, the similarity between the different sequences varies considerably, demonstrating a certain degree of diversity.

[0136] This invention also possesses the capabilities of motif scaffolding and inpainting. Motif scaffolding aims to design protein sequences with stable structures and functions around a specific functional motif. A motif typically refers to a short amino acid sequence in a protein that has a specific function or structural feature, such as the active site of an enzyme or the interface of a protein-protein interaction. Protein inpainting refers to filling in missing information segments in a protein sequence, often applicable to needs such as introducing mutations or inserting new fragments in certain regions of a target protein, or improving the distribution of secondary structures.

[0137] We analyzed the protease case, such as Figure 6 and Figure 7 As shown, in the experiments on the generation domain, the secondary structures of the generated sequences exhibit a trend of diversity and a reduced loop ratio. In terms of scores, PMSA scoring, ESM2 scoring, and structure prediction plddt (local prediction confidence) are all significantly improved, indicating high mutation tolerance. Experiments demonstrate that this method can significantly improve the scoring and quality of the generated sequences.

[0138] This invention proposes a two-stage training approach. The first stage obtains an efficient representation extraction model that extracts paired information from sequences / MSAs. The second stage involves rapid fine-tuning to obtain a generative model. This first-stage model can serve as a base model for fine-tuning various sub-task data. Simultaneously, this invention proposes an evolutionary information modeling paradigm, independently diffusing each homologous sequence within the model, leveraging the concept of intervals in evolutionary information to generate sequences of different lengths. This invention introduces deep mining methods for co-evolutionary information into deep learning, providing end-to-end interpretability during large-scale training. Visual representations help accurately capture co-mutations between functional sites, supporting complex system design, verifying design logic, and reducing experimental trial-and-error costs.

[0139] The above description is merely a preferred embodiment of the present invention, and the scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A multiple sequence alignment based sequence diffusion training method, characterized by The method comprises the following steps: S1 evolutionary encoder: input MSA and initial pairwise representation, which is in the form of sequence distance matrix, mask and embed the MSA to obtain MSA representation; The MSA representation and initial pairwise representation are corrected by a pairwise update module and an MSA update module to obtain updated MSA and pairwise representation; S2 diffusion decoder: randomly select an initial sequence from the MSA, add noise to the initial sequence in batches to obtain a noisy sequence, vectorize the noisy sequence to obtain a single representation, fuse the single representation and the updated pairwise representation obtained in the S1 step, and input a time step, denoise by a denoising module, and output an updated single representation, and obtain predicted sequence information according to the updated single representation and the MSA; The total loss function of the predicted sequence information and the true sequence information is calculated to correct the denoising module, the update module and the MSA update module; Repeat the S1 and S2 steps until a preset training end condition is reached.

2. The sequence diffusion training method based on multiple sequence alignment according to claim 1, wherein, In the S1 step, the input MSA needs to be sampled, and the sampling requirement is that the coverage rate is above 0.75; the character dimension of the sequence is 21, and the characters of the sequence include 20 standard amino acids and a spacer character.

3. The method of claim 1, wherein, In the S1 step, the pairwise update module includes a triangular update, a triangular attention and a transition layer; in the S1 step, the MSA update module includes a row attention and a transition layer.

4. The method of claim 1, wherein, In the S2 step, the denoising module includes an attention layer and a transition layer.

5. The method of claim 1, wherein, The S1 step is repeated for 6-84 layers, and then the S2 step is performed; In the S2 step, the single representation is generated by a diffusion transformer with adaptive layer normalization of 12-120 layers for each noisy sequence.

6. The method of claim 1, wherein, The S1 step is repeated for 12-48 layers, and then the S2 step is performed; In the S2 step, the single representation is generated by a diffusion transformer with adaptive layer normalization of 24-72 layers for each noisy sequence.

7. The method of claim 1, wherein, In the S2 step, the single representation is in the form of a JAX computation graph, and the JAX computation graph is obtained by vectorizing the single representation by vmap.

8. A sequence diffusion generation method based on multiple sequence alignment, wherein any one of the sequence diffusion training methods based on multiple sequence alignment in claims 1-7 is used to complete training, If the original sequence is unknown, the MSA without masking is input in the S1 step, and a random Gaussian distribution is input as the initial sequence in the S2 step, and the sequence is generated by gradually denoising in the reverse process; If the original sequence is known protein motif, the MSA without masking is input in the S1 step, and the protein motif is input as the initial sequence in the S2 step, and the sequence is generated by gradually denoising to complete the remaining part in the reverse process; If the original sequence is known, the MSA without masking is input in the S1 step, In the S2 step, the protein motif of the original sequence is input as the initial sequence, and the sequence is generated by gradually denoising to complete the remaining part in the reverse process; Or in the S2 step input the original sequence as the initial sequence, first add noise and then perform the reverse process of step-by-step denoising to generate the sequence.

9. An electronic device, comprising: Comprising: a processor and a memory storing executable code that, when executed by the processor, causes the processor to perform a sequence diffusion training method based on multiple sequence alignment according to any one of claims 1-7.

10. An electronic device, comprising: Comprising: a processor and a memory storing executable code that, when executed by the processor, causes the processor to perform a sequence diffusion generation method based on multiple sequence alignment according to claim 8.

Images