Protein sequence and structure co-design method and device, computer equipment and medium

Abstract

The invention belongs to the technical field of bioengineering and drug development, and provides a protein sequence and structure co-design method and device, computer equipment and a medium, and the method comprises the steps: inputting initial noise which comprises a discrete protein sequence Token, a discrete structure Token and a discrete mask Token; based on a generative flow model, a continuous time Markov chain is subjected to iteration of a plurality of time steps to realize gradual generation from initial noise to a target protein sequence and structure, discrete structures in the target protein sequence and structure are reduced into three-dimensional atomic coordinates, and a final protein structure is generated. Sequence and structural design are coupled together, and high-quality generation from initial conditions to target protein is achieved.

Description

Technical Field

[0001] The present invention mainly relates to the fields of bioengineering and drug development technology, and in particular to a protein sequence and structure co-design method, device, computer equipment and medium. Background Art

[0002] Protein design is a key area in bioengineering and drug development, which aims to achieve specific biological functions by designing new protein sequences and structures. Traditional protein design methods are mainly divided into two categories: directed evolution and rational design. Directed evolution gradually optimizes the function of proteins by simulating the natural selection process, but this method is time-consuming and costly. Rational design relies on energy functions and geometric constraints to optimize the structure of proteins, but this method usually requires a lot of computing resources and has limited effectiveness in designing complex proteins.

[0003] In recent years, with the development of artificial intelligence technology, generative models have been widely used in protein design. In particular, protein language models (such as ESM, ProtGPT2, etc.) and diffusion models (such as RFDiffusion, FrameDiff, etc.) have made significant progress in protein sequence and structure generation. These models can generate protein sequences or structures with specific functions, greatly improving the efficiency of protein design. However, existing methods still have some limitations: 1. Separation of sequence and structure design: Most existing methods adopt a two-stage design strategy, that is, first designing the protein backbone structure and then designing the corresponding sequence. This separate design approach cannot fully capture the complex relationship between sequence and structure, resulting in poor performance of the designed protein in practical applications.

[0004] 2. Limitations of continuous space modeling: Generative methods such as diffusion models usually model protein structures in a continuous three-dimensional space. Although they can generate high-quality structures, they are less efficient when dealing with discrete amino acid sequences and it is difficult to achieve joint optimization of sequence and structure.

[0005] These limitations restrict the application of existing methods in protein design, especially in complex tasks that require simultaneous optimization of sequence and structure. Therefore, a design method that can effectively jointly optimize protein sequence and structure is urgently needed to improve the accuracy and efficiency of protein design. Summary of the invention

[0006] The purpose of the present invention is to provide a protein sequence and structure co-design method, device, computer equipment and medium, aiming to solve the problems of separation of sequence and structure design, low efficiency of continuous space modeling and insufficient multimodal learning in existing protein design methods.

[0007] To achieve the above purpose, the technical solution adopted by the present invention is as follows: In one aspect, the present invention provides a method for co-designing protein sequence and structure, comprising: Input initial noise ,in and Represent discrete protein sequence tokens and structure tokens respectively. The protein sequence in the initial noise or / and structure Contains a mask Token, indicating that the protein sequence and / or structure information at the position corresponding to the mask Token is hidden; Based on the generative flow model, the continuous-time Markov chain is iterated through T time steps to achieve the initial noise To the target protein sequence and structure The gradual generation of The mask token is not included, indicating that the target protein sequence and the protein sequence at all positions in the structure and structure The information has been confirmed; Target protein sequence and structure The discrete structure in the protein is reduced to three-dimensional atomic coordinates to generate the final protein structure.

[0008] Furthermore, a protein sequence and structure co-design device is provided, comprising: Input module, used to input initial noise ,in and Represent discrete protein sequence tokens and structure tokens respectively. The protein sequence in the initial noise or / and structure Contains a mask Token, indicating that the protein sequence and / or structure information at the position corresponding to the mask Token is hidden; The generation module is used to generate the initial noise through the continuous time Markov chain through T time steps based on the generative flow model. To the target protein sequence and structure The gradual generation of The mask token is not included, indicating that the target protein sequence and the protein sequence at all positions in the structure and structure The information has been confirmed; Decoding module, used to convert target protein sequence to structure The discrete structure in the protein is reduced to three-dimensional atomic coordinates to generate the final protein structure.

[0009] On the other hand, the present invention provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above-mentioned protein sequence and structure co-design method when executing the computer program.

[0010] On the other hand, the present invention provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of the above-mentioned protein sequence and structure co-design method when the computer program is executed by a processor.

[0011] On the other hand, the present invention provides a computer program product, which is stored on a computer-readable storage medium and includes computer instructions, which, when executed by a processor, enable a computer device to implement the steps of the above-mentioned protein sequence and structure co-design method.

[0012] Compared with the prior art, the technical effects of the present invention are: By introducing a discrete generative flow model, the present invention can jointly generate protein sequences and structures in a discrete space, thereby significantly improving the accuracy, efficiency and diversity of protein design.

[0013] The present invention supports de novo design and conditional generation (such as specific fragment completion), and can be widely used in drug development, industrial enzyme design, and basic biological research. By introducing asynchronous generation strategies and time characteristics, the consistency of the generated sequence and structure is significantly improved, and the scRMSD value (the lower the index, the higher the consistency) can reach an average of 2.7Å, which is about 8 times lower than the ESM3 method. At the same time, the generated protein structure is closer to the natural protein. On the other hand, it shows higher stability and design capabilities when dealing with long sequence proteins (400 to 500 residues). BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0015] Figure 1 It is a schematic diagram of the principle of a protein sequence and structure co-design method provided by an embodiment; Figure 2 It is a schematic diagram of the architecture of a generative flow model provided by an embodiment. DETAILED DESCRIPTION

[0016] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0017] Reference Figure 1 , Figure 1 The present invention provides a protein sequence and structure co-design method, comprising: Input initial noise ,in and Represent discrete protein sequence Tokens and structure Tokens respectively. The protein sequence in the initial noise or / and structure Contains a mask Token, indicating that the protein sequence and / or structure information at the position corresponding to the mask Token is hidden; Based on the generative flow model, the continuous-time Markov chain is iterated through T time steps to achieve the initial noise To the target protein sequence and structure The gradual generation of The mask token is not included, indicating that the target protein sequence and the protein sequence at all positions in the structure and structure The information has been confirmed; Target protein sequence and structure The discrete structure in the protein is reduced to three-dimensional atomic coordinates to generate the final protein structure.

[0018] The function of the generative flow model is to predict the real token corresponding to the masked token in the protein sequence and structure in the initial noise. If the protein sequence and structure in the initial noise are all masked tokens, unconstrained generation is performed. If the protein sequence and structure contain some non-masked tokens, these non-masked tokens are used as input conditions, and the model only predicts the masked tokens, and the given non-masked tokens (i.e., the given protein sequence tokens and structure tokens) remain unchanged.

[0019] Figure 1The right side of the middle represents that the model can complete different tasks. It includes four parts from top to bottom. The first part includes the structure and sequence of the protein, which represents the model's unconstrained generation and generates the protein's skeleton structure and sequence at the same time. The sequence is represented by a string, and each letter represents an amino acid. Different letters represent different types of amino acids. For example, M represents methionine, K represents lysine, W represents tryptophan, E represents glutamate, T represents threonine, and so on. The second part only includes the structure of the protein, which represents the model generating the structure based on the sequence. The third part only includes the sequence of the protein, which represents the model generating the sequence based on the skeleton structure. The same sequence is represented by a string, and each letter represents an amino acid. Different letters represent different types of amino acids. The fourth part represents the model generating a complete protein based on the given partial sequence and structure completion. Each letter represents an amino acid, and different letters represent different types of amino acids.

[0020] In order to uniformly process the sequence and structure of proteins, the present invention represents proteins as discrete protein sequences and structures. The standard amino acid encoding method is used to represent protein sequences as discrete symbol sequences to obtain protein sequence tokens; the three-dimensional atomic coordinates of proteins are discretized into structural tokens using vector quantization-variational autoencoder (VQ-VAE), while providing the ability to restore from discrete tags to three-dimensional coordinates.

[0021] Reference Figure 2 , Figure 2 Schematic diagram of the architecture of a generative flow model provided by an embodiment. The present invention implements the generation of initial noise by iterating a continuous-time Markov chain through T time steps. To the target protein sequence and structure In the gradual generation process, at each time step, the neural network is based on Protein sequence and structure data at the moment predict Protein sequence and structure data at the moment , , Specifically, it includes: Will Protein sequence and structure data at the moment Protein sequences in and structure Mapping to vector representation; Through multi-frequency sine and cosine functions The time step corresponding to the moment is encoded as a high-dimensional vector to capture the periodic characteristics of time information, as Fourier time series characteristics of the moment; Get Vector representation of protein sequence and structure fused with Fourier temporal features at every moment; Using neural network based The vector representation of protein sequence and structure fused with Fourier temporal features is predicted Protein sequence and structure data at the moment ; Continue to iterate until the maximum number of iterations is reached, then the iteration stops and the final target protein sequence and structure are generated. , The maximum number of iterations is determined based on actual conditions or experience, and is generally set to 200, 400 or 500 steps.

[0022] The protein sequence and structure are mapped to vector representations using the one-hot encoding process. Proteins are composed of 20 amino acids, each with a unique symbol. When using one-hot encoding, a 20-dimensional vector is assigned to each amino acid, each vector corresponds to an amino acid, and only one position in each vector is 1, while the other positions are 0. If a protein sequence has m amino acids, then the entire protein sequence will be an m×20 matrix. Similarly, protein structures correspond to multiple structural types. When using one-hot encoding, a unique integer index is assigned to each structural category; then, an all-zero vector with a length equal to the number of structural types is initialized, and the corresponding position in the vector is set to 1 according to the index of the structural type, and the remaining positions remain 0. This encoding method converts discrete values ​​into unique binary vectors, making it easier for machine learning models to process categorical data. After one-hot encoding, a linear transformation is used to convert the sparse one-hot vector into a dense vector.

[0023] An embodiment proposes that Fourier time series characteristics of the moment , as shown below:

[0024] in, is a set of predefined frequencies used to generate sine and cosine components of different periods, , ,in Indicates the number of current iterations. This encoding method can effectively capture the continuity and periodicity of time steps and provide richer time information for the generation model.

[0025] Without loss of generality, the frequency The expression is as follows:

[0026] ind The value of is Fourier time series feature Embedding( t ) is divided by 2, Fourier time series feature Embedding( t ) is consistent with the vector dimension obtained by protein sequence and structure encoding. , is a hyperparameter set without loss of generality. The general value is 10 -2 , The possible value is 10 3 .

[0027] Fourier time series features convert time steps into Encoded as a high-dimensional vector, it can effectively capture multi-scale temporal dependencies while providing a smooth and continuous temporal representation. This encoding method not only enhances the model's ability to model short-term and long-term dynamics, but is also particularly suitable for processing periodic data. In addition, Fourier features are efficient to calculate and do not require additional learnable parameters, which significantly improves the efficiency of temporal information utilization in the diffusion model, thereby improving generation quality and training stability.

[0028] The feature vector of protein sequence and structure fused with Fourier time series features is input into the bidirectional Transformer. The bidirectional Transformer uses its powerful self-attention mechanism to capture the global dependencies between positions in the protein sequence. For protein sequences, the bidirectional Transformer can effectively learn the interactions and structural information between amino acids.

[0029] The present invention is based on a bidirectional Transformer as a backbone network combined with Fourier temporal features, guiding the model to dynamically adjust in each generation process, and outputting the category distribution of protein sequences and structures.

[0030] Figure 2 In the Medium Sampling " represents the process of sampling from the conditional distribution. The generative flow model distributes the probability according to the condition ,current Protein sequence and structure data at the moment (including protein sequence and structure information) and model parameters , predict the next moment Protein sequence and structure data at the moment (including protein sequence and structure information). This process can be formally expressed as ,in The model parameters are represented by The category distribution predicted by the neural network, Represents the unique hot vector of the masked Token. At this moment, if a position in the protein sequence or structure is a mask Token, then At this moment, there will be The probability of is sampled from the category distribution predicted by the model, and the sampled data is used as the data at the mask Token position at the next moment. The probability of keeping the mask Token unchanged, so we get the next moment Protein sequence and structure data at the moment .

[0031] The prediction is based on the vector representation of protein sequence and structure fused with Fourier temporal features. Protein sequence and structure data at the moment In the process, an asynchronous generation strategy is adopted to give priority to generating protein sequences in each generation step, and then use the generated protein sequences to guide the generation of structures. This strategy effectively improves the coupling degree and generation consistency of sequence and structure.

[0032] The asynchronous generation strategy refers to generating sequence data of two modalities, protein sequence and structure, in an alternating order. Specifically, the model first generates the first element of the protein sequence, then the first element of the structure, then the second element of the protein sequence, and so on. This strategy allows the model to dynamically consider the dependency between the two modalities in the generation process, thereby better capturing the association between the modalities.

[0033] The generative flow model in the present invention realizes the step-by-step generation from initial noise to target sequence and structure labels through a continuous-time Markov chain. The generation process of protein sequence and structure is dynamically coupled through a joint flow model. Specifically, during the generation process, protein sequence and structure are mapped to a common latent space through a neural network for information interaction to ensure mutual influence and information sharing between the two. An asynchronous generation strategy is adopted to give priority to generating protein sequence in each generation step, and then the generated protein sequence is used to guide the generation of protein structure. This strategy effectively improves the coupling degree and generation consistency of sequence and structure.

[0034] The present invention effectively solves the problems of low sequence and structure generation matching, low design efficiency and insufficient long sequence protein generation capability in the prior art by constructing a protein sequence structure co-design system based on a generative flow model. The joint flow mechanism couples the sequence and structure generation processes of proteins, and dynamically generates sequence and structure tags alternately through an asynchronous generation strategy, which closely associates the two and significantly improves the consistency and accuracy of generation. Experiments show that the consistency index (scRMSD) of the present invention in unconditional generation tasks is about 8 times better than that of existing methods. The generated long sequence protein structure is stable and highly natural, and its template modeling score (pTM) distribution is closer to natural proteins.

[0035] In addition, the present invention has significant advantages in multifunctional design support, and can realize de novo design, conditional completion and function-driven generation according to different task requirements. By optimizing the VQ-VAE model's discretization representation and generation strategy for the structure, the present invention shows a high success rate in specific fragment completion tasks, while improving generation efficiency and resource utilization. Compared with traditional methods, the present invention significantly shortens the calculation time while ensuring generation quality, providing an efficient and reliable technical means for the application of protein design in drug development, industrial enzyme design and other fields.

[0036] Based on the generative flow model, the present invention constructs an efficient and flexible protein design system. The system includes a data preprocessing module, a generation module and a decoding module. The sequence and structure design are coupled together through the joint flow technology to achieve high-quality generation from initial conditions to target proteins. Specifically, it includes: Input module, used to input initial noise ,in and Represent discrete protein sequence Tokens and structure Tokens respectively. The protein sequence of the initial noise or / and structure Contains a mask Token, indicating that the protein sequence and / or structure information at the position corresponding to the mask Token is hidden; The generation module is used to generate the initial noise through the continuous time Markov chain through T time steps based on the generative flow model. To the target protein sequence and structure The gradual generation of The mask token is not included, indicating that the protein sequence and structure information of all positions in the target protein sequence and structure have been determined; Decoding module, used to convert target protein sequence to structure The discrete structure in the protein is reduced to three-dimensional atomic coordinates to generate the final protein structure.

[0037] The input module supports the input of functional requirements, sequence fragments or structural fragments of the target protein, providing flexible conditional generation capabilities. The generation module uses a bidirectional Transformer as the backbone network, combined with Fourier temporal features, to guide the model to dynamically adjust in each step of the generation process; the decoding module restores discrete structural markers to three-dimensional coordinates through the structure decoder to generate the final protein structure.

[0038] The generative model provides flexible conditional generation capabilities, thus enabling it to accomplish a variety of tasks. Specifically, if the initial noise Only mask tokens are included (i.e., protein sequence and structure are unknown), and the model will generate unconditionally; if the initial noise Protein sequence Known structure Unknown, the model predicts the structure; conversely, if the initial noise Protein structure Known protein sequence unknown, the model will perform protein sequence design; finally, if the initial noise Protein structure and sequence All are partially known, and the model will be designed to be complete.

[0039] The model training process is divided into two stages: pre-training and fine-tuning. The pre-training stage uses large-scale protein databases (such as MGnify30) to ensure that the model has a wide range of generation capabilities. The fine-tuning stage is further trained on high-quality PDB datasets to optimize the ability of the generated results to be close to the natural structure. The inference stage uses multi-step iterative sampling to alternately generate sequence and structure tags to ensure that the generated proteins have high consistency and high accuracy.

[0040] On the other hand, the present invention provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the protein sequence and structure co-design method provided in any of the above embodiments are implemented. The computer device can be a server. The computer device includes a processor, a memory, a network interface and a database connected via a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store sample data. The network interface of the computer device is used to communicate with an external terminal via a network connection.

[0041] On the other hand, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the protein sequence and structure co-design method provided in any of the above embodiments.

[0042] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0043] Matters not covered by the present invention are known technologies.

[0044] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0045] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention. It should be pointed out that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

[0046] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A protein sequence and structure co-design method, characterized in that: include: Input initial noise ,in and Represent discrete protein sequence tokens and structure tokens respectively. The protein sequence in the initial noise or / and structure Contains a mask Token, indicating that the protein sequence and / or structure information at the position corresponding to the mask Token is hidden; Based on the generative flow model, the continuous-time Markov chain is iterated through T time steps to achieve the initial noise To the target protein sequence and structure The gradual generation of The mask token is not included, indicating that the target protein sequence and the protein sequence at all positions in the structure and structure The information has been confirmed; Target protein sequence and structure The discrete structure in the protein is reduced to three-dimensional atomic coordinates to generate the final protein structure.

2. The protein sequence and structure co-design method according to claim 1, characterized in that: The continuous-time Markov chain is iterated through T time steps to achieve the initial noise To the target protein sequence and structure In the gradual generation process, at each time step, the neural network is based on Protein sequence and structure data at the moment predict Protein sequence and structure data at the moment .

3. The protein sequence and structure co-design method according to claim 2, characterized in that: according to Protein sequence and structure data at the moment predict Protein sequence and structure data at the moment ,include: Will Protein sequence and structure data at the moment Protein sequences in and structure Mapping to vector representation; Through multi-frequency sine and cosine functions The time step corresponding to the moment is encoded as a high-dimensional vector to capture the periodic characteristics of time information, as Fourier time series characteristics of the moment; Get Vector representation of protein sequence and structure fused with Fourier temporal features at every moment; Using neural network based The vector representation of protein sequence and structure fused with Fourier temporal features is predicted Protein sequence and structure data at the moment ; Continue to iterate until the maximum number of iterations is reached, then the iteration stops and the final target protein sequence and structure are generated. , .

4. The protein sequence and structure co-design method according to claim 3, characterized in that: Fourier time series characteristics of the moment , as shown below: in, is a set of predefined frequencies used to generate sine and cosine components of different periods, , ,in Indicates the current iteration number.

5. The protein sequence and structure co-design method according to claim 4, characterized in that: frequency The expression is as follows: in d The value of is Fourier time series feature Embedding( t ) is divided by 2, Fourier time series feature Embedding( t ) is consistent with the vector dimension obtained by protein sequence and structure encoding. , is a hyperparameter of the setting.

6. The protein sequence and structure co-design method according to claim 5, characterized in that: The value is 10 -2 , The value is 10 3 .

7. The protein sequence and structure co-design method according to claim 3 or claim 4 or claim 5 or claim 6, characterized in that: Using neural network based The vector representation of protein sequence and structure fused with Fourier temporal features is predicted Protein sequence and structure data at the moment , this process is expressed as ,in The model parameters are represented by The category distribution predicted by the neural network, Represents the unique hot vector of the masked Token, At this moment, if a position in the protein sequence or structure is a mask token, then At this moment, there will be The probability of is sampled from the category distribution predicted by the model, and the sampled data is used as the data at the mask Token position at the next moment. The probability of keeping the mask Token unchanged, so we get the next moment Protein sequence and structure data at the moment .

8. A protein sequence and structure co-design device, characterized in that: include: Input module, used to input initial noise ,in and Represent discrete protein sequence tokens and structure tokens respectively. The protein sequence in the initial noise or / and structure Contains a mask Token, indicating that the protein sequence and / or structure information at the position corresponding to the mask Token is hidden; The generation module is used to generate the initial noise through the continuous time Markov chain through T time steps based on the generative flow model. To the target protein sequence and structure The gradual generation of The mask token is not included, indicating that the target protein sequence and the protein sequence at all positions in the structure and structure The information has been confirmed; Decoding module, used to convert target protein sequence to structure The discrete structure in the protein is reduced to three-dimensional atomic coordinates to generate the final protein structure.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the protein sequence and structure co-design method as claimed in claim 1 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the protein sequence and structure co-design method as claimed in claim 1 are implemented.

Images