Protein fitness prediction method, device, terminal and storage medium

Abstract

The application relates to the technical field of protein engineering and bioinformatics, and specifically provides a protein fitness prediction method and device, a terminal and a storage medium. The method comprises the following steps: extracting wild-type sequence-level representation and mutant sequence-level representation by using a pre-trained protein language model, and calculating a representation difference vector between the two; deducing coevolution coupling information from multiple sequence alignment information, and constructing pairwise evolutionary constraint information reflecting spatial proximity relationship; then modeling the interaction between the representation difference vector and the pairwise evolutionary constraint information through an attention mechanism, so that the pairwise coevolution information guides the propagation and weighting of the representation difference vector between residues to generate enhanced representation; and inputting the enhanced representation into a downstream prediction head to regress a scalar value as a fitness prediction result. The application models the interaction between the sequence-level difference vector and the pairwise evolutionary constraint, so that the non-additive effect between mutations can be accurately captured without relying on any experimental structure.

Description

Technical Field

[0001] This application relates to the fields of protein engineering and bioinformatics, and in particular to a method, device, terminal and storage medium for predicting protein fitness. Background Technology

[0002] In protein function optimization and directed evolution research, accurately predicting the impact of mutations on protein fitness is crucial. Traditional methods mainly rely on experimental measurements or energy calculations based on physical force fields; however, the former is costly and has low throughput, while the latter is computationally complex and has limited accuracy. In recent years, with the development of deep learning, methods based on pre-trained protein language models have gradually emerged. These methods capture statistical regularities in amino acid sequences through large-scale unsupervised learning, extracting mutation-related characterization information at the sequence level and using it for fitness prediction, thus improving prediction efficiency and accuracy to some extent.

[0003] However, accurate prediction of protein fitness depends on a deep understanding of three complementary information sources: sequence, structure, and evolution. Existing fusion strategies usually process sequence information in parallel with evolutionary / structural information or simply splice them together, lacking effective cross-modal interaction mechanisms and making it difficult to fully explore epistatic effects. At the same time, most high-performance models rely heavily on high-quality experimental resolution of three-dimensional structures as input priors, which limits their application in protein families lacking structural information. Summary of the Invention

[0004] To address the aforementioned issues, this application provides a protein fitness prediction method, device, terminal, and storage medium, aiming to accurately capture the non-additive effects between mutations without relying on any experimental structure, thereby achieving accurate prediction of protein fitness.

[0005] To achieve the objectives of this application, the following technical solution is provided:

[0006] In a first aspect, this application provides a method for predicting protein fitness, comprising:

[0007] Obtain wild-type protein sequences and at least one mutant protein sequence, and use a pre-trained protein language model to extract wild-type sequence-level representations and mutant sequence-level representations from the wild-type protein sequences and the mutant protein sequences, respectively.

[0008] Calculate the characterization difference vector based on the wild-type sequence-level characterization and the mutant sequence-level characterization;

[0009] Multiple sequence alignment information is generated based on the wild-type protein sequence, and then co-evolutionary coupling information is inferred from the multiple sequence alignment information to construct pairwise evolutionary constraint information that reflects spatial proximity relationships.

[0010] By modeling the interaction between the representation difference vector and the pairwise evolutionary constraint information through an attention mechanism, the pairwise co-evolutionary information guides the propagation and weighting of the representation difference vector among residues, so as to capture the epistatic effect between mutations and generate enhanced representations.

[0011] The enhanced characterization is input into a downstream prediction head, which then regresses and outputs a scalar value as the fitness prediction result for the mutant protein.

[0012] A further improvement of this invention is that the step of calculating the representation difference vector based on the wild-type sequence-level representation and the mutant sequence-level representation includes: constructing a fusion feature vector based on the wild-type sequence-level representation and the mutant sequence-level representation through linear interpolation. : ;in, To fuse feature vectors, For mutant sequence-level characterization, For wild-type sequence-level characterization, The interpolation coefficients are used to extract the wild-type baseline signal from the fused feature vector, yielding the original mutation difference vector. : ; for the original mutation difference vector application The activation function performs a non-linear transformation, as shown in the formula:

[0013] ;

[0014] Wherein, scaling factor and offset factor These are learnable parameters, which are generated by a multilayer perceptron (MLP) based on the input. Dynamic prediction of channel features to achieve adaptive adjustment of the activation range of different channel features. This is a sequence-level mutation effect characterization vector.

[0015] A further improvement of the present invention is that the protein language model is ESM-2; the step of extracting wild-type sequence-level representations and mutant sequence-level representations from the wild-type protein sequence and the mutant protein sequence respectively using the pre-trained protein language model includes: for the wild-type protein sequence, using the contextual compilation capability of ESM-2 to perform end-to-end encoding of the wild-type protein sequence, integrating the local context and global semantics of the wild-type protein sequence into a high-dimensional vector, and extracting the wild-type sequence-level representation; for each mutant protein sequence, using the contextual compilation capability of ESM-2 to perform end-to-end encoding of the mutant protein sequence, integrating the local context and global semantics of the mutant protein sequence into a high-dimensional vector, and extracting the corresponding mutant sequence-level representation.

[0016] A further improvement of the present invention is that the step of generating multiple sequence alignment information based on the wild-type protein sequence includes: constructing multiple sequence alignment information for the wild-type protein sequence using a sequence search tool; wherein, using the wild-type protein sequence as the query sequence, a set of homologous protein sequences is obtained from the database through HHblits, and then multiple sequence alignment is performed using MAFFT or Clustal Omega tools to generate multiple sequence alignment information containing sequence conservation and variation patterns.

[0017] A further improvement of this invention is that the step of inferring co-evolutionary coupling information from the multiple sequence alignment information and constructing pairwise evolutionary constraint information reflecting spatial proximity relationships includes: using the multiple sequence alignment information as input to the PconsC4 model to generate co-variation scores of residue pairs; and converting the co-variation scores into a... The contact map matrix is ​​used as the pairwise evolutionary constraint information; wherein... The value of each element in the contact map matrix represents the probability of contact between residues at the corresponding positions, where the sequence length of the wild-type protein is given.

[0018] A further improvement of this invention is that the step of modeling the interaction between the representation difference vector and the pairwise evolutionary constraint information through an attention mechanism, so that the pairwise co-evolutionary information guides the propagation and weighting of the representation difference vector among residues to capture the epistatic effect between mutations and generate enhanced representations, includes: employing a multimodal fusion module to model the interaction between the representation difference vector and the pairwise evolutionary constraint information; the multimodal fusion module includes multiple layers of bidirectional interaction units, each layer of bidirectional interaction units including sequence difference flow and pairwise evolutionary flow;

[0019] In each layer of bidirectional interaction unit, the update of the sequence difference stream is guided by the pairwise evolution stream, and the pairwise evolution stream updates and receives feedback from the sequence difference stream.

[0020] A further improvement of the present invention is that the update of the sequence difference stream is guided by the pairwise evolution stream, comprising: in the sequence difference stream, using the representation difference vector of the current layer input sequence level. As input, feature updates are achieved through a gated row attention module; specifically, during the attention calculation and bias injection process, the representation difference vector is first processed. A linear transformation is performed to generate query vectors, key vectors, value vectors, and gating vectors. When calculating the basic attention score, the contact graph matrix of the pairwise evolutionary flows input to the current layer is directly injected as an attention bias, thus guiding the evolutionary constraints on sequence features. The formula is:

[0021] ;

[0022] In the formula, The element in the i-th row and j-th column of the contact graph matrix is ​​the input to the pairwise evolutionary flow; It is the dimension of the query vector or key vector in the attention head; Let be the attention score matrix, where This represents the degree of attention or dependence weight of the i-th residue on the j-th residue; The query vector matrix (Query) is generated by linear transformation of the representation difference vector, representing the active query features of residues in attention interactions; The key vector matrix Key, generated by linear transformation of the representation difference vector, is used to match the query vector to measure the relevance of the sequence content. The value vector after attention weighted summation is multiplied element-wise with the gated vector, and the output of the gated row attention module is processed by a column attention module. After processing, it is transformed by the position feedforward network and integrated with the residual connection and layer normalization to obtain the output representation of the sequence difference flow.

[0023] The pairwise evolutionary flow updates and receives feedback from the sequence difference flow, including: taking the contact graph matrix input from the current layer as input, updating the relationship representation between residue pairs through a pairwise symmetric attention module; during the update process of the pairwise evolutionary flow, generating a new pairwise representation by performing an outer product mean operation on the output representation of the sequence difference flow, and injecting the new pairwise representation into the calculation of the pairwise symmetric attention module to realize the feedback of sequence features on evolutionary constraints, so as to generate an updated pairwise evolutionary representation for input to the next layer bidirectional interaction unit.

[0024] A further improvement of this invention is that, during model training, the total loss function is calculated based on the fitness prediction results output by the downstream prediction head and the attention map generated in the multimodal fusion module; the total loss function It consists of three weighted parts, as shown in the following formula:

[0025] ;

[0026] in, This is the total loss function; Using ranking loss, based on the fitness prediction results output by the downstream prediction head, for any pair of mutants with different fitness, the incorrect ranking made by the model based on the predicted fitness score is penalized. Using mean squared error, the model prediction score output by the downstream prediction head is found to be close to the true fitness score; To account for structural consistency loss, the attention map learned from the multimodal fusion module is compared with a known protein contact distance map, which is calculated from the protein PDB (Protein Data Bank) structure file. For sorting loss The corresponding loss weight coefficient, Mean square error The corresponding loss weight coefficient, For structural consistency loss The corresponding loss weight coefficient.

[0027] Secondly, this application provides a protein fitness prediction device for implementing the above-described protein prediction method, the device comprising:

[0028] The module is used to acquire wild-type protein sequences and at least one mutant protein sequence, and to extract wild-type sequence-level representations and mutant sequence-level representations from the wild-type protein sequences and the mutant protein sequences, respectively, using a pre-trained protein language model.

[0029] The difference calculation module is used to calculate the characterization difference vector based on the wild-type sequence-level characterization and the mutant sequence-level characterization;

[0030] The evolution extraction module is used to generate multiple sequence alignment information based on the wild-type protein sequence, and then infer co-evolutionary coupling information from the multiple sequence alignment information to construct pairwise evolutionary constraint information reflecting spatial proximity relationships.

[0031] The multimodal fusion module is used to model the interaction between the representation difference vector and the pairwise evolutionary constraint information through an attention mechanism, so that the pairwise co-evolutionary information guides the propagation and weighting of the representation difference vector between residues to capture the epistatic effect between mutations and generate enhanced representations.

[0032] The prediction output module is used to input the enhanced characterization into the downstream prediction head, and the downstream prediction head regresses and outputs a scalar value as the fitness prediction result of the mutant protein.

[0033] Thirdly, this application provides a terminal, the terminal including a memory and one or more processors; the memory stores one or more programs; the programs include instructions for performing the protein fitness prediction method as described above; the processor is used to execute the programs.

[0034] Fourthly, this application provides a computer-readable storage medium storing a plurality of instructions adapted to be loaded and executed by a processor to implement the steps of the protein fitness prediction method described above.

[0035] Compared with the prior art, the present invention has the following beneficial effects:

[0036] The protein fitness prediction method, device, terminal, and storage medium provided in this application calculate the characterization difference vector based on wild-type sequence-level characterization and mutant sequence-level characterization. An interactive modeling mechanism is designed between this sequence-level characterization difference vector and pairwise evolutionary constraint information inferred from MSA (Maximum Sequence Aspect-Oriented Analysis). This provides a context for understanding sequence mutation signals by utilizing spatial constraints provided by co-evolution, thereby effectively modeling non-additive effects between mutations and significantly improving prediction accuracy. Simultaneously, the method completely eliminates the need for experimentally determined three-dimensional structures, overcoming the application bottlenecks of traditional structural methods and enabling its widespread application to almost any protein sequence, greatly expanding its application scope. Attached Figure Description

[0037] The accompanying drawings are provided to further understand this application and form part of the specification. They are used together with the embodiments of this application to explain this application and do not constitute a limitation thereof.

[0038] Figure 1 An optional schematic diagram of the protein prediction method provided in the embodiments of this application;

[0039] Figure 2 This is a schematic diagram of the DeltaFormer model structure provided in the embodiments of this application;

[0040] Figure 3 This is a performance diagram of a random partitioning strategy provided in an embodiment of this application;

[0041] Figure 4 This is a performance diagram illustrating the sequential position-based partitioning strategy provided in an embodiment of this application.

[0042] Figure 5 This is a performance diagram illustrating the partitioning strategy based on sequence position modulus provided in an embodiment of this application.

[0043] Figure 6 The fitness characterization manifold distribution map of the representative protein FKBP3_HUMAN generated based on the t-SNE algorithm is provided for the embodiments of this application.

[0044] Figure 7 The fitness characterization manifold distribution map of the representative protein FECA_ECOLI generated based on the t-SNE algorithm is provided for embodiments of this application. Detailed Implementation

[0045] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0046] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature; in the description of this application, unless otherwise stated, "multiple" means two or more.

[0047] Protein fitness landscape quantitatively describes the complex mapping between protein sequence variants and their functional phenotypes, and is one of the core research questions in molecular biology. Accurate modeling of this landscape is of vital importance for systematically explaining mutation effects and guiding protein engineering (such as directed evolution and artificial intelligence-assisted design).

[0048] Currently, accurate prediction of protein fitness relies on a deep understanding of three complementary sources of information: sequence, structure, and evolution. Sequence-based methods, particularly large protein language models (PLMs), capture functional signals by learning amino acid contextual relationships; structure-based methods utilize three-dimensional spatial information to model residue interactions, but their application is limited by scarce experimentally determined structural data; evolutionary-based methods extract conservation and variation patterns of homologous sequences from multiple sequence alignment (MSA) information as evolutionary constraints.

[0049] While cutting-edge models in the field attempt to integrate these multimodal information modes, they have significant limitations. Some methods (such as SPIRED-Fitness) rely on external structural prediction modules, which not only increases the complexity of the system and the potential risk of error propagation, but also fails to fully exploit the inherent synergistic effects between different information modes. To address the above technical problems, this invention proposes the following technical solutions and corresponding embodiments.

[0050] The following is combined Figures 1 to 7 The illustrated embodiments describe the technical solution of the present invention:

[0051] Example 1

[0052] This application provides an embodiment of a protein fitness prediction method, referring to... Figure 1 As shown, the process includes the following steps S101 to S105:

[0053] Step S101: Obtain the wild-type protein sequence and at least one mutant protein sequence, and use a pre-trained protein language model to extract wild-type sequence-level characterization and mutant sequence-level characterization from the wild-type protein sequence and the mutant protein sequence, respectively.

[0054] In the embodiments of this application, the mutant protein sequence is a variant sequence formed by base / amino acid substitution, insertion or deletion of the wild-type protein sequence, while the wild-type protein sequence is the original sequence.

[0055] In the embodiments of this application, the wild-type protein sequence is used. and a set of mutant protein sequences As input data, a pre-trained protein language model ESM-2 (Evolutionary Scale Modeling-2) is used as the feature encoder for sequence-level representation generation. Specifically, the original wild-type protein sequence is first... and Each mutant protein sequence The sequence is converted into a token format recognizable by the ESM-2 model, preserving the linear arrangement information of each sequence. Then, leveraging the contextual compilation capabilities of ESM-2, end-to-end encoding is performed on each input sequence (wild-type protein sequence / mutant protein sequence). In this embodiment, the ESM-2 model learns long-range dependencies between amino acid residues through a multi-layer Transformer structure, integrating the local context and global semantics of the sequence into a high-dimensional vector; subsequently, for wild-type protein sequences... The corresponding wild-type sequence-level characterization was extracted. For each mutant protein sequence The corresponding mutant sequence-level characterization was extracted. Therefore, by utilizing the biological prior knowledge obtained from pre-training ESM-2 on massive protein sequences, discrete amino acid sequences can be transformed into continuous, quantifiable high-dimensional vectors, enabling wild-type sequence-level characterization. and mutant sequence-level characterization It can objectively reflect the inherent characteristics of sequences at the evolutionary, structural and functional levels, providing a basic characterization for subsequent quantitative analysis of mutation effects.

[0056] Step S102: Calculate the characterization difference vector based on the wild-type sequence-level characterization and the mutant sequence-level characterization.

[0057] In this embodiment, firstly, based on wild-type sequence-level characterization and mutant sequence-level characterization, a fused feature vector is constructed through linear interpolation. : ;in, To fuse feature vectors, For mutant sequence-level characterization, For wild-type sequence-level characterization, These are the interpolation coefficients; then from the fused feature vectors By stripping the wild-type baseline signal, the original mutation difference vector is obtained. : ; For the original mutation difference vector application The activation function performs a non-linear transformation, as shown in the formula: ; where scaling factor and offset factor These are learnable parameters, generated by a Multi-Layer Perceptron (MLP) based on the input. Dynamic prediction of channel features enables adaptive adjustment of the activation range of different channel features, allowing the model to adaptively adjust the activation range of mutation signals, thereby capturing the effect intensity of different mutations more precisely. This is the sequence-level mutation effect characterization vector, i.e., the characterization difference vector. Therefore, the final output is... This is a refined sequence-level mutation effect characterization vector that can finely characterize the effect intensity and pattern differences of different mutations. This characterization difference vector explicitly models the latent space perturbation caused by mutations, avoiding the interference of redundant information caused by directly using the original sequence or residue embedding.

[0058] In the embodiments of this application, The activation function compresses the difference signal to Within the range, extreme noise is suppressed and nonlinear characterization capability is enhanced.

[0059] Step S103: Generate multiple sequence alignment information based on the wild-type protein sequence, and then infer co-evolutionary coupling information from the multiple sequence alignment information to construct pairwise evolutionary constraint information reflecting spatial proximity.

[0060] In this embodiment of the application, the homology sequence search tool HHblits is used to search for a sequence with a length of of Constructing multiple sequence alignment information (MSA). Specifically, using wild-type protein sequences... To retrieve sequences, a set of homologous protein sequences was obtained from the UniProt database using HHblits. Then, MAFFT (Multiple Alignment using Fast Fourier Transform) or Clustal Omega was used to align the sequences, generating multiple sequence alignment information containing sequence conservation and variation patterns. This multiple sequence alignment information is essentially a structured record of sequence variations within the same protein family during evolution. It should be noted that this step does not rely on any experimentally determined three-dimensional structural data.

[0061] In this embodiment, based on multiple sequence alignment information, a pre-trained deep learning model (PconsC4) is used to directly learn the co-evolutionary coupling patterns between residues and infer the coupling strength parameter. This coupling strength parameter characterizes the degree of co-variation or contact probability between any two residue positions (i, j) in the wild-type protein sequence. This coupling strength parameter is then converted into a quantified residue contact map or similar structural bias information to serve as pairwise evolutionary constraint information. Specifically, the coupling strength parameter is organized into a... Contact diagram matrix , to serve as pairwise evolutionary constraint information; among which, The contact map matrix represents the sequence length of the wild-type protein sequence. The value of each element in the contact map matrix indicates the probability of a contact between residues at the corresponding position. It is considered as a predicted residue contact map or evolutionary coupling map (or a similar form of structural bias).

[0062] As a feasible implementation method, PconsC4 is used to calculate the covariance fraction of residue pairs from multiple sequence alignment information to generate a... Contact diagram matrix The contact diagram matrix It can be viewed as a predicted residue contact map or evolutionary coupling map.

[0063] Thus, in the multimodal information fusion framework called DeltaFormer proposed in this embodiment, the traditional paradigm of directly using the original sequence or structural representation is abandoned. Instead, the change in protein state before and after mutation is explicitly modeled by constructing a representation difference vector. On this basis, evolutionary constraint information without experimental structure is introduced to achieve efficient fusion and interactive optimization of sequence-level mutation signals and spatial interactions between residues.

[0064] Step S104: Model the interaction between the representation difference vector and the pairwise evolutionary constraint information through an attention mechanism, so that the pairwise co-evolutionary information guides the propagation and weighting of the representation difference vector among residues, thereby capturing the epistatic effect between mutations and generating enhanced representation.

[0065] In this embodiment, the multimodal fusion module (also referred to as the Coformer module in this embodiment) stacks multiple layers of bidirectional interaction units. Each layer includes a sequence difference stream, a pairwise evolution stream, and a bidirectional interaction mechanism. Iterative optimization based on the bidirectional information stream achieves deep coupling and dynamic updating of sequence difference information (representing the difference vector) and pairwise evolution constraint information. (Refer to...) Figure 2 As shown, the information processing procedure for each layer of bidirectional interaction unit is described in detail below:

[0066] Step S1041, Sequence Differential Stream Update (guided by evolutionary flow): In the sequence differential stream, the representation difference vector of the current layer input sequence level is updated. As input, feature updates are achieved through a gated row attention module. Specifically, during attention calculation and bias injection, the first step is to... Perform a linear transformation to generate a query vector. Key vector Value vector and gate vector When calculating the basic attention score, the contact graph matrix of the pairwise evolutionary flow input to the current layer is used. As an attention bias, it is directly injected (e.g.) Figure 2 (As indicated by the "attention bias" arrow in the middle), the formula is:

[0067] ;

[0068] in, This refers to the dimension of the query vector or key vector in the attention head; The elements of the contact graph matrix are inputs to the paired evolutionary flow; Let be the attention score matrix, where This represents the degree of attention or dependence weight of the i-th residue on the j-th residue; The query vector matrix (Query) is generated by linear transformation of the representation difference vector, representing the active query features of residues in attention interactions; The key vector matrix (Key) is generated by linear transformation of the representation difference vector and is used to match the query vector to measure the relevance of the sequence content.

[0069] This step forces the model to consider the evolutionary coupling strength of residues when evaluating their relationships, thus enabling structural constraints to directly guide sequence signals.

[0070] In terms of gating mechanisms and feature selection, attention-weighted summation (i.e., ...) will be used. The value vector after ) and the gate vector Perform element-wise multiplication: The output of the gated row attention module is further processed by a column attention module, and the final output is transformed by a position feed-forward network (FFN), and then integrated with residual connections and layer normalization to obtain the output representation of the sequence difference flow.

[0071] Step S1042. Pairwise evolutionary stream update (and receive sequence stream feedback):

[0072] In pairwise evolutionary streams, with contact graph matrices As input, the relationship characterization between residue pairs is updated through a pairwise symmetric attention module. (Correction feedback process) In each layer structure, during the update calculation of the pairwise evolutionary flow, the output characterization of the sequence difference flow obtained in step S1041 is processed through an outer product mean operation (e.g., Figure 2 As shown), a new pairwise representation is generated; this new pairwise representation is injected into the intermediate layer of the pairwise evolutionary flow (as shown). Figure 2 The “outer product average” in the middle points to the “+” sign between the blue modules, thus feeding back and updating the pairwise characterization with the inter-residue interaction information learned from the sequence context.

[0073] Step S1043. Iterative optimization: Thus, through layer stacking, the bidirectional interaction process of steps S1041 and S1042 above is repeated in multiple layers. The sequence difference flow and the pairwise evolution flow complete the interaction of guidance (step S1041) and feedback (step S1042) in each layer, and finally generate a powerful representation that deeply integrates the advantages of the two modalities and has both statistical consistency and structural rationality.

[0074] In this way, through the bidirectional coupling of the attention mechanism, sequence difference information and evolutionary constraint information are no longer static inputs, but evolve together through iterative interaction, ultimately generating a fusion representation that combines sequence statistical features with the rationality of structural constraints, providing a highly biologically interpretable input for tasks such as mutation fitness prediction.

[0075] In the embodiments of this application, the total loss function of the model It consists of three weighted parts, as shown in the following formula:

[0076] ;

[0077] in, This is the total loss function; For sorting loss The corresponding loss weight coefficient, Mean square error The corresponding loss weight coefficient, For structural consistency loss The corresponding loss weight coefficients; for example, , and The values ​​can be 1, 1, and 0.5 respectively. (Ranking loss) A differentiable ranking strategy based on NeuralSort is adopted. Specifically, the predicted fitness score vector is... and the true fitness score vector The NeuralSort operator is input separately, and the corresponding soft permutation matrix is ​​generated through relaxation operations. and (Soft Permutation Matrices), where matrix elements Let represent the probability that the i-th mutant is located at the j-th position in the sort; then, calculate the Kullback-Leibler (KL) divergence between the two as the loss value:

[0078] ;

[0079] In the formula, The calculated sorting loss value; This represents the total number of mutant protein samples participating in the current sorting calculation. The element value in the i-th row and j-th column of the soft permutation matrix corresponding to the true fitness ranking represents the probability that the i-th mutant protein is in the j-th position in the true fitness ranking. The element in the i-th row and j-th column of the soft permutation matrix corresponding to the model's predicted fitness ranking represents the probability that the i-th mutant protein is in the j-th position in the model's predicted fitness ranking. This loss function constrains the model's predicted ranking distribution to approximate the true ranking distribution at the list-wise level. To more intuitively understand the above optimization objective, refer to... Figure 2 The "Multi-Objective Loss Function Optimization" module in the upper right corner illustrates the geometric meaning of ranking optimization using example mutants "True A" and "True B" in the diagram: the model's goal is to ensure that the relative order of predicted values ​​(e.g., "Predicted B" is to the left of "Predicted A," i.e., Predicted B < Predicted A) is consistent with the relative order of true values ​​("True B" is to the left of "True A") (as shown by the checkmark path in the diagram); conversely, if the relative positions of the predicted values ​​are reversed (as shown by the cross path in the diagram), it means that the ranking is incorrect. This intuitive relative position constraint echoes the mathematical principles of NeuralSort mentioned above, ensuring that the model can correctly capture the superiority / inferiority relationships between mutants.

[0080] Using mean squared error, the predicted score of the detection model is close to the true fitness score.

[0081] To address structural consistency loss, a distance distribution prediction strategy is employed. Specifically, this involves first extracting data from the protein PDB file. Atomic coordinates (glycine is) The Euclidean distances between residues are calculated, and the distance matrix is ​​discretized into 64 intervals (Bins) as the true labels Y; the model outputs a predicted log odds of shape (L,L,64). After applying a mask to filter out invalid regions, the multi-class cross-entropy loss between the predicted distribution and the true label is calculated:

[0082] ;

[0083] In the formula, To predict log odds, The labels are real; the pairwise representations learned by this loss-constrained model can accurately predict the spatial distance distribution between residues.

[0084] Step S105: Input the enhanced characterization into the downstream prediction head, and have the downstream prediction head regress and output a scalar value as the fitness prediction result of the mutant protein.

[0085] In this embodiment, the downstream prediction head serves as the final output unit of the entire model, mapping the enhanced characterization output by the multimodal fusion module in step S104 to a single scalar value to quantify the fitness level of the mutant protein. This enhanced characterization deeply integrates the sequence differences between the mutant and wild-type, inter-residue co-evolutionary constraints, and epistatic effects, combining sequence statistical consistency with reasonable structural constraints.

[0086] In the embodiments of this application, the output of the Coformer module passes through a prediction head composed of a fully connected layer, and finally regresses to obtain a single scalar value, namely the fitness prediction result of the mutant protein.

[0087] The protein fitness prediction method provided in this embodiment calculates the representation difference vector based on wild-type and mutant sequence-level representations, and designs an interactive modeling mechanism between this sequence-level representation difference vector and pairwise evolutionary constraint information inferred from MSA. This enables the understanding of the context of sequence mutation signals by utilizing the spatial constraints provided by co-evolution, thereby effectively modeling the non-additive effects between mutations and significantly improving prediction accuracy. At the same time, it does not rely on experimentally determined three-dimensional structures, overcoming the application bottleneck of traditional structural methods, and can be widely applied to almost any protein sequence, greatly expanding its application scope.

[0088] Example 2

[0089] Based on the above embodiments, this embodiment also provides a protein fitness prediction device for implementing the above protein prediction method. The device of this embodiment includes:

[0090] The module is used to acquire wild-type protein sequences and at least one mutant protein sequence, and to extract wild-type sequence-level representations and mutant sequence-level representations from the wild-type protein sequences and the mutant protein sequences, respectively, using a pre-trained protein language model.

[0091] The difference calculation module is used to calculate the characterization difference vector based on the wild-type sequence-level characterization and the mutant sequence-level characterization;

[0092] The evolution extraction module is used to generate multiple sequence alignment information based on the wild-type protein sequence, and then infer co-evolutionary coupling information from the multiple sequence alignment information to construct pairwise evolutionary constraint information reflecting spatial proximity relationships.

[0093] The multimodal fusion module is used to model the interaction between the representation difference vector and the pairwise evolutionary constraint information through an attention mechanism, so that the pairwise co-evolutionary information guides the propagation and weighting of the representation difference vector between residues to capture the epistatic effect between mutations and generate enhanced representations.

[0094] The prediction output module is used to input the enhanced characterization into the downstream prediction head, and the downstream prediction head regresses and outputs a scalar value as the fitness prediction result of the mutant protein.

[0095] Example 3

[0096] To systematically evaluate the model's generalization ability in different application scenarios, this embodiment employs three core data partitioning strategies with different challenge levels, combined with the appendix... Figures 3 to 5 This document presents a performance comparison between the method of this embodiment (Our_Result, indicated by the pentagram in the figure) and existing mainstream methods (such as ESM-1v (Evolutionary Scale Modeling-1v, protein sequence language model version 1v), ProteinNPT (Protein Non-Parametric Transformer, non-parametric protein model), etc.):

[0097] 1) Random Split:

[0098] Definition: Mutants are randomly divided into training, validation, and test sets. This primarily evaluates the model's interpolation ability within a known sequence space.

[0099] Effects: such as Figure 3 As shown, while most baseline methods achieve good correlations under this setting, the method in this embodiment (pentagram) still occupies the far right of the ranking for the vast majority of protein instances (i.e., the highest Spearman correlation coefficient). This demonstrates the effectiveness of the model's basic feature extraction.

[0100] 2) Contiguous Split:

[0101] Definition: The test set is defined as a continuous sequence of fragments based on residue position (e.g., any mutations at positions 100-120 that have never been seen in the training set). This presents a significant challenge to the model, aiming to evaluate its ability to extrapolate to unseen regions.

[0102] Effects: such as Figure 4 As shown, under this highly challenging setting, many contrastive methods that rely solely on local sequence patterns experience significant performance degradation (left shift of positions). In contrast, the method in this embodiment exhibits superior robustness. This is primarily attributed to the NeuralSort ranking loss function introduced in this application. Because the NeuralSort ranking loss function constrains the predicted distribution at the list-wise level, rather than focusing solely on local sample pairs, it forces the model to learn the global topological structure of the protein fitness landscape. Even when unseen mutation regions appear in the test set, the model can still maintain the correct relative ranking based on global evolutionary constraints and structural consistency.

[0103] 3) Modulo Split based on sequence position modulus:

[0104] Definition: Partitioning based on the modulus of the residue index (e.g., all indices are 0). (The mutations are included in the test set). This requires the model to have strong context integration capabilities, enabling it to infer the mutation effects of missing links using scattered contextual information.

[0105] Effects: such as Figure 5 As shown, the method in this application still maintains a leading position on most proteins. The multimodal fusion module (Coformer) uses an attention mechanism to enable bidirectional interaction between discrete sequence differences and the global evolutionary contact map, successfully filling the gaps in contextual information and further verifying the superiority of list-wise sorting optimization in handling complex sparse data.

[0106] In addition, to visually demonstrate the quality of the "enhanced representation" generated by the model, this embodiment selects two representative proteins (FKBP3_HUMAN and FECA_ECOLI) and performs dimensionality reduction visualization analysis on their high-dimensional representations before entering the downstream prediction head. Figure 6 The fitness characterization manifold distribution map of the representative protein FKBP3_HUMAN, generated based on the t-SNE (t-Distributed Stochastic Neighbor Embedding) algorithm, is provided in the embodiments of this application. Figure 7The fitness characterization manifold distribution map of the representative protein FECA_ECOLI, generated based on the t-SNE algorithm, is provided for the embodiments of this application. Each scatter point in the map represents a mutant sample, and the color of the point maps its true fitness value (GroundTruth Fitness). The color bar transitions from purple (low fitness / lethal mutation) to yellow (high fitness / beneficial mutation).

[0107] Effects: such as Figure 6 , Figure 7 As shown, the latent space learned by the model in this application exhibits excellent semantic separability: clear clustering: high-fitness mutations (yellow / bright green dots) and low-fitness mutations (purple / dark blue dots) are automatically separated spatially, forming well-defined clusters. Natural gradient transition: mutants with similar fitness values ​​are also closer in spatial distance, forming a continuous and smooth fitness manifold. This indicates that, guided by the bidirectional interaction and structural constraints of the Coformer module, the enhanced representation generated by the model has successfully encoded the functional semantics of mutations. Downstream prediction heads do not need to handle complex nonlinear relationships; they can obtain high-precision prediction results simply by performing regression based on this high-quality linearly separable representation.

[0108] This application also provides a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the method in any of the embodiments of this application. Specifically, a system or apparatus equipped with a storage medium may be provided, on which software program code implementing the functions of any of the above embodiments is stored, and the computer (or CPU (Central Processing Unit) or MPU (Microprocessor Unit) of the system or apparatus may read and execute the program code stored in the storage medium.

[0109] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a computer-readable storage medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication component, and / or installed from a removable medium. When the computer program is executed by a central processing unit (CPU), it performs the functions defined in the system of this application.

[0110] It should be noted that the computer-readable storage medium shown in this invention can be a computer-readable signal medium, a computer-readable storage medium, or any combination thereof. For example, a computer-readable storage medium can be an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM) or flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this invention, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. The transmitted data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable storage medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable storage medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF (Radio Frequency), etc., or any suitable combination thereof.

[0111] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0112] The units described in the embodiments of the present invention can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the specific unit itself.

[0113] It should be noted that although several modules or units of the device for performing actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of the present invention, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0114] In the several embodiments provided in this application, it should be understood that the disclosed systems, modules, and methods can be implemented in other ways. For example, the module embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between modules or units, and may be electrical, mechanical, or other forms.

[0115] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit it. This application is not limited to the exact structures described above and illustrated in the accompanying drawings, and it should not be considered that the specific implementation of this application is limited to these descriptions. For those skilled in the art, various changes and modifications made without departing from the concept of this application should be considered to fall within the protection scope of this application.

Claims

1. A protein fitness prediction method, characterized by, The method comprises the following steps: obtaining a wild-type protein sequence and at least one mutant protein sequence, and extracting a wild-type sequence-level representation and a mutant sequence-level representation from the wild-type protein sequence and the mutant protein sequence respectively by using a pre-trained protein language model; calculating a representation difference vector based on the wild-type sequence-level representation and the mutant sequence-level representation; generating multiple sequence alignment information based on the wild-type protein sequence, and then inferring co-evolution coupling information from the multiple sequence alignment information to construct pairwise evolutionary constraint information reflecting spatial proximity relationship; modeling the interaction between the representation difference vector and the pairwise evolutionary constraint information by using an attention mechanism, so that the pairwise evolutionary constraint information guides the propagation and weighting of the representation difference vector between residues to capture epistatic effects between mutations and generate an enhanced representation; inputting the enhanced representation into a downstream prediction head, and outputting a scalar value as the fitness prediction result of the mutant protein by the downstream prediction head.

2. The protein fitness prediction method of claim 1, wherein, The method comprises the following steps: based on the wild-type sequence level characterization and the mutant sequence level characterization, a fusion feature vector is constructed by linear interpolation : ； wherein, is a fusion feature vector, is a mutant sequence level representation, is a wild type sequence level representation, is an interpolation coefficient; stripping the wild type reference signal from the fusion feature vector to obtain an original mutation difference vector : ; Apply to the original mutation difference vector The activation function performs a non-linear transformation, as shown in the formula: ； wherein the scaling factor and the offset factor are learnable parameters dynamically predicted by a multi-layer perceptron based on the channel feature of the input to achieve adaptive adjustment of the activation range of different channel features, is a sequence-level mutation effect representation vector; The protein language model is ESM-2; the method comprises the following steps: for the wild-type protein sequence, using the context compilation capability of ESM-2 to perform end-to-end encoding on the wild-type protein sequence, integrating the local context and global semantics of the wild-type protein sequence into a high-dimensional vector, and extracting a wild-type sequence-level representation; for each mutant protein sequence, using the context compilation capability of ESM-2 to perform end-to-end encoding on the mutant protein sequence, integrating the local context and global semantics of the mutant protein sequence into a high-dimensional vector, and extracting a corresponding mutant sequence-level representation.

3. The protein fitness prediction method of claim 2, wherein, The method comprises the following steps: using a sequence search tool to construct multiple sequence alignment information for the wild-type protein sequence; wherein, taking the wild-type protein sequence as a query sequence, obtaining a set of homologous protein sequences from a database by HHblits, and then performing multiple sequence alignment by using MAFFT or Clustal Omega tool to generate multiple sequence alignment information containing sequence conservation and variation pattern.

4. The protein fitness prediction method of claim 3, wherein, The method comprises the following steps: inputting the multiple sequence alignment information into the PconsC4 model to generate a co-variation score of residue pairs; transforming the co-variation score into a contact map matrix to take the contact map matrix as the pair-wise evolutionary constraint information; wherein, is a sequence length of the wild type protein sequence, and a value of each element in the contact map matrix identifies a size of a possibility of contact between residues at corresponding positions. is a sequence length of the wild type protein sequence, and a value of each element in the contact map matrix identifies a size of a possibility of contact between residues at corresponding positions.

5. The protein fitness prediction method of claim 4, wherein, The method comprises the following steps: The multi-modal fusion module models the interaction between the representation difference vector and the pair-wise evolutionary constraint information, and the multi-modal fusion module includes multiple layers of bidirectional interaction units, and each layer of bidirectional interaction units includes a sequence difference flow and a pair-wise evolutionary flow. The updating of the sequence difference flow is guided by the pair-wise evolutionary flow, and the pair-wise evolutionary flow updates and receives feedback of the sequence difference flow.

6. The protein fitness prediction method of claim 5, wherein, The updating of the sequence difference flow is guided by the pair-wise evolutionary flow, and the updating includes: In the sequence difference flow, the representation difference vector of the sequence level input in the current layer For input, feature update is realized by a gate row attention module; In the attention calculation and bias injection process, first, the representation difference vector is subjected to linear transformation to generate a query vector, a key vector, a value vector, and a gate vector When calculating the basic attention score, the contact map matrix of the pair evolution flow input by the current layer is directly injected as the attention bias, and the formula is as follows: ； wherein, is the element in the i-th row and j-th column of the contact map matrix input to the pair-wise evolutionary flow; is the dimension of the query vector or key vector in the attention head; is the attention score matrix, where represents the dependency weight of the i-th residue pair to the j-th residue; is the query vector matrix generated by linear transformation of the representation difference vector, representing the active query feature of the residue in the attention interaction; is the key vector matrix Key generated by linear transformation of the representation difference vector, used to match with the query vector to measure the relevance of the sequence content; The value vector obtained through attention weighted summation is subjected to element-level multiplication with a gating vector, and the output of the gating row attention module is further processed through a column attention module, and then converted through a position feedforward network, and integrated through a residual connection and layer normalization to obtain the output representation of the sequence difference flow. The pair-wise evolutionary flow updates and receives feedback of the sequence difference flow, and the updating includes: The contact map matrix input by the current layer is input to update the relationship representation between residue pairs through a pair-wise symmetric attention module; in the updating process of the pair-wise evolutionary flow, the output representation of the sequence difference flow is subjected to an outer product averaging operation to generate a new pair-wise representation, and the new pair-wise representation is injected into the calculation of the pair-wise symmetric attention module to realize feedback of the sequence feature pair-wise evolutionary constraint to generate an updated pair-wise evolutionary representation for input of a next layer of bidirectional interaction units.

7. The protein fitness prediction method of claim 6, wherein, In the model training process, a total loss function is calculated based on the fitness prediction result output by the downstream prediction head and the attention map generated in the multi-modal fusion module; the total loss function is composed of three parts, as follows: ； wherein, is a total loss function; adopting ranking loss, based on fitness prediction results output by the downstream prediction head, for any pair of mutants with different fitness, punishing the model for making an error in the ranking of fitness scores; adopting mean square error, detecting that the model prediction score output by the downstream prediction head is close to the true fitness score; is a structure consistency loss, comparing the attention map learned from the inside of the multi-modal fusion module with a known protein contact distance map, wherein the protein contact distance map is calculated from a protein PDB structure file; is a ranking loss is a corresponding loss weight coefficient, is a mean square error is a corresponding loss weight coefficient, is a structure consistency loss is a corresponding loss weight coefficient.

8. A protein fitness prediction device, characterized by, The device includes: The representation acquisition module is configured to acquire a wild-type protein sequence and at least one mutant protein sequence, and extract wild-type sequence-level representation and mutant sequence-level representation from the wild-type protein sequence and the mutant protein sequence respectively by using a pre-trained protein language model; The difference calculation module is configured to calculate a representation difference vector based on the wild-type sequence-level representation and the mutant sequence-level representation; The evolutionary extraction module is configured to generate multiple sequence alignment information based on the wild-type protein sequence, and further infer co-evolution coupling information from the multiple sequence alignment information to construct pair-wise evolutionary constraint information reflecting spatial proximity relationship; The multi-modal fusion module is configured to model the interaction between the representation difference vector and the pair-wise evolutionary constraint information through an attention mechanism, so that the pair-wise evolutionary constraint information guides the propagation and weighting of the representation difference vector between residues to capture epistatic effects between mutations and generate enhanced representation; The prediction output module is configured to input the enhanced representation into a downstream prediction head, and output a scalar value as a fitness prediction result of the mutant protein by the downstream prediction head.

9. A terminal, characterized by comprising: The terminal includes a memory and one or more processors; the memory stores one or more programs; the programs contain instructions for executing the protein fitness prediction method according to any one of claims 1 to 7; and the processor is configured to execute the programs.

10. A computer readable storage medium having stored thereon a plurality of instructions, the plurality of instructions comprising: The instructions are adapted to be loaded and executed by the processor to implement the protein fitness prediction method according to any one of claims 1 to 7.

Images