Protein thermal stability prediction method and system based on feature fusion

Abstract

The invention discloses a protein thermal stability prediction method and system based on feature fusion. The method comprises the following steps: collecting original species sequence data; constructing a calculation model integrating a double-branch feature extraction structure, a local-global attention fusion mechanism and a feature aggregation and prediction module; the model is trained; in the inference stage, a to-be-analyzed protein sequence is input into the trained model, and local mode features and global context features of the to-be-analyzed protein sequence are extracted through a double-branch structure; in each layer, the local features and the global features of the same layer are spliced and interacted; and integrating the output sequence representation, and outputting a protein Tm value prediction result. The system comprises a data preprocessing unit, a model building unit, a model training unit and a model application unit. According to the invention, the Tm value can be predicted with high precision only by using the protein sequence and species information. The method can be widely applied to the field of bioinformatics.

Description

Technical Field

[0001] This invention relates to the field of bioinformatics, and in particular to a method and system for predicting protein thermal stability based on feature fusion. Background Technology

[0002] Protein thermal stability is a critical attribute in protein engineering and design, directly impacting its applications in industry, medicine, and scientific research. Traditional experimental methods (such as differential scanning calorimetry and circular dichroism spectroscopy) for measuring melting temperature (Tm) are costly and time-consuming, failing to meet high-throughput requirements. In recent years, deep learning-based computational methods have made significant progress in predicting protein thermal stability.

[0003] However, existing methods largely rely on auxiliary data, such as the optimal growth temperature (OGT) and experimental conditions, which limits their applicability in real-world scenarios. Furthermore, existing models often employ single-protein language model embeddings, failing to fully integrate local and global sequence information, resulting in limited representational capabilities. Summary of the Invention

[0004] In view of this, in order to address the technical problem that most existing protein thermal stability prediction methods use a single protein language model embedding, which cannot simultaneously capture local residue constraints and global sequence semantics, leading to information loss and consequently low prediction accuracy, this invention proposes a protein thermal stability prediction method based on feature fusion. This method includes the following steps: First, raw species sequence data are collected and preprocessed and quality controlled to form a standardized dataset. Then, a computational model integrating a two-branch feature extraction structure, a local-global attention fusion mechanism, and a feature aggregation and prediction module is constructed. The cleaned data is used to train the model until it converges, resulting in a stable model that can be used for inference. In the inference phase, the protein sequence to be analyzed is input into the pre-trained model, and its local pattern features and global context features are extracted through a dual-branch structure. In each Transformer module, the local and global features at the same level are concatenated and interacted with by a local-global attention fusion mechanism to generate a hidden layer representation that integrates multi-scale information. Finally, the output sequence representation is integrated by the aggregation prediction module, and the final protein Tm value prediction result is obtained based on the integrated features.

[0005] In addition to the overall process of the above method, the present invention also proposes a protein thermal stability prediction system based on feature fusion, which includes a data preprocessing unit, a model building unit, a model training unit, and a model application unit.

[0006] Based on the above scheme, the present invention provides a protein thermal stability prediction method and system based on feature fusion, which can predict Tm values ​​with high accuracy without relying on auxiliary data and using only protein sequence and species information; furthermore, through a local-global attention mechanism, it realizes multi-level interaction between sequence features and species context, and through a local-global feature fusion architecture, it fully integrates the advantages of different protein language models. Attached Figure Description

[0007] Figure 1 This is a flowchart of the steps of a protein thermal stability prediction method based on feature fusion according to the present invention. Figure 2 This is a schematic diagram of the data flow in a specific embodiment of the present invention; Figure 3 This is a model structure diagram of LGstabp, a specific embodiment of the present invention. Detailed Implementation

[0008] In addition to the issues mentioned in the background technology, existing models rely on auxiliary data such as OGT and other metadata, which are often missing or inaccurate in real-world scenarios, limiting the applicability of the models, and most models use single protein language model embeddings, which cannot simultaneously capture local residue constraints and global sequence semantics, leading to information loss. Furthermore, existing methods for predicting protein Tm values ​​also suffer from the following problems: insufficient utilization of species information: existing methods treat OGT as a continuous numerical input, ignoring discrete differences and evolutionary relationships between species, affecting the model's generalization ability; and simple architecture: existing models often use parallel branching or simple splicing methods to fuse features, failing to achieve dynamic interaction between local and global information.

[0009] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0010] It should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings. Unless otherwise specified, the embodiments and features described in this application can be combined with each other.

[0011] It should be understood that the terms "system," "apparatus," "unit," and / or "module" used in this application are a method of distinguishing different components, elements, parts, sections, or assemblies at different levels. However, if other terms can achieve the same purpose, they may be replaced by other expressions.

[0012] As indicated in this application and claims, unless the context clearly indicates otherwise, the words "a," "an," "a," and / or "the" are not specifically singular and may include the plural. Generally, the terms "comprising" and "including" only indicate the inclusion of expressly identified steps and elements, which do not constitute an exclusive list, and the method or apparatus may also include other steps or elements. An element defined by the phrase "comprising an..." does not exclude the presence of other identical elements in the process, method, product, or apparatus that includes the element.

[0013] In the description of the embodiments of this application, "a plurality of" refers to two or more. The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature.

[0014] Furthermore, flowcharts are used in this application to illustrate the operations performed by the system according to embodiments of this application. It should be understood that the preceding or following operations are not necessarily performed precisely in sequence. Instead, the steps can be processed in reverse order or simultaneously. Additionally, other operations can be added to these processes, or one or more steps can be removed from them.

[0015] Reference Figure 1 The diagram below illustrates an optional example of the protein thermal stability prediction method based on feature fusion proposed in this invention. This method can be applied to computer devices, and the prediction method proposed in this embodiment may include, but is not limited to, the following steps: Step S1: Obtain species sequence data and perform data cleaning; Step S2: Construct a prediction model based on the dual-branch feature extraction module, the local-global attention fusion module, and the aggregation prediction module; Step S3: Train the prediction model based on the cleaned dataset; Step S4: Input the sequence data to be tested into the trained prediction model and output the predicted protein Tm value.

[0016] In some feasible embodiments, step S1 specifically includes: Based on the publicly available Meltome dataset, redundant and duplicate data were removed. MMseqs2 was used for sequence search, grouping sequences with a similarity of 20% or higher into the same cluster. The dataset was then partitioned rigorously at the cluster level: 80% of the clusters were randomly selected, and all sequences contained within them were included in the training set; the remaining 20% ​​of the clusters were strictly reserved for evaluation, with only one representative sequence selected from each cluster to form the test set, ensuring that the similarity between the test sequence and any training sequence was less than 20%. 10% of the training set was randomly selected to form the validation set. The final dataset partitioning ratio was approximately 8:1:1 (training:validation:test).

[0017] In some feasible embodiments, step S2 specifically includes: The core of this invention lies in the design of a novel local-global feature fusion Transformer architecture, which aims to synergistically utilize complementary information from different protein language models and dynamically integrate sequence semantics and species context.

[0018] Dual-branch feature extraction module: The model sets up two parallel branches to process local and global features respectively; Local-Global Attention Fusion Module: Designed to address the problem of insufficient information fusion caused by simply splicing together different feature spaces; Aggregation prediction module: Aggregates the input data and generates the final feature vector, and then outputs the final predicted Tm value through the regression head.

[0019] In some feasible embodiments, step S3 specifically includes: This invention models protein thermal stability prediction as a regression task, using mean squared error (MSE) as the loss function, and its calculation formula is as follows: This loss function effectively guides the model's predictions to converge towards the true distribution by imposing a higher penalty on larger errors. The model parameters are iteratively optimized using the Adam optimizer. During training, each batch of protein sequences and species labels is sequentially forward-propagated to obtain predicted values; then the loss is calculated, and gradients are obtained through backpropagation; finally, the Adam optimizer uses these gradients to update all parameters (including the Transformer, MLP, and species embedding table). This process is repeated until the model converges on the validation set, thereby learning a stable mapping from input features to Tm values.

[0020] In some feasible embodiments, the dual-branch extraction module in the prediction model operates as follows: Local Branching: Based on residue-level embeddings from the protein language model ProtT5, this embodiment constructs a Transformer encoder structure. This structure, through its self-attention mechanism, can calculate the association weights of each amino acid in the sequence with all other amino acids, thus effectively modeling long-range dependencies between residues. Specifically, the projected local embeddings are fed into multi-layer Transformer blocks. In each layer, self-attention is calculated using query (Q), bond (K), and value (V) matrices, and the output is a deeply interactive, context-aware representation of local features. This mechanism enables the model to precisely focus on local motifs or individual residues that have a critical impact on protein stability.

[0021] in, ∈ [0,3] represents the th element of the Transformer. layer; It is the embedded input of a local sequence. , and These represent the query matrix, key matrix, and value matrix in the self-attention layer, respectively.

[0022] Global Branch: Sequence-level global embedding based on the protein language model ESM2—this embedding benefits from ESM2's pre-training method based on a masked language model, effectively capturing the overall evolutionary semantics and coconservation of the sequence—in this embodiment, it is concatenated with the species embedding, where the species embedding is mapped from species labels to a 20-dimensional vector through a trainable lookup table. This enables the model to learn the thermal adaptation relationships between species. Structurally, we use a multilayer perceptron (MLP) as the processing core. In the specific process, the concatenated vector passes through multiple MLPs, which include operations such as SeLU activation, layer normalization, and Dropout, to gradually complete nonlinear transformation and information fusion, and finally output a highly abstract comprehensive representation vector that combines global contextual information and species-specific thermal adaptation relationships.

[0023] in, It is a global sequence embedding input. Embedding of species tags.

[0024] In some feasible embodiments, the local-global attention fusion module in the prediction model works as follows: This layer involves layer-by-layer, depth-based interactions within each Transformer layer. Specifically, in each Transformer layer, the output of the self-attention module is concatenated with the global vector corresponding to the global branch at the same depth. Then, a dedicated projection layer fuses and reduces the dimensionality of the concatenated high-dimensional vector. Finally, the fused result is fed into the feedforward network of the Transformer block for further processing, and its output serves as the input to the next Transformer layer. This design allows global species information and sequence semantics to continuously guide and refine the extraction process of local sequence features, achieving true collaborative representation learning.

[0025] Indicates the first The output of the transformer self-attention module of the layer represents the local features of the protein. Indicates the first The output of the MLP module of the layer represents the global features of the protein. This represents the concatenated features. Due to the increased dimensionality after concatenation, it is necessary to use... Map the features back to the original dimensions. This represents the feature that ultimately integrates local and global information.

[0026] This merged representation It will be fused and dimensionality reduced through a dedicated projection layer, and then fed into a feedforward neural network for processing.

[0027] Output It will be passed to the next Transformer module. This embodiment uses four Transformer modules and four global feature MLP layers to maintain structural alignment.

[0028] In some feasible embodiments, the aggregation prediction module in the prediction model operates as follows: After feature extraction and fusion across all layers, the model first aggregates the final Transformer output sequence using an attention pooling mechanism. This mechanism utilizes a learnable query vector to calculate the weight of each residue position, then performs a weighted sum to obtain a global sequence representation, which is then projected to output a refined sequence feature. Compared to simple average pooling, this method can more effectively focus on residues that contribute critically to thermal stability. Subsequently, in the prediction phase, the aggregated sequence representation is added to the final output of the global branch to form a comprehensive feature vector that simultaneously contains refined local information and rich global information. Finally, this vector is input into a regression head (usually a multilayer perceptron) to output the final predicted Tm value.

[0029] The overall data flow of this method is referenced. Figure 2 Furthermore, compared to existing technologies, this invention exhibits higher prediction accuracy: In the Meltome benchmark test, this embodiment uses the Pearson correlation coefficient (PCC) and mean squared error (MSE) to measure the model's performance. PCC is positively correlated with model performance, while MSE is negatively correlated. LGstabp achieves PCC=0.919 and MSE=0.003, outperforming DeepSTABp (PCC=0.906, MSE=0.006) and PPTStab (0.867, MSE=0.006), and maintains high performance without relying on OGT (PCC=0.888, MSE=0.005).

[0030] The model structure diagram of the LGstabp of this invention is shown below. Figure 3 Where (a) is the overall model architecture diagram and (b) is the architecture diagram of the local-global attention fusion module.

[0031] This invention also provides relevant data examples: Data Preparation Phase: This embodiment uses the publicly available Meltome dataset as the data source. First, the raw data is preprocessed, filtering out proteins with amino acid sequences longer than 2800 to accommodate typical GPU memory limitations. To improve data quality, only Tm values ​​measured under lysis buffer conditions are selected as label data. Subsequently, the data is divided according to protein sequence homology, resulting in a training set (approximately 15,000 samples), a validation set (approximately 1,600 samples), and a test set (approximately 2,000 samples).

[0032] Feature extraction stage: Two pre-trained protein language models are used to extract sequence features. The ProtT5 (prot_T5_XL_uniref50) model is used to extract local sequence embedding features. This model excels at capturing local contextual information through a span-breaking pre-training strategy. Simultaneously, the ESM2 (esm2_t33_650M_UR50D) model is used to extract global sequence embedding features. This model is trained based on masked language modeling objectives and can capture the evolutionary constraints and global semantics of the sequence. Species labels are mapped to 20-dimensional vectors through learnable embedding layers, and concatenated with the global sequence embeddings to form the complete global feature input.

[0033] Model Construction Phase: Constructing a local-global feature fusion architecture. The local branch uses a 4-layer Transformer module with a hidden layer dimension of 512 to model the local context of the ProtT5 embedding. The global branch uses a 4-layer MLP network with layer dimensions of 1024, 512, 256, and 128 respectively, using the SeLU activation function and LayerNorm layers to perform non-linear transformations on the concatenated global features. The key innovation lies in the local-global attention fusion module, which concatenates the attention output of each Transformer layer with the corresponding global feature vector, reduces the dimensionality through a projection layer, and inputs it into the next layer, achieving deep interaction between the two types of features.

[0034] Model training phase: End-to-end training was performed using the Adam optimizer with a learning rate of 1e-4 and a batch size of 32. The outputs of the first four Transformer layers and four MLP layers were used as the fusion basis, and the sequence representation was finally aggregated through an attention pooling mechanism. This aggregated representation was then added to the output of the global branch and input into the regression prediction head. The training objective was to minimize the mean squared error (MSE) between the predicted Tm value and the true value. The entire training process was completed on a single RTX 4090 GPU.

[0035] Performance evaluation phase: Five metrics, namely RMSE, MSE, MAE, PCC, and R², were calculated on the test set. Referring to Table 1, the experimental results show that this method achieves optimal performance across all metrics, significantly outperforming baseline methods such as DeepSTABp and PPTStab, thus validating the effectiveness of this invention.

[0036] Table 1: Meltome benchmarks incorporating species and sequence information Example 2: Simplified prediction using only sequence information To verify the effectiveness of the local-global fusion mechanism itself, this embodiment conducts a comparative experiment without using species information. The model architecture remains consistent with Embodiment 1, except that the species embedding part is removed, and the input to the global branch is only the sequence embedding extracted by ESM2.

[0037] Table 2: Meltome benchmarks with only sequence information and no species information Experimental results show that even without species information, our method (LGstabp*) maintains superior performance, significantly outperforming DeepSTABp* and PPTStab, which also do not use species information. This indicates that the local-global feature fusion mechanism of this invention can effectively capture the thermal stability-related features of the sequence itself, without relying on external species information, thus enhancing the applicability of the model in scenarios where metadata is missing.

[0038] A protein thermal stability prediction system based on feature fusion, comprising: The data preprocessing unit is used to perform step S1; The model building unit is used to execute step S2; The model training unit is used to perform step S3; The model application unit is used to execute step S4.

[0039] The content of the above method embodiments is applicable to this system embodiment. The specific functions implemented in this system embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0040] A protein thermal stability prediction device based on feature fusion: At least one processor; At least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements a protein thermal stability prediction method based on feature fusion as described above.

[0041] The content of the above method embodiments is applicable to the device embodiments. The specific functions implemented by the device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0042] A storage medium storing processor-executable instructions, which, when executed by a processor, are used to implement a protein thermal stability prediction method based on feature fusion as described above.

[0043] The content of the above method embodiments is applicable to this storage medium embodiment. The specific functions implemented in this storage medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0044] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the embodiments described. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of this application.

Claims

1. A method for predicting protein thermal stability based on feature fusion, characterized in that, Includes the following steps: Acquire species sequence data and perform data cleaning to obtain a cleaned dataset; A prediction model is constructed based on a dual-branch feature extraction module, a local-global attention fusion module, and an aggregation prediction module. The prediction model is trained based on the cleaned dataset to obtain a trained prediction model. The sequence data to be tested is input into the trained prediction model, which outputs the predicted protein Tm value.

2. The protein thermal stability prediction method based on feature fusion according to claim 1, characterized in that, The step of acquiring species sequence data and cleaning the data to obtain a cleaned dataset specifically includes: Species sequence data were obtained and duplicate data were removed to obtain the data after removal; The removed data is subjected to sequence search, and sequences with sequence consistency greater than a preset threshold are grouped into the same cluster to obtain the cleaned dataset.

3. The protein thermal stability prediction method based on feature fusion according to claim 1, characterized in that, Also includes: The cleaned dataset is divided into training set, validation set and test set according to a preset ratio.

4. The protein thermal stability prediction method based on feature fusion according to claim 1, characterized in that, The loss function during training is expressed as: in, This indicates the total amount of protein in the batch. Indicates the first The true Tm value of a protein. Indicates the first Predicted Tm value for each protein, This indicates the batch number of the protein.

5. The protein thermal stability prediction method based on feature fusion according to claim 1, characterized in that, The step of inputting the sequence data to be tested into the trained prediction model and outputting the predicted protein Tm value specifically includes: Input the sequence data to be tested into the trained prediction model; Based on the dual-branch feature extraction module, feature extraction is performed on the test sequence data to generate local and global features; Based on the local-global attention fusion module, in each Transformer layer, the local features are concatenated with the global features corresponding to the same depth and fused to obtain the output sequence; Based on the aggregation prediction module, the output sequence is aggregated, and the predicted protein Tm value is output according to the aggregated features.

6. The protein thermal stability prediction method based on feature fusion according to claim 5, characterized in that, The dual-branch feature extraction module includes a local branch and a global branch. The step of extracting features from the test sequence data based on the dual-branch feature extraction module to generate local and global features specifically includes: The local branch projects the residue-level embeddings in the sequence data to be tested and feeds them into a multi-layer Transformer block. In each Transformer block, self-attention is calculated through query, key, and value matrices, and local features are output. The global branch concatenates the sequence-level global embedding and species embedding in the sequence data to be tested, and transforms and fuses them through a multi-layer MLP to output global features.

7. The protein thermal stability prediction method based on feature fusion according to claim 5, characterized in that, The step of concatenating the local features with the corresponding global features at the same depth in each Transformer layer based on the local-global attention fusion module to obtain the merged features specifically includes: In each Transformer layer, the output of the self-attention module is concatenated with the global vector corresponding to the global branch at the same depth. A dedicated projection layer is used to fuse and reduce the dimensionality of the stitched high-dimensional vectors; The fused result is fed into the feedforward network of the Transformer block for further processing, and the output is used as the input of the next Transformer layer.

8. The protein thermal stability prediction method based on feature fusion according to claim 7, characterized in that, In the local-global attention fusion module, the splicing process is expressed by the following formula: in, Indicates the first The output of the transformer self-attention module of the layer, Indicates the first The output of the MLP module of the layer, Indicates the features after splicing. This means mapping the features back to the original dimensions. This represents the feature that ultimately integrates local and global information.

9. A protein thermal stability prediction system based on feature fusion, characterized in that, include: The data preprocessing unit is used to acquire species sequence data and perform data cleaning to obtain a cleaned dataset. The model building unit constructs a prediction model based on a dual-branch feature extraction module, a local-global attention fusion module, and an aggregation prediction module. The model training unit trains the prediction model based on the cleaned dataset to obtain the trained prediction model. The model application unit is used to input the sequence data to be tested into the trained prediction model and output the predicted protein Tm value.

10. A protein thermal stability prediction device based on feature fusion, characterized in that, include: At least one processor; At least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements a protein thermal stability prediction method based on feature fusion as described in any one of claims 1-8.

Images