Multi-modal protein learning method and system
Through multimodal protein learning method, the characteristics of enzymes, substrates and products are extracted and fused, and combined with auxiliary regularizers, the problem of poor performance of existing models when modal data is missing is solved, and the enzyme catalytic constant kcat is realized High accuracy and robustness in value prediction.
Patent Information
- Application Number
- CN202411148780.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-21
- Publication Date
- 2024-12-03
AI Technical Summary
The existing deep learning model ignores the effects of reaction products in the prediction of enzyme catalytic constant kcat value, and significantly decreases performance when modal data is missing, resulting in limited effectiveness in practical applications.
Using multimodal protein learning method, by extracting features from multimodal biological data, mask processing and feature fusion, combined with auxiliary regularizers, a model that can accurately predict enzyme catalytic constants when the modal data is incomplete .
The accuracy and robustness of the prediction of the enzyme catalytic constant kcat value are improved, the model performance in the absence of modal data is improved, and the prediction effect is shown.
Smart Images

Figure CN119069036A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of biotechnology, and particularly relates to a multimodal protein learning method and system Background Art
[0002] In biochemistry, the catalytic constant (k cat The k value is an important indicator to measure the maximum substrate conversion rate of an enzyme at its active site, which is crucial for evaluating catalytic efficiency and understanding the mechanism of biochemical reactions. cat Since it is time-consuming and resource-intensive to calculate the value of k, deep learning (DL) methods have become an attractive alternative. However, existing DL models often ignore the effect of reaction products on k. cat Due to the multimodal nature of enzymes, substrates, and products as inputs, some modalities are unavailable during inference due to incomplete data or experimental limitations, which poses additional challenges to the applicability of existing DL models. Summary of the invention
[0003] One of the embodiments of the present disclosure is a multimodal protein learning method, comprising the following steps:
[0004] A1, extracting features from multimodal biological data, the multimodal biological data including substrates, products and enzymes;
[0005] A2, performing mask processing on the features;
[0006] A3, fuse the masked features to obtain the enzyme catalytic constant k cat Predicted value;
[0007] A4, introduces an auxiliary regularizer to predict enzyme catalytic constants.
[0008] One of the embodiments of the present disclosure is a multimodal protein learning system, which predicts enzyme catalytic constants based on a multimodal protein learning model. The method for constructing the model includes:
[0009] A1, extracting features from multimodal biological data, the multimodal biological data including substrates, products and enzymes;
[0010] A2, performing mask processing on the features;
[0011] A3, fuse the masked features to obtain the enzyme catalytic constant k cat Predicted value;
[0012] A4, introduces an auxiliary regularizer to predict enzyme catalytic constants.
[0013] The beneficial effect of the present disclosure is to provide a multimodal protein learning method and system to solve the problem that existing methods have poor performance in predicting enzyme values in the absence of modal data, thereby improving the accuracy and robustness of the prediction. BRIEF DESCRIPTION OF THE DRAWINGS
[0014] The above and other objects, features and advantages of the exemplary embodiments of the present invention will become readily understood by reading the detailed description below with reference to the accompanying drawings. In the accompanying drawings, several embodiments of the present invention are shown in an exemplary and non-limiting manner, in which:
[0015] Figure 1 Schematic diagram of a multimodal protein learning system according to one embodiment of the present invention. DETAILED DESCRIPTION
[0016] In the prior art, DLKCAT, TURNUP and UNIKP are three different enzyme catalytic constant prediction methods, among which:
[0017] DLKCAT, based on deep learning, was developed by a research team at Chalmers University of Technology for high-throughput prediction of catalytic constants (k cat ). This method can be used to characterize metabolic enzymes from any organism based on substrate structure and protein sequence. cat Prediction. DLKCAT can capture the k cat changes, and identify the k cat The amino acid residues that have a strong influence on the value.
[0018] TURNUP is a general and organism-independent model that successfully predicts turnover numbers for natural reactions of wild-type enzymes. The model represents the complete chemical reaction via differential reaction fingerprints and uses a modified and retrained protein sequence Transformer Network model to represent the enzyme.
[0019] UNIKP is a unified framework based on a pre-trained large language model, proposed by Luo Xiaozhou's team from the Institute of Synthesis, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences. UniKP can directly predict enzyme kinetic parameters, including enzyme catalytic constant (k) from protein sequences and substrate structures with high accuracy. cat ), Michaelis constant (Km) and catalytic efficiency (k cat / Km).
[0020] Although the above three methods all represent progress in the field of enzyme kinetic parameter prediction, they provide good tools and methods for understanding the catalytic efficiency of enzymes through different technical means. catAll prediction methods have a drawback, that is, they can only work when all necessary modes are present. When some modal data are missing, the performance of these methods is significantly reduced, resulting in limited effectiveness in practical applications. Therefore, how to accurately predict k when the modal data is incomplete? cat value, has become an urgent problem to be solved in the current field.
[0021] According to one or more embodiments, a multimodal protein learning system can be used to predict the catalytic efficiency of an enzyme on a substrate, as well as the chemical structures of the substrate and the product. Figure 1 As shown, the system will integrate multiple bioinformatics databases and algorithm models to achieve accurate prediction of enzyme activity.
[0022] The system uses a multimodal protein learning model to predict enzyme catalytic constants. The construction method includes the following steps:
[0023] S101. Data collection: Collect enzyme, substrate, and product related information and sequence data from databases such as BRENDA, SABIO-RK, and UniProt.
[0024] S102. Data preprocessing: Convert chemical structures into SMILES format and encode protein sequences. S103. Multimodal protein learning model training: Use ESM2 and Transformer models with more than 20,000 data entries for training.
[0025] S104. Parameter optimization: Optimize model performance while maintaining model stability by adjusting model parameters, including freezing the weights of specified layers.
[0026] S105. Prediction and analysis: Use the trained model to predict k cat The catalytic efficiency of the enzyme on different substrates was analyzed.
[0027] S106. Result verification: Verify the prediction accuracy of the model through experimental data or independent test sets.
[0028] Among them, k cat The value is the rate constant of the enzyme-catalyzed reaction. The model of the system of the embodiment of the present disclosure predicts this value and uses this value as an indicator of the enzyme's catalytic efficiency.
[0029] In step S101, the data acquisition database integrates multiple bioinformatics databases, including:
[0030] BRENDA is a widely used enzyme function database;
[0031] SABIO-RK is a database of enzyme reactions and kinase / phosphatase reactions;
[0032] UniProt is a database of protein sequence and function information.
[0033] The database contains more than 20,000 substrate SMILES data entries, as well as the corresponding protein sequences and product SMILES, allowing the model to learn the catalytic properties of enzymes for specific substrates from these data and adapt to the chemical properties of different substrates to predict the catalytic efficiency of the enzyme.
[0034] In step S102, SMILES (Simplified Molecular-Input Line-Entry System) is a string representation method for describing chemical structures. This step converts the chemical structures of substrates and products into SMILES format to facilitate processing by the machine learning model.
[0035] In step S103, the model architecture used includes ESM2 (Error-Suppressing Mixture of Experts Model 2) and Transformer model to better process sequence data and identify and learn specific amino acid residues in the enzyme active site.
[0036] The model input data trained in the embodiment of the present disclosure can be expressed as G=(N,E), where N represents the protein sequence and E represents the enzyme or other related information.
[0037] One of the benefits of the embodiments of the present disclosure is that by predicting the catalytic properties of enzymes, the discovery of new enzymes and the improvement of existing enzymes are accelerated.
[0038] Further, the beneficial effects of the embodiments of the present disclosure also include, through a multimodal deep learning method, the comprehensive use of chemical molecule representations, protein sequence information and protein structure features for complex biochemical prediction tasks. Multimodality here refers to the simultaneous processing and analysis of multiple different types of data or information in protein prediction. These data types may include, but are not limited to, protein sequence information, structural information, functional information, interaction information, as well as protein expression profiles, protein modifications, etc. Multimodal methods can comprehensively utilize these different types of data to improve the accuracy and reliability of predictions.
[0039] According to one or more embodiments, a method for k cat The multimodal protein learning framework for predicting protein values can be named MMKCAT, where “MM” stands for multimodal and “KCAT” stands for k cat The value is expressed as a multimodal protein learning method, which aims to solve the problem of poor prediction performance in the absence of modal data. The overall framework is divided into four stages:
[0040] The first stage is modality-specific feature extraction. SMILES TRANSFORMERS is used to extract the SMILES representation of substrates and products, ESM2 is used to extract enzyme sequences, and a customized graph neural network (GNN) is used to extract enzyme structure diagrams. Here, TRANSFORMERS is a deep learning model used to process natural language processing (NLP) tasks. Its core is the self-attention mechanism (Self-Attention), which can capture long-distance dependencies in sequence data. SMILES TRANSFORMERS combines SMILES representation and TRANSFORMERS model. In the disclosed embodiment, it is used to process SMILES strings of chemical molecules, and is used for tasks such as property prediction and reaction prediction of chemical molecules. ESM2 is a protein language model, which is an application of the TRANSFORMERS model. In the disclosed embodiment, ESM2 extracts protein representation information by learning a large number of protein sequences, and is used for tasks such as protein structure prediction and function prediction.
[0041] The formula for SMILES representation extraction is as follows:
[0042]
[0043] Formula (1) for each substrate molecule SMILES string Encode using the SMILES Transformer model to obtain the feature representation of each molecule, sum the feature representations of all molecules, and then divide by the total number of substrate molecules n sub , and get the average feature representation f sub .
[0044] Formula (2), for each product molecule SMILES string Use the SMILES Transformer model to encode and obtain the feature representation of each molecule. Sum the feature representations of all molecules and then divide it by the total number of product molecules n. prod , and get the average feature representation f prod .
[0045] Among them, n sub and n prod denote the number of substrates and products, respectively, and d1 is the potential dimension of the SMILES TRANSFORMER. If there is no product in the reaction, f prod will be set to the zero vector, i.e. f prod =0.
[0046] Extract the length l using ESM2 enzWhen the enzyme protein sequence is obtained, the amino acid representation of each position in the enzyme protein sequence is first obtained, and then the average operation of these position-specific representations is performed to obtain the sequence-level representation. The formula is as follows:
[0047]
[0048] Where d2 is the latent dimension of ESM2.
[0049] In the extraction of enzyme structure graphs, ESMFOLD is first used to predict the three-dimensional structure of the enzyme, and the corresponding protein graph G is constructed based on these three-dimensional structures. Then, a graph convolutional network (GCN) is used to generate a graph representation. The formula is as follows:
[0050] f graph =GCN(G) (4)
[0051] Among them, d3 is the final output dimension of GCN. ESMFOLD is an end-to-end single sequence structure predictor based on the large protein language model ESM2, which can quickly achieve atomic resolution level structure prediction from protein sequences.
[0052] The second stage is the mask bottleneck processing. Based on the non-uniform mechanism guided by the leader, different feature maps are masked or retained to dynamically adjust the retention or masking of features. Before each feature representation is input into the feature fusion TRANSFORMER, the adapter is used for feature enhancement. The structure of each adapter is the same, processing substrate SMILES, product SMILES, enzyme sequence and enzyme map structure. The formula is as follows:
[0053] f am =Linear(Dropout(ReLU(Linear(f m )))), (5)
[0054] m∈{sub,enz,graph,prod}
[0055] in, is the feature processed by the adapter, d model It is the common dimension used by feature fusion TRANSFORMER. Input feature f m After transformation through a linear layer, the ReLU activation function is applied to introduce nonlinearity, then Dropout is applied for regularization, and finally another linear layer is used for feature transformation.
[0056] The third stage is feature fusion. The processed features are input into the feature fusion TRANSFORMER, the intra-modal relationship is constructed through the multi-head self-attention mechanism, and the modality-invariant features across modalities are extracted through the stacked encoder layers to generate the predicted k catValue. The design architecture is as follows:
[0057] Feature fusion TRANSFORMER is used to build extensive long-distance associations across multiple modalities, capturing and maintaining modality-invariant features related to global semantics. The formula is as follows:
[0058]
[0059] in, concat represents the concatenation operation, which means concatenating multiple feature vectors along the feature dimension to form f global The global eigenvector. Among them,
[0060] Represents the feature vector related to the substrate;
[0061] Represents the feature vector related to enzyme;
[0062] f mask_graph , represents the feature vector associated with the mask graph. In the GNN, it is used to represent the features in the graph structure of the enzyme-substrate interaction.
[0063] f mask_prod , represents the feature vector associated with the product.
[0064] Modality invariant features are extracted through multi-head self-attention mechanism and stacked encoder layers. The formula is as follows:
[0065]
[0066] Formula (7) first transforms the global feature After processing through the MSA layer, the MSA feature is obtained. Then the original global feature and the MSA feature are added element by element to obtain the fused feature. Finally, the layer normalization operation is applied to the fused feature to obtain the final normalized global feature.
[0067]
[0068] Formula (8) first converts the normalized features after layer l into Through a feedforward network, further extracted features are obtained. Then the normalized features are The features processed by the feedforward network are added element by element to obtain the fused features. Finally, the layer normalization operation is applied to the fused features to obtain the normalized global features.
[0069] Finally, the final k is generated through average pooling and linear layers cat Value prediction, the formula is as follows:
[0070]
[0071] Formula (9) first transforms the global feature It is transformed through a Linear layer, and then the ReLU activation function is applied to introduce nonlinearity, followed by Dropout for regularization, and finally AvgPooling average pooling is applied to reduce the dimension of the feature.
[0072] The fourth stage is auxiliary regularization, which improves the generalization ability of the model. Combined with the auxiliary regularizer, each modality-specific adapter is encouraged to independently generate informative features based on its encoder, which supports the prediction of k even in the presence of certain modalities (such as the lack of spatial structure of proteins). cat . A shared weight assisted regularizer is used for prediction. The final loss function is defined as:
[0073] L total =λ1·L pred +λ2·L aux (9)
[0074] Among them, L pred =MSE(k cat ,MMKCAT MLP (f all )) (10)
[0075]
[0076] Formula (10) uses the MMKCAT of the embodiment of the present disclosure MLP The model is based on the multimodal protein learning method of the embodiment of the present disclosure, according to the feature vector f all To predict the catalytic constant of the enzyme, the model prediction value and the actual catalytic constant k are calculated. cat The mean square error between pred Predict losses.
[0077] Formula (11) for each subset subset i , using the auxiliary model MMKCAT of the embodiment of the present disclosure aux , based on the multimodal protein learning method of the disclosed embodiment, to predict the catalytic constant of the enzyme, and calculate the predicted value of the auxiliary model and the actual catalytic constant k cat The mean square error between them is summed up on all subsets to get the total auxiliary loss L aux .
[0078] In order to further verify the multimodal protein learning system and model disclosed in the present invention, the model MMKCAT disclosed in the present invention was trained using the public BRENDA and SABIO-RK datasets to achieve the enzyme catalytic constant k cat The results show that MMKCAT performs better than other baseline models (such as DLKCAT, TURNUP, and UNIKP) in both complete and missing modal scenarios. The fine-tuned model MMKCAT+ has significantly improved on its performance, with average RMSE, R 2 and SRCC indicators increased by 6.89%, 21.04% and 8.15% respectively.
[0079] It should be understood that in the embodiments of the present invention, the term "and / or" is only a description of the association relationship of the associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.
[0080] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.
[0081] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, or it can be an electrical, mechanical or other form of connection.
[0082] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.
[0083] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.
Claims
1. A multimodal protein learning method, characterized in that: The following steps are involved: A1, extracting features from multimodal biological data, the multimodal biological data including substrates, products and enzymes; A2, performing mask processing on the features; A3, fuse the masked features to obtain the enzyme catalytic constant k cat Predicted value.
2. The multimodal protein learning method according to claim 1, characterized in that: The method further comprises the steps of: A4, introduces an auxiliary regularizer to optimize the prediction of enzyme catalytic constants.
3. The multimodal protein learning method according to claim 1, characterized in that: The step A1 further comprises: The SMILES representations of substrates and products were extracted based on the TRANSFORMER model, and the enzyme sequences were extracted based on ESM2.
4. The multimodal protein learning method according to claim 3, characterized in that: Graph neural network (GNN) is used to extract enzyme structure diagrams.
5. The multimodal protein learning method according to claim 3, characterized in that: The step A2 further comprises: Based on a pilot-guided non-uniform mechanism, different features are masked or retained to dynamically adjust the retention or masking of features.
6. The multimodal protein learning method according to claim 5, characterized in that: Before each feature representation is fed into the TRANSFORMER model, feature enhancement is performed using an adapter.
7. The multimodal protein learning method according to claim 6, characterized in that: The input features are fused through the TRANSFORMER model, the intra-modal relationship is constructed through the multi-head self-attention mechanism, and the modality-invariant features across modalities are extracted through the stacked encoder layers to generate the prediction value k cat .
8. A multimodal protein learning system for predicting enzyme catalytic constants based on a multimodal protein learning model, characterized in that: The model is constructed using the method according to any one of claims 1 to 7.
9. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.
10. A computer program product, comprising a computer program, characterized in that The computer program is executed by a processor to implement the method according to any one of claims 1 to 7.
Citation Information
Patent Citations
Enzyme-substrate affinity constant prediction method, storage medium and device
CN116417062A
Protein sequence chemical shift prediction method based on deep learning
CN118116505A
Computer device, method and application for multi-modal protein characterization based on sequence and structure
CN118398080A
Method and system for structure-based drug design using a multi-modal deep learning model
US20230154573A1
Multi-mode information fusion method and system for protein representative learning, and terminal and storage medium
WO2023109714A1
Cited By
Method and device for predicting protein mutation effect
CN119580837A
