Multi-modal phosphorylation site prediction method, device, equipment and medium

Abstract

The invention relates to the technical field of biological information, and discloses a multi-mode phosphorylation site prediction method and device, equipment and a medium, and the method comprises the steps: extracting a subsequence of a candidate phosphorylation site from a to-be-predicted protein sequence, and generating a sequence feature of the subsequence through a deep learning model; constructing a graph structure based on the three-dimensional structure information of the candidate phosphorylation sites, and extracting structural features of the candidate sites through a graph neural network according to the graph structure; performing multi-modal feature fusion on the sequence features and the structural features to generate fusion features; and predicting the phosphorylation probability of the candidate site based on the fusion feature to obtain a phosphorylation probability prediction result. According to the method, the protein sequence language model and the multi-modal features of the three-dimensional structure diagram convolutional network are fused, so that the accuracy and comprehensiveness of phosphorylation site prediction can be effectively improved, and meanwhile, the calculation efficiency and the bioinformatics interpretation are considered.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of biological information, in particular to a multi-modal phosphorylation site prediction method, device, equipment and medium. BACKGROUND

[0002] Protein phosphorylation is a key modification process of intracellular signal transduction, and is closely related to numerous life activities and disease development. Accurate prediction of phosphorylation sites has extremely important scientific value and practical significance for in-depth understanding of protein function regulation mechanism, exploration of disease molecular mechanism, and acceleration of new drug research and development process. In today's biomedical research field, the research on phosphorylation site prediction method has attracted much attention.

[0003] However, the existing phosphorylation site prediction methods have many limitations. On the one hand, traditional experimental methods such as mass spectrometry can accurately identify phosphorylation sites, but they are complex, costly and time-consuming, and are difficult to meet the needs of large-scale proteome analysis. On the other hand, the prediction methods based on algorithms and traditional machine learning often rely on manually extracted sequence features, which cannot fully capture the complexity and non-linear relationship of proteins, resulting in insufficient prediction accuracy. It can be seen that the existing technology has the problems of low prediction efficiency and insufficient accuracy.

[0004] The preceding description is to provide general background information and does not necessarily constitute the prior art. SUMMARY

[0005] The embodiments of the present application provide a multi-modal phosphorylation site prediction method, device, equipment and medium, which can effectively improve the accuracy and comprehensiveness of phosphorylation site prediction, while taking into account the computational efficiency and bioinformatics interpretability, solving the problems of low prediction efficiency and insufficient accuracy caused by traditional prediction methods relying only on sequence information or structure information.

[0006] In a first aspect, the embodiments of the present application provide a multi-modal phosphorylation site prediction method, comprising:

[0007] extracting a subsequence of a candidate phosphorylation site from a protein sequence to be predicted, and generating sequence features of the subsequence through a deep learning model;

[0008] constructing a graph structure based on three-dimensional structure information of the candidate phosphorylation site, and extracting structure features of the candidate site according to the graph structure through a graph neural network;

[0009] performing multi-modal feature fusion on the sequence features and the structure features to generate fusion features;

[0010] predicting the phosphorylation probability of the candidate site based on the fusion features to obtain a phosphorylation probability prediction result.

[0011] Optionally, in some embodiments of the present application, the subsequence of the candidate phosphorylation site is extracted from the protein sequence to be predicted, and the sequence feature of the subsequence is generated by a deep learning model, including:

[0012] obtaining a protein sequence to be predicted;

[0013] centered on the candidate phosphorylation site, the protein sequence to be predicted is intercepted according to a preset window length to generate a subsequence of the candidate phosphorylation site;

[0014] the length of the subsequence of the candidate phosphorylation site is normalized, including being completed or truncated to a preset sequence length;

[0015] the length-normalized subsequence is input into a pre-trained protein language model, and the model parameters are adjusted by low-rank adaptive technology, and the sequence embedding representation corresponding to the subsequence is output as the sequence feature.

[0016] Optionally, in some embodiments of the present application, the three-dimensional structure information of the candidate phosphorylation site is constructed based on the three-dimensional structure information of the candidate phosphorylation site, and the structure feature of the candidate site is extracted by a graph neural network according to the graph structure, including:

[0017] obtaining three-dimensional structure information of the candidate phosphorylation site;

[0018] According to the three-dimensional structure information of the candidate phosphorylation site, the spatial distance of adjacent amino acid residues is calculated to generate an adjacency matrix;

[0019] taking the candidate phosphorylation site as a center node, a local subgraph containing multi-hop neighbor nodes is extracted by breadth-first search;

[0020] the local subgraph is input into a graph attention network, the node features are aggregated by the multi-layer self-attention mechanism of the graph attention network, and the skip connection is introduced to retain the shallow network information, and the structure embedding representation corresponding to the local subgraph is generated as the structure feature of the candidate site.

[0021] Optionally, in some embodiments of the present application, the sequence feature and the structure feature are respectively mapped to the same dimensional feature space by a linear projection layer to generate a fusion feature, including:

[0022] the sequence feature and the structure feature are respectively mapped to the same dimensional feature space by a linear projection layer;

[0023] cross attention interaction is performed between the fusion latent representation and the projected sequence feature and structure feature to generate a fusion feature.

[0024] Optionally, in some embodiments of the present application, after the fusion feature is generated, the method further comprises:

[0025] performing multi-head self-attention calculation on the fusion feature;

[0026] performing residual connection and layer normalization processing on the fusion feature after multi-head self-attention calculation to generate an optimized fusion feature;

[0027] inputting the optimized fusion feature into a feedforward neural network for nonlinear transformation to obtain a fusion feature with enhanced feature expression.

[0028] Optionally, in some embodiments of the present application, the predicting, based on the fusion feature, the phosphorylation probability of the candidate site to obtain a phosphorylation probability prediction result comprises:

[0029] performing attention weight-based weighted pooling on the fusion feature to generate a fixed-length global feature vector;

[0030] inputting the global feature vector into a fully connected layer and outputting a phosphorylation probability prediction value through a Sigmoid activation function to generate a phosphorylation probability prediction result.

[0031] Optionally, in some embodiments of the present application, the method further comprises:

[0032] based on the phosphorylation probability prediction result, marking a candidate site with a phosphorylation probability prediction value higher than a preset threshold as a high-confidence phosphorylation site and generating a corresponding visual distribution map;

[0033] labeling a biological pathway or disease mechanism associated with the high-confidence phosphorylation site through a protein function annotation database.

[0034] In a second aspect, embodiments of the present application provide a multi-modal phosphorylation site prediction device, comprising:

[0035] a sequence feature module configured to extract a subsequence of a candidate phosphorylation site from a protein sequence to be predicted and generate sequence features of the subsequence through a deep learning model;

[0036] a structure feature module configured to construct a graph structure based on three-dimensional structure information of the candidate phosphorylation site and extract structure features of the candidate site through a graph neural network according to the graph structure;

[0037] a feature fusion module configured to perform multi-modal feature fusion on the sequence features and the structure features to generate fusion features;

[0038] a prediction module configured to predict, based on the fusion features, a phosphorylation probability of the candidate site to obtain a phosphorylation probability prediction result.

[0039] In a third aspect, an electronic device is provided, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the multi-modal phosphorylation site prediction method according to the first aspect when executing the computer program.

[0040] In a fourth aspect, a storage medium is provided, which stores a computer program capable of being loaded and executed by a processor to implement the multi-modal phosphorylation site prediction method according to the first aspect.

[0041] The present application provides a multi-modal phosphorylation site prediction method, device, equipment and medium. First, a subsequence of a candidate phosphorylation site is extracted from a protein sequence to be predicted, and a sequence feature of the subsequence is generated by a deep learning model. Then, a graph structure is constructed based on three-dimensional structure information of the candidate phosphorylation site, and a structure feature of the candidate site is extracted from the graph structure by a graph neural network. Next, the sequence feature and the structure feature are fused to generate a fusion feature. Finally, a phosphorylation probability of the candidate site is predicted based on the fusion feature to obtain a phosphorylation probability prediction result. In the multi-modal phosphorylation site prediction scheme provided in the embodiments of the present application, the subsequence is extracted from the protein sequence and the sequence feature is generated, which can capture complex sequence patterns. Then, the graph structure is constructed based on the three-dimensional structure information and the structure feature is extracted, which can obtain structure information that cannot be reflected by the sequence. Next, the two features are fused, so that the model can consider sequence and structure information at the same time. Finally, the phosphorylation probability is predicted based on the fusion feature to improve the prediction accuracy. It can be seen that, by fusing the multi-modal features of the protein sequence language model and the three-dimensional structure graph convolutional network, and combining the dual information of the protein sequence and structure for prediction, the present application improves the accuracy and comprehensiveness of the phosphorylation site prediction, while taking into account the computational efficiency and bioinformatics interpretability, and solves the problems of low prediction efficiency and insufficient accuracy caused by the traditional prediction method which only relies on sequence information or structure information. BRIEF DESCRIPTION OF DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creating any inventive labor.

[0043] Figure 1 is an application environment diagram of the multi-modal phosphorylation site prediction method provided by the embodiments of the present application;

[0044] Figure 2 is a flowchart of the multi-modal phosphorylation site prediction method provided by the embodiments of the present application;

[0045] Figure 3 FIG. 1 is a structural schematic diagram of a multi-modal feature fusion module provided by an embodiment of the present application;

[0046] Figure 4 FIG. 2 is a framework schematic diagram of a protein phosphorylation site prediction model provided by an embodiment of the present application;

[0047] Figure 5 FIG. 3 is a structural schematic diagram of a multi-modal phosphorylation site prediction device provided by an embodiment of the present application;

[0048] Figure 6 FIG. 4 is a structural schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0049] The exemplary embodiments will be described in detail herein below with reference to the drawings. In the following description, the same drawings refer to the same or similar elements. The following exemplary embodiments described in the exemplary embodiments do not represent all the embodiments in accordance with the present application. Rather, they are merely examples in accordance with some aspects of the present application as detailed in the appended claims.

[0050] It should be noted that, in this document, the terms "comprise", "comprising", or any other variant thereof are intended to cover a non-exclusive inclusion, such that processes, methods, articles, or apparatuses that comprise a list of elements are not limited to those elements, but can also include other elements not expressly listed, or inherent to such processes, methods, articles, or apparatuses. Without further limitation, an element defined by the phrase "comprising a" does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element. In addition, components, features, elements with the same name in different embodiments of the present application can have the same meaning or different meanings, and the specific meaning thereof should be determined in the explanation of the specific embodiment or further in conjunction with the context of the specific embodiment.

[0051] It should be understood that the specific embodiments described herein are merely intended to explain the present application, and are not intended to limit the present application.

[0052] In the following description, the suffixes "module", "part", or "unit" used for an element are merely intended for facilitating explanation of the present application, and it does not have a specific meaning by itself. Thus, "module", "part", or "unit" can be mixedly used.

[0053] With the rapid development of bioinformatics, the prediction of protein phosphorylation sites has become a key task for analyzing intracellular signaling mechanisms and finding potential therapeutic targets. Protein phosphorylation modification regulates the function, stability, and interaction of proteins by adding a phosphate group to specific amino acid residues. Accurate prediction of these phosphorylation sites is of great significance for revealing the biological processes of cells and the molecular mechanisms of related diseases. However, existing phosphorylation site prediction methods mostly rely on protein amino acid sequence information, ignoring the key role of protein three-dimensional structure in functional performance. The three-dimensional structure of a protein not only determines its functional activity but also affects the exposure and accessibility of phosphorylation sites. Therefore, traditional sequence-based prediction methods may not accurately identify all phosphorylation sites in some cases.

[0054] To solve the above technical problems and overcome the defects of the prior art, the embodiments of the present application provide a multi-modal phosphorylation site prediction method, device, equipment and medium, which can effectively improve the accuracy and comprehensiveness of phosphorylation site prediction, while taking into account the computational efficiency and bioinformatics interpretability, solving the problems of low prediction efficiency and insufficient accuracy caused by traditional prediction methods relying only on sequence information or structure information.

[0055] Figure 1 An application environment diagram of the multi-modal phosphorylation site prediction method in an embodiment is shown in FIG. 1. Referring to FIG. 1, Figure 1 The multi-modal phosphorylation site prediction method is applied to a multi-modal phosphorylation site prediction system. The multi-modal phosphorylation site prediction system includes a terminal 110 and a server 120. The terminal 110 and the server 120 are connected through a network, and the terminal 110 can be a desktop terminal or a mobile terminal, specifically at least one of a mobile phone, a tablet computer, a notebook computer, etc. The server 120 can be implemented by an independent server or a server cluster composed of multiple servers. The terminal 110 can be used to extract a subsequence of a candidate phosphorylation site from a protein sequence to be predicted, and generate sequence features of the subsequence through a deep learning model; construct a graph structure based on three-dimensional structure information of the candidate phosphorylation site, and extract structure features of the candidate site according to the graph structure through a graph neural network; perform multi-modal feature fusion on the sequence features and the structure features to generate fused features; and predict the phosphorylation probability of the candidate site based on the fused features to obtain a phosphorylation probability prediction result.

[0056] Please refer to Figure 2 , Figure 2 is a flowchart of the multi-modal phosphorylation site prediction method provided by an embodiment of the present application. The embodiment mainly takes the application of the multi-modal phosphorylation site prediction method to a computer device as an example for illustration. The multi-modal phosphorylation site prediction method provided by an embodiment of the present application can specifically include the following steps:

[0057] S1. Extracting sub-sequences of candidate phosphorylation sites from the protein sequence to be predicted, and generating sequence features of the sub-sequences through a deep learning model;

[0058] Specifically, for step S1, first, the protein sequence to be predicted is obtained, and then the sequence is truncated according to a preset window length to generate a sub-sequence centered on the candidate phosphorylation site. Then, the sub-sequence is subjected to length normalization processing, including completion or truncation to a preset sequence length. Finally, the processed sub-sequence is input into a pre-trained protein language model, and the model parameters are adjusted through low-rank adaptive technology, and the sequence embedding representation corresponding to the sub-sequence is output as the sequence feature. This step can effectively capture complex patterns and relationships in the sub-sequence. For example, the protein language model can learn the general features and rules of protein sequences by pre-training on large-scale protein sequence data. The low-rank adaptive technology decomposes and adjusts the model parameters through low-rank decomposition, so that the model can better adapt to the specific task of phosphorylation site prediction, while reducing the parameter amount and improving the fine-tuning efficiency.

[0059] S2. Constructing a graph structure based on the three-dimensional structure information of the candidate phosphorylation site, and extracting the structural features of the candidate site through a graph neural network according to the graph structure;

[0060] Specifically, for step S2, first, the three-dimensional structure information of the candidate phosphorylation site is obtained. Then, according to the three-dimensional structure information, the spatial distance between adjacent amino acid residues is calculated to generate an adjacency matrix. Then, taking the candidate phosphorylation site as the center node, a local subgraph containing multi-hop neighborhood nodes is extracted through breadth-first search. Finally, the local subgraph is input into a graph attention network, the node features are aggregated through a multi-layer self-attention mechanism, and a skip connection is introduced to preserve the shallow network information, to generate a structure embedding representation as the structure feature. Through the graph neural network, the spatial dependence relationship in the three-dimensional structure of the protein can be effectively captured. By constructing a graph structure, the three-dimensional structure of the protein can be converted into a representation of nodes and edges, and then the node features can be aggregated and updated using a graph neural network. The graph attention network can dynamically adjust the connection weights between nodes through a self-attention mechanism, and better capture the complex relationships between nodes. The skip connection can avoid the problem of gradient disappearance, preserve the information of the shallow network, and improve the performance of the model.

[0061] S3. Multi-modal feature fusion of sequence features and structure features to generate fusion features;

[0062] Specifically, for step S3, the sequence features and the structure features are respectively mapped to the same dimensional feature space through linear projection layers. Then, the fusion features are generated by performing cross-attention interaction between the latent representation and the projected sequence features and structure features. Through multi-modal feature fusion, the sequence and structure features can be integrated, so that the model can consider both one-dimensional sequence and three-dimensional spatial structure information of the protein. The linear projection layer maps features of different dimensions to the same space, facilitating subsequent fusion operations. The cross-attention mechanism allows different modalities of features to interact with each other, excavates the potential relationship between them, and generates more rich fusion features.

[0063] S4. Predict the phosphorylation probability of the candidate site based on the fusion features to obtain a phosphorylation probability prediction result;

[0064] Specifically, for step S4, the fusion features are weighted pooled based on attention weights to generate a fixed-length global feature vector. Then, the global feature vector is input into a fully connected layer, and a phosphorylation probability prediction value is output through a Sigmoid activation function to generate a phosphorylation probability prediction result. The weighted pooling based on attention weights can highlight important features and improve the prediction accuracy. The fully connected layer can further perform nonlinear transformation on the fusion features to enhance the expression ability of the model. The Sigmoid activation function limits the output value to between 0 and 1, representing the probability of phosphorylation occurrence.

[0065] The embodiment extracts sequence features and structure features by combining the dual information of protein sequence and structure, and performs multi-modal feature fusion, thereby overcoming the problems of low prediction efficiency and insufficient accuracy caused by relying on single information in traditional methods, and improving the accuracy and comprehensiveness of phosphorylation site prediction.

[0066] Optionally, in some embodiments, step S1 "extracts a subsequence of a candidate phosphorylation site from a protein sequence to be predicted, and generates sequence features of the subsequence through a deep learning model" can specifically include:

[0067] S11. Obtain a protein sequence to be predicted;

[0068] Specifically, protein sequence data to be predicted is obtained from various sources, such as from a genomic sequencing database, a protein database, or protein sequence data obtained directly from experiments, etc. In order to ensure that the obtained sequence data is accurate, a data preprocessing step is also needed, such as removing noise, correcting sequence errors, handling missing values, etc., to ensure the prediction accuracy of the subsequent model.

[0069] S12. Centering on a candidate phosphorylation site, a subsequence of the candidate phosphorylation site is generated by cutting the protein sequence to be predicted according to a preset window length;

[0070] Specifically, in the protein sequence, a sequence fragment around each candidate phosphorylation site is extracted as a subsequence with a preset window length (e.g., 41 amino acid residues) centered on the site. The selection of the window length is usually based on the study of the influence range of the sequence environment around the phosphorylation site. Qi et al., the selection of the window length is a key factor that affects the feature richness and computational complexity of the subsequence. A shorter window length may not contain enough context information, while a longer window length will increase the computational load. In addition, methods of variable window length or adaptive window length can also be considered to adapt to the characteristics of different protein sequences and phosphorylation sites at different positions.

[0071] S13. Length normalization processing is performed on the subsequence of the candidate phosphorylation site, including padding or truncation to a preset sequence length;

[0072] Specifically, since the actual extracted subsequence may not conform to the preset window length due to the length of the protein sequence or the position of the candidate site, length normalization processing is needed for these subsequences. For example, for subsequences that are not long enough, special markers (such as 'X') are used for padding; for subsequences that are too long, they are truncated. Length normalization processing not only makes all subsequences have a uniform length before inputting the model, which is convenient for model processing, but also avoids the problem of inconsistent information caused by sequence length differences. In addition, in the padding operation, how to choose the appropriate padding method (such as using fixed markers, randomly generated amino acid residues, etc.) will also affect the learning effect of the model.

[0073] S14. The length-normalized subsequence is input into a pre-trained protein language model, and the model parameters are adjusted through low-rank adaptive technology, and the sequence embedding representation corresponding to the subsequence is output as the sequence feature;

[0074] Specifically, the processed sub-sequences are input into a pre-trained protein language model, such as ESM-2, etc. Protein language models have learned general features and patterns of protein sequences through pre-training on large-scale protein sequence data. Low-rank matrix decomposition parameters are introduced in the attention layer of the protein language model, and the model weights are adjusted by training the matrix parameters to adapt to the phosphorylation site prediction task. Through the low-rank adaptive (LoRA) technology, the model parameters are adjusted to better adapt to the specific task of phosphorylation site prediction. Protein language models can capture long-range dependencies and complex patterns in sequences. LoRA technology adjusts model weights by introducing low-rank matrices, avoiding fine-tuning of all model parameters, thereby reducing parameter quantity and computational resource consumption, and improving fine-tuning efficiency. In addition, other model fine-tuning techniques or feature extraction methods can be combined to further optimize the extraction of sequence features.

[0075] The embodiment improves the accuracy and quality of sequence features by detailed definition of the sequence feature extraction process, especially by obtaining accurate sequence data, generating sub-sequences containing local context information, length normalization processing, and extracting sequence features using a protein language model combined with low-rank adaptive technology, ensuring the accuracy and reliability of subsequent multi-modal feature fusion and phosphorylation site prediction.

[0076] In a specific embodiment, for step S1, the LoRA fine-tuned protein language model ESM-2 is mainly used to extract sub-sequence features.

[0077] First, candidate site identification and sub-sequence extraction, the given protein sequence is scanned to extract candidate phosphorylation site sub-sequences with a fixed window size (for example, the window size is 41, and the center site is the phosphorylation site to be predicted). Then each sub-sequence is formatted, and for sequences less than 41 in length, special markers 'X' are used to complete them, with 'X' representing unknown amino acids in biology. After obtaining the sub-sequences, the protein language model ESM-2 is fine-tuned to extract the features of the sub-sequences. ESM-2 is a high-efficiency pre-trained protein language model that can effectively capture complex patterns and relationships in protein sequences. By fine-tuning the ESM-2 model, it can better adapt to specific tasks and improve its performance in phosphorylation site prediction. The embodiment uses LoRA (Low-Rank Adaptation) technology to fine-tune the ESM-2 model. LoRA adjusts the model weights by introducing low-rank matrices in specific layers of the model (such as the attention layer), rather than directly fine-tuning all parameters, which reduces the required parameter quantity and improves the efficiency of fine-tuning.

[0078] LoRA (Low-Rank Adaptation) is an efficient fine-tuning technique for large language models, aiming to adjust the model's weights by introducing trainable low-rank decomposition matrices while freezing the pre-trained model's weights. This approach significantly reduces the number of parameters that need to be trained in downstream tasks, improving training efficiency while maintaining model quality and reducing storage and computational requirements for model deployment. In traditional fine-tuning, all parameters of the model are updated to adapt to specific downstream tasks, meaning that the fine-tuned model will contain the same number of parameters as the original model. However, as model sizes continue to grow, full-parameter fine-tuning becomes impractical due to the enormous storage space and computational resources required. The core idea of LoRA is that the changes (updates) in weights during model adaptation can be approximated by low-rank decomposition.

[0079] For a pre-trained weight matrix Its update ΔW when adapting to a task can be represented as the product of two smaller matrices BA, where And r << min(d, k). Therefore, the parameters that are actually trained during adaptation are A and B, rather than directly updating W. This representation significantly reduces the number of parameters that need to be trained.

[0080] The mathematical expression of LoRA is as follows: for an input x and a pre-trained weight W, the traditional linear transformation can be represented as h = Wx. In LoRA, the modified forward propagation of the model becomes:

[0081] h = Wx + ΔWx = Wx + BAx

[0082] Where ΔW = BA is the update matrix of low-rank decomposition, obtained by training A and B. When applying LoRA to ESM-2 fine-tuning, the sequence feature S is converted to embedding representation E s after fine-tuning the model, the mathematical expression is as follows:

[0083] E s = W s S + ΔW s S = W s S + B s A s S

[0084] Where W s is the weight matrix of the pre-trained model, S is the input sequence feature, and the weight update matrix ΔW s after LoRA fine-tuning s B s A s is optimized by low-rank decomposition. Finally, Eis the sequence feature embedding generated after LoRA fine-tuning, which can better adapt to the phosphorylation site prediction task.

[0085] Optionally, in some embodiments, the step S2 of "constructing a graph structure based on the three-dimensional structure information of the candidate phosphorylation site, and extracting the structural features of the candidate site according to the graph structure through a graph neural network", in particular, can include:

[0086] S21. Obtain the three-dimensional structure information of the candidate phosphorylation site;

[0087] Specifically, obtaining the three-dimensional structure information of the candidate phosphorylation site is the first step of structural feature extraction. The three-dimensional structure information can be obtained in various ways, such as directly obtaining known structures from protein structure databases (such as PDB), or predicting unknown structures using structure prediction tools (such as AlphaFold2). Structure prediction tools such as AlphaFold2 use deep learning techniques to accurately predict the three-dimensional structure of proteins, even in cases where experimental data is insufficient, providing reliable structure information. In addition, methods such as homology modeling can also be used to infer the three-dimensional structure of the target protein based on known structures of proteins.

[0088] S22. Calculate the spatial distance between adjacent amino acid residues according to the three-dimensional structure information of the candidate phosphorylation site, and generate an adjacency matrix;

[0089] Specifically, according to the obtained three-dimensional structure information, the spatial distance between each amino acid residue and its adjacent residues is calculated, and an adjacency matrix is generated according to the spatial distance to represent the spatial adjacency relationship between residues in the protein structure. The calculation of spatial distance is usually based on three-dimensional coordinates, and can use Euclidean distance and other metrics. The generation of the adjacency matrix can determine whether there is an adjacency relationship between residues according to the distance threshold, for example, if the distance between two residues is less than a certain threshold, they are considered to be adjacent, and the corresponding position in the adjacency matrix is marked as 1, otherwise marked as 0. In addition, other factors such as the type of chemical bonds between residues can also be considered to further enrich the information of the adjacency matrix.

[0090] S23. Take the candidate phosphorylation site as the center node, and extract the local subgraph containing multi-hop neighborhood nodes through breadth-first search;

[0091] Specifically, taking the candidate phosphorylation site as the center node, the breadth-first search (BFS) algorithm is used to expand the multi-hop neighbor nodes outward in the graph structure, and the local subgraph containing these nodes is extracted. Multi-hop neighborhood means not only including nodes directly connected to the center node, but also including nodes within a certain range indirectly connected to the center node. Breadth-first search is a commonly used graph traversal algorithm that can systematically explore the nodes in the graph structure. When extracting the local subgraph, the size and range of the subgraph can be controlled by setting the hop number parameter, thereby balancing the detail level and computational complexity of feature extraction. For example, by setting the hop number to 2, the structural information of the two layers of neighbor nodes around the center node can be extracted.

[0092] S24. Input the local subgraph into the graph attention network, aggregate the node features through the multi-layer self-attention mechanism of the graph attention network, and introduce a skip connection to retain the shallow network information, to generate a structural embedding representation corresponding to the local subgraph as the structural feature of the candidate site;

[0093] Specifically, the extracted local subgraph is input into the graph attention network (GAT), and the node features are aggregated through the multi-layer self-attention mechanism. Among them, the attention coefficient of the graph attention network (GAT) is calculated in the following way: for node i and its neighbor node j, the attention score is generated based on the node feature and the learnable weight matrix, and the attention weight is obtained through the LeakyReLU activation function and Softmax normalization. The self-attention mechanism can dynamically adjust the connection weight according to the importance between nodes, so as to better capture the complex relationship between nodes. At the same time, the skip connection is introduced to pass the information of the shallow network to the deep network, avoiding the problem of gradient disappearance, and enhancing the model's ability to capture different levels of information. The finally generated structural embedding representation is the structural feature of the candidate site. The multi-layer structure of the graph attention network can gradually extract high-order features of the nodes, and the self-attention mechanism of each layer can learn the relationships between nodes at different levels. The introduction of the skip connection not only helps to alleviate the gradient disappearance problem of the deep network, but also enables the model to better integrate feature information at different levels, improving the richness and accuracy of feature expression. In addition, other graph neural network architectures or attention mechanism variants can also be combined to further optimize the extraction effect of the structural feature.

[0094] This embodiment effectively captures the local spatial relationships and residue interactions in the three-dimensional structure of the protein by detailing the structural feature extraction process, especially by obtaining three-dimensional structure information, generating an adjacency matrix, extracting a local subgraph, and extracting a structural feature using a graph attention network combined with a skip connection. This improves the quality and expressiveness of the structural feature, thereby enhancing the accuracy and reliability of the phosphorylation site prediction.

[0095] In specific embodiments, for step S2, the substructure features are mainly extracted based on a graph attention network, and the specific process is as follows:

[0096] First, the candidate site substructure extraction module is used to obtain the structural features of the protein through the three-dimensional structure generated by AlphaFold2. Then, the structural features between nodes in the graph are extracted based on the graph attention network (GAT), and the complex relationship between nodes is modeled through the self-attention mechanism. Next, the Graph Query Attention module is introduced in this embodiment, which extracts fixed-length graph structure features through a learnable query vector and a cross-attention mechanism.

[0097] (1) Candidate site substructure extraction module

[0098] First, the three-dimensional structure of the protein is generated using AlphaFold2 to obtain an accurate protein conformation. Next, the adjacency matrix A∈{0,1} |V|×|V| between amino acids is calculated according to the distance matrix D, which is used to represent the connection relationship between each amino acid in the protein. Then, the target position is selected as the center point, and the hop number is set to extract the substructure around the position point. The subgraph centered on the candidate site is extracted through breadth-first search (BFS), and the corresponding candidate site subgraph is constructed. Finally, the initial feature matrix

[0099] (2) Extracting substructure features based on GAT

[0100] The graph attention network (GAT) is a graph neural network based on attention mechanism, which aggregates the information of neighboring nodes by learning the different importance (attention coefficients) between nodes. Compared with the traditional graph convolution network (GCN), GAT can more flexibly capture the complex relationships in the graph structure. The core idea of GAT is to dynamically adjust the connection weights between nodes through attention mechanism. For each node in the graph, GAT calculates the attention coefficients between it and its neighbor nodes, which reflect the contribution degree of neighbor nodes to the feature representation of the node. Specifically, for node i and its neighbor node j, the attention coefficient α ij is calculated by the following formula:

[0101] e i，j =score([Wh i ||Wh j ]),j∈N i

[0102]

[0103] where h i and h j are the feature vectors of node i and i respectively, w is a learnable weight matrix, a is a weight vector used to calculate attention coefficients, LeakyReLU is an activation function, and softmax ensures that the sum of all attention coefficients is 1. The new feature representation h' of node i is obtained by weighted summation of the features of its neighbors (including itself): i

[0104]

[0105] where denotes the set of neighbors of node i, and σ is an activation function.

[0106] This embodiment uses three layers of GAT to extract the substructure features of the graph, and the output of each layer of GAT represents the node representation of the graph structure at a certain depth. Since the connection between GAT layers has the function of information transmission, this embodiment introduces skip connections between each GAT to ensure that the deep network can effectively transmit information and avoid problems such as gradient disappearance. Specifically, the node representation of the input graph is where N is the number of nodes and d is the dimension of the node feature. For the l-th layer of GAT, its output can be represented as:

[0107]

[0108] where the initial feature of the node H 0 = X, is the adjacency matrix, representing the connection relationship between nodes in the graph, W (l) is the learning parameter of the l-th layer, is the attention coefficient of the l-th layer, and σ is an activation function. The output H (l) (l = 1, 2, 3) of each layer represents the high-dimensional feature representation of each node in the graph. These features contain high-order information of the node itself and its neighbors. By introducing skip connections, the information in the deep network can be better preserved and transmitted, thereby improving the performance of the model. The skip connection allows information to be transmitted directly in the network without passing through each layer of transformation, which helps to alleviate the problem of gradient disappearance in deep networks and enhances the model's ability to capture local and global information.

[0109] (3) Extracting the graph structure based on the GraphQueryAttention module

[0110] ​In view of the fact that the number of nodes of the input graph structure is not fixed, the embodiment proposes a GraphQueryAttention module to convert it into a regular graph structure with a fixed number of nodes by using a set of learnable query vectors, and the process involves a cross-attention mechanism.

[0111] The GraphQueryAttention module processes the node features by using a learnable query vector (Query) with the help of a cross-attention mechanism. It is assumed that the input node feature matrix is X∈R where B represents the batch size, N is the number of nodes, and C is the feature dimension; the adjacency matrix is A∈R M learnable query vectors are set in the module Here, D represents the hidden dimension. At the same time, the input node features are mapped into keys (Key) and values (Value) by linear transformation, i.e., K = XW K , V = XW V where W is a learnable weight matrix.

[0112] To implement the multi-head cross-attention mechanism, the dimensions of the query, key and value are split. It is assumed that the number of attention heads is H, and the dimension of each head is d = D / H. The split keys K and values V are After dimension permutation, they become The query vector Q is also expanded and split to obtain

[0113] The attention score (AttentionScore) of each head is calculated, and the formula is:

[0114]

[0115] where

[0116] If there is a mask used to mark invalid nodes filled with padding, the mask is expanded to and the attention score is shielded:

[0117] Score = Score · M + (1 - M) · (-∞)

[0118] The attention score of each head is normalized by softmax to obtain the attention weight

[0119] W attn = softmax (Score, dim = -1)

[0120] The outputs of the plurality of heads are concatenated to obtain a final output

[0121] O h = W attn V

[0122] The outputs of the plurality of heads are concatenated to obtain a final output

[0123] E t = Concat(O1, O2, …, O H )

[0124] Through the above operations, the GraphQueryAttention module converts the input graph structure features into fixed-length vector representations with the help of cross-attention mechanism, providing a basis for subsequent feature fusion and network training.

[0125] Optionally, in some embodiments, the step S3 of "performing multi-modal feature fusion on the sequence features and the structure features to generate fused features" can specifically include:

[0126] S31. mapping the sequence features and the structure features to the same dimensional feature space through linear projection layers, respectively;

[0127] Specifically, the sequence features and the structure features usually have different dimensions. In order to effectively fuse them, it is necessary to first map them to the same dimensional feature space through linear projection layers. The linear projection layer can be regarded as a simple neural network layer, which converts the input features into new feature representations by learning trainable weight matrices and bias terms. The use of linear projection layers ensures that the features from different sources have the same dimension and comparability before fusion. This step is the premise of feature fusion, because only when the features are in the same feature space, the subsequent interaction and fusion operations can be performed.

[0128] S32. performing cross-attention interaction on the fused latent representation and the projected sequence features and structure features to generate fused features;

[0129] Specifically, after mapping the sequence features and the structure features to the same dimension, a fused latent representation is introduced and interacts with the projected sequence features and the structure features through cross-attention. The cross-attention mechanism allows the features of different modalities to interact with each other, enhancing the fusion effect by calculating the correlation between the features. For example, the fused latent representation is taken as a query vector, and attention calculation is performed with the key-value pairs of the sequence features and the structure features, respectively, and the fused features are generated through weighted summation. The cross-attention mechanism is one of the key technologies in the Transformer architecture, which captures the dependency between features by calculating the dot product attention between query, key and value. In this process, the fused latent representation can be regarded as a learnable query vector, which interacts with the sequence features and the structure features to generate more rich fused features.

[0130] By detailing the multi-modal feature fusion process, especially by aligning the dimensions of the sequence features and the structure features through the linear projection layer, and realizing the deep interaction between the features through the cross-attention mechanism, the quality and expression ability of the fused features are significantly improved, thereby improving the accuracy and reliability of the phosphorylation site prediction.

[0131] Optionally, in some embodiments, after step S32, the multi-modal phosphorylation site prediction method provided by the embodiment can further include:

[0132] S33. Perform multi-head self-attention calculation on the fused features;

[0133] Specifically, the multi-head self-attention mechanism takes the fused features as input, calculates the correlation between the features through multiple attention heads respectively, and then splices or averages the outputs of the multiple attention heads to obtain the features after multi-head self-attention calculation. Each attention head independently calculates query, key and value, and obtains attention weight through dot product attention mechanism. The multi-head self-attention mechanism can capture the correlation of the features in different dimensions, and each attention head can focus on different aspects of the features. Through parallel calculation of multiple attention heads, the model can more comprehensively understand the complex relationship between the features. In addition, the multi-head self-attention mechanism is widely used in the Transformer architecture, which proves its effectiveness in processing sequential and structured data.

[0134] S34. Perform residual connection and layer normalization processing on the fused features after multi-head self-attention calculation to generate optimized fused features;

[0135] Specifically, the residual connection adds the input features before the multi-head self-attention calculation to the calculated features, forming a residual block. Layer normalization normalizes the features after the residual connection, ensuring the stability of the features and accelerating the training process of the model. The residual connection can alleviate the gradient vanishing problem in deep networks, allowing gradients to be more directly backpropagated, thereby improving the training efficiency and performance of the model. Layer normalization stabilizes the training process of the model by normalizing the input of each layer, reduces internal covariate shift, and improves the convergence speed and stability of the model.

[0136] S35. The optimized fusion features are input into a feed-forward neural network for nonlinear transformation to obtain fusion features with enhanced feature expression;

[0137] Specifically, a feed-forward neural network (FFNN) usually consists of two fully connected layers with a nonlinear activation function (such as ReLU) in between. The optimized fusion features are nonlinearly transformed by the feed-forward neural network to enhance the expression ability of the features. The feed-forward neural network can introduce nonlinearity, allowing the model to learn more complex feature representations. Through nonlinear transformation, the model can capture high-order relationships between features, further improving the expression ability of the fusion features and the prediction performance of the model.

[0138] This embodiment significantly enhances the expression ability of the fusion features and the prediction performance of the model through further processing of the fusion features, including multi-head self-attention calculation, residual connection and layer normalization, and nonlinear transformation of the feed-forward neural network, thereby improving the accuracy and reliability of phosphorylation site prediction.

[0139] In specific embodiments, for the multi-modal feature fusion of step S3, the specific process is as follows:

[0140] Since the dimensions of sequence features and structure features are inconsistent, they need to be mapped to the same feature space through a projection layer. This embodiment projects the embedding representation E s of sequence features and the embedding representation E t of structure features to the same feature space dimension through a linear projection layer respectively. The mathematical expression of the linear projection layer is as follows:

[0141] E′ s = W s E s + b s

[0142] E′ t = W t Et +b t

[0143] where W s and W t are learnable weight matrices, b s and b t are bias terms.

[0144] The multi-modal feature fusion module proposed in this embodiment is shown in FIG. 1. Unlike traditional feature fusion methods, the method of this embodiment can deeply mine the potential relationship between sequence features and structure features and achieve more effective feature fusion by introducing fused latent and cross-attention mechanisms. Figure 3

[0145] First, the sequence feature E s and the structure feature E t are processed through multi-head self-attention mechanisms, respectively. The purpose of the multi-head self-attention mechanism is to capture the internal dependency in each feature, so that each feature vector not only contains local information but also considers global context information. For the sequence feature, the representation after the self-attention mechanism is E' s , and the representation of the structure feature after the self-attention mechanism is E' t . The formula of these steps is:

[0146] E' s = MultiHeadSelfAttention(E s )

[0147] E' t = MultiHeadSelfAttention(E t )

[0148] In this process, the sequence feature E s and the structure feature E t are processed separately, enhancing their representation ability in the self-field and providing more rich information for subsequent fusion.

[0149] Next, in order to further enhance the fusion effect, this embodiment introduces a set of learnable fused latent representations L, which can serve as a fixed-size representation to help the model capture the potential relationship between sequence features and structure features. In the feature fusion process, the sequence feature E' s and the structure feature E' t processed by the self-attention mechanism are first spliced to obtain the merged feature matrix E concat , and the formula is:

[0150] E​concat = Concat(E' s , E' t )

[0151] The concatenated feature matrix contains all the information of the sequence and structure, which can provide comprehensive feature representation for subsequent fusion.

[0152] In the cross-attention calculation between the fusion latent representation L and the concatenated feature E concat , the latent representation interacts with the concatenated feature, further mining the deep relationship between the sequence and structure features. The specific cross-attention calculation formula is as follows:

[0153] E fused = CrossAttention(L, E concat )

[0154] In this process, the cross-attention mechanism can effectively capture the mutual relationship between the fusion latent representation and the sequence and structure features, thereby generating the fused latent feature E fused , which integrates the sequence and structure features and their interaction information.

[0155] Next, this embodiment respectively performs further cross-attention calculation on the sequence feature E' s and the structure feature E' t to enhance the sequence feature and the structure feature. First, the sequence feature and the fusion latent representation E fused perform cross-attention calculation to obtain the updated sequence feature E" s , and the formula is as follows:

[0156] E" s = E' s + a s · CrossAttention(E' s , E fused )

[0157] Wherein, a s is a learnable scaling coefficient for controlling the interaction strength between the sequence feature and the fusion latent representation. Similarly, the structure feature and the fusion latent representation E fused perform cross-attention calculation to obtain the updated structure feature E" t , and the formula is as follows:

[0158] E" t = E' t + a t · CrossAttention(E' t , E fusedIn this process, both the structure features and the sequence features interact with the fused latent representation, further enhancing their relationship with other features. Finally, the updated sequence features E" s and the structure features E" t can be used as input for subsequent tasks, such as phosphorylation site prediction.

[0159] Optionally, in some embodiments, the step S4 "predicting the phosphorylation probability of the candidate site based on the fused features, obtaining the phosphorylation probability prediction result" can specifically include:

[0160] S41. Perform attention weight-based weighted pooling on the fused features to generate a fixed-length global feature vector;

[0161] Specifically, attention weight-based weighted pooling is a pooling method based on self-attention mechanism. Specifically, first, a linear transformation is used to generate the attention score of each feature, which represents the "importance" of the feature. Then, the scores are normalized using the Softmax function to obtain the attention weights. Finally, the normalized weights are used to weight the sum of the input features to obtain the fixed-length global feature vector after pooling. Dynamically assigning weights to each feature can focus on the most important features and generate more effective feature representations. Compared with traditional pooling methods such as average pooling or max pooling, attention pooling can better capture the importance of features, thereby improving the performance of the model.

[0162] S42. Input the global feature vector into a fully connected layer, output the phosphorylation probability prediction value through a Sigmoid activation function, and generate the phosphorylation probability prediction result;

[0163] Specifically, the fully connected layer performs further nonlinear transformation on the global feature vector to enhance the expression ability of the model. The Sigmoid activation function limits the output value to between 0 and 1, representing the probability of phosphorylation occurring. Specifically, the output of the fully connected layer is converted to a probability value through the Sigmoid function, representing the likelihood of the candidate site being phosphorylated. The fully connected layer can learn the complex patterns of the global feature vector, while the Sigmoid function can map the output value to the probability space, making it easier to interpret and use. This combination is widely used in binary classification problems and can effectively output prediction probabilities.

[0164] This embodiment effectively generates a fixed-length global feature vector and accurately outputs a phosphorylation probability prediction value by further processing the fused features, including attention weight-based weighted pooling and the prediction method of the fully connected layer combined with the Sigmoid activation function, thereby improving the accuracy and reliability of phosphorylation site prediction.

[0165] In specific embodiments, the specific process of network training in step S4 is as follows:

[0166] First, the sequence features and structure features are respectively subjected to a pooling operation, with the purpose of effectively integrating the two kinds of features. This embodiment adopts the SequenceAttentionPooling (sequence attention pooling) and StructureAttentionPooling (structure attention pooling) methods to extract the most representative feature vectors from the sequence features and structure features, thereby obtaining a more accurate feature representation.

[0167] AttentionPooling is a pooling method based on the self-attention mechanism, aiming to weight the aggregation of features by calculating the importance of each feature, thereby generating a fixed-length representation. Specifically, given the input features where B is the batch size, N is the number of features or node numbers, and D is the dimension of each feature, first generate the attention score a i of each feature through a linear transformation

[0168] a i = Linear(x i )

[0169] Then, the scores are normalized using the Softmax function to obtain the attention weights a weights :

[0170] a weights = Softmax(a)

[0171] Next, the input features are weighted and summed using these normalized weights a weights to obtain the pooled feature representation e:

[0172]

[0173] In this way, AttentionPooling dynamically assigns weights to each feature, thereby focusing on the most important features and generating a more effective feature representation, ultimately improving the performance of the model.

[0174] When processing sequence features, SequenceAttentionPooling multiplies each node feature in the sequence with the corresponding attention weight, and then aggregates them into a fixed-length feature representation. Let the sequence features be where N s is the number of nodes in the sequence, and d is the feature dimension of each node. The pooled representation e s of the sequence is calculated by the following formula:

[0175] e s s )

[0176] Similarly, the structural features are also pooled by StructureAttentionPooling. Let the structural features be where N t is the number of nodes in the structure, and the pooled structural representation e t is computed by the following formula:

[0177] e t t )

[0178] After the pooling operation, the sequence feature e s and the structural feature e t are concatenated into a higher-dimensional feature vector e concat , which contains the global information of the sequence feature and the structural feature:

[0179] e concat s t )

[0180] Next, the concatenated feature vector e concat is input to a fully connected layer for classification. Let the weights of the fully connected layer be W and the bias be b, then the output y of the fully connected layer is computed as:

[0181] y = σ(We concat +b)

[0182] where σ is an activation function (such as Sigmoid or Softmax) used to generate the prediction probability for each class.

[0183] To train the model, this embodiment uses the cross-entropy loss function to optimize the parameters of the network. Let y s be the true label of sample s, be the predicted value of the network for sample s, the formula of the cross-entropy loss function is as follows:

[0184]

[0185] where D is the training data set, |·| represents the size of the data set, is the predicted value of the network for sample s, y s is the true label of the sample.

[0186] ​​​​By minimizing this loss function, the network can continuously optimize its parameters, thereby improving the classification accuracy of the phosphorylation site prediction task.

[0187] Optionally, in some embodiments, the multi-modal phosphorylation site prediction method provided by the present embodiment further comprises:

[0188] S51. Based on the phosphorylation probability prediction result, mark the candidate sites with phosphorylation probability prediction values higher than the preset threshold as high-confidence phosphorylation sites, and generate a corresponding visual distribution map;

[0189] Specifically,

[0190] According to the predicted phosphorylation probability, mark the candidate sites with probability values higher than the preset threshold as high-confidence phosphorylation sites. These sites are considered to be more likely to occur phosphorylation. At the same time, a visual distribution map is generated to intuitively show the position distribution of these high-confidence phosphorylation sites in the protein sequence or structure. The preset threshold can be adjusted according to the specific application requirements or the characteristics of the data set, to balance the sensitivity and specificity of the prediction. The visual distribution map can be generated in various ways, such as marking the sites on the protein sequence, or highlighting the phosphorylation sites on the three-dimensional structure model. For example, professional bioinformatics visualization tools or software libraries are used to realize this function.

[0191] S52. Mark the biological pathways or disease mechanisms associated with high-confidence phosphorylation sites through protein function annotation databases;

[0192] Specifically, use protein function annotation databases such as Gene Ontology (GO), KEGG, etc. to find biological pathways and disease mechanisms related to high-confidence phosphorylation sites. These databases provide rich annotation information that can help researchers understand the role of phosphorylation sites in cell function and disease. Protein function annotation databases usually contain a large amount of experimental data and literature information. Through association analysis with these databases, the potential function of phosphorylation sites in specific biological processes or diseases can be revealed. API or other data query interfaces can be used to retrieve relevant information from the database and integrate it into the prediction results.

[0193] Through further processing and analysis of the prediction results, including marking high-confidence phosphorylation sites, generating visual distribution maps, and associating biological pathways and disease mechanisms, the present embodiment significantly improves the practicality and interpretability of phosphorylation site prediction, providing researchers with deeper biological insights, thereby enhancing the application value of phosphorylation site prediction in biomedical research.

[0194] In some embodiments, as Figure 4 shown,Figure 4 A framework diagram of a multi-modal phosphorylation site prediction model based on a protein language model ESM-2 and a graph convolutional network (GNN) is provided. First, the model fine-tunes the ESM-2 protein language model using the LoRA technology to extract the features of the phosphorylation site from the sequence; the ESM-2 protein language model is fine-tuned using LoRA to extract the candidate site sequence features, LoRA can adjust the model weights by introducing a low-rank matrix, reduce the parameter amount and improve the fine-tuning efficiency, and ensure that the model can better adapt to the specific task requirements. Second, based on the graph attention network (GAT), the model extracts the local structure features from the three-dimensional structure of the protein, and captures the spatial dependence relationship between the amino acid residues; the GNN is used to extract the local structure features from the predicted three-dimensional structure of the protein, and the spatial dependence relationship of the protein is represented by the graph structure, and the structure information that cannot be reflected by the sequence information is captured. Then, the Graph Query Attention module further extracts the fixed-length graph structure features through the learnable query vector and the cross-attention mechanism; finally, the model weights and fuses the sequence features and the graph structure features through the multi-modal feature fusion module, and adopts the feature fusion method based on the attention mechanism to fuse the sequence features and the structure features together, and combines the multi-modal model to accurately predict the phosphorylation site. Thus, the protein sequence and structure information can be better captured, thereby providing accurate feature representation for the prediction of the phosphorylation site and improving the accuracy of the prediction of the phosphorylation site.

[0195] Compared with the prior art, the multi-modal phosphorylation site prediction method provided in the embodiment fuses the multi-modal features of the protein sequence language model and the three-dimensional structure graph convolutional network, combines the dual information of the protein sequence and structure for prediction, improves the accuracy and comprehensiveness of the phosphorylation site prediction, and at the same time, takes into account the calculation efficiency and bioinformatics interpretability, solves the problems of low prediction efficiency and insufficient accuracy caused by the traditional prediction method which only relies on sequence information or structure information.

[0196] It should be understood that, although Figure 2 The steps in the flowchart of the method are displayed in sequence according to the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, the execution of these steps is not strictly limited in sequence, and these steps can be executed in other orders. Moreover, Figure 2 At least part of the steps in the method can include multiple sub-steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution order of the sub-steps or stages is not necessarily sequential, but can be executed in rotation or alternation with at least part of other steps or sub-steps or stages of other steps.

[0197] To facilitate better implementation of the multi-modal phosphorylation site prediction method of the embodiments of the present application, the embodiments of the present application also provide a multi-modal phosphorylation site prediction device based on the above-mentioned multi-modal phosphorylation site prediction method. The meanings of the terms are the same as in the above-mentioned multi-modal phosphorylation site prediction method, and the specific implementation details can be referred to the description in the method embodiment.

[0198] Please refer to Figure 5 , Figure 5 The multi-modal phosphorylation site prediction device provided by the embodiments of the present application has a structure diagram, which can specifically include a sequence feature module 201, a structure feature module 202, a feature fusion module 203 and a prediction module 204, and can be as follows:

[0199] The sequence feature module 201 is configured to extract a subsequence of a candidate phosphorylation site from a protein sequence to be predicted, and generate sequence features of the subsequence through a deep learning model.

[0200] The structure feature module 202 is configured to construct a graph structure based on three-dimensional structure information of the candidate phosphorylation site, and extract structure features of the candidate site according to the graph structure through a graph neural network.

[0201] The feature fusion module 203 is configured to perform multi-modal feature fusion on the sequence features and the structure features to generate fusion features.

[0202] The prediction module 204 is configured to predict a phosphorylation probability of the candidate site based on the fusion features to obtain a phosphorylation probability prediction result.

[0203] Optionally, in some embodiments, the sequence feature module 201 can specifically include:

[0204] The first acquisition unit is configured to acquire a protein sequence to be predicted.

[0205] The cutting unit is configured to cut the protein sequence to be predicted according to a preset window length with the candidate phosphorylation site as the center to generate a subsequence of the candidate phosphorylation site.

[0206] The normalization unit is configured to perform length normalization processing on the subsequence of the candidate phosphorylation site, including completing or truncating to a preset sequence length.

[0207] The sequence feature unit is configured to input the length normalized subsequence into a pre-trained protein language model, and adjust the model parameters through low-rank adaptive technology, and output the sequence embedding representation corresponding to the subsequence as the sequence features.

[0208] Optionally, in some embodiments, the structure feature module 202 can specifically include:

[0209] The second acquisition unit is configured to acquire three-dimensional structure information of the candidate phosphorylation site.

[0210] The adjacency matrix unit is configured to calculate spatial distances of adjacent amino acid residues according to the three-dimensional structure information of the candidate phosphorylation site, and generate an adjacency matrix.

[0211] The local subgraph unit is configured to take the candidate phosphorylation site as a center node, and extract a local subgraph containing multi-hop neighbor nodes through a breadth-first search.

[0212] The structure feature unit is configured to input the local subgraph into a graph attention network, aggregate node features through a multi-layer self-attention mechanism of the graph attention network, and introduce a skip connection to retain shallow network information, so as to generate a structure embedding representation corresponding to the local subgraph as a structure feature of the candidate site.

[0213] Optionally, in some embodiments, the feature fusion module 203 can specifically include:

[0214] The projection unit is configured to map the sequence features and the structure features into a feature space with the same dimension through a linear projection layer, respectively.

[0215] The fusion unit is configured to perform cross-attention interaction on the fusion latent representation and the projected sequence features and structure features, so as to generate a fusion feature.

[0216] Optionally, in some embodiments, the feature fusion module 203 can specifically further include:

[0217] The self-attention unit is configured to perform multi-head self-attention calculation on the fusion feature.

[0218] The optimization unit is configured to perform residual connection and layer normalization processing on the fusion feature after the multi-head self-attention calculation, so as to generate an optimized fusion feature.

[0219] The enhancement unit is configured to input the optimized fusion feature into a feedforward neural network to perform nonlinear transformation, so as to obtain a fusion feature with enhanced feature expression.

[0220] Optionally, in some embodiments, the prediction module 204 can specifically include:

[0221] The pooling unit is configured to perform attention weight-based weighted pooling on the fusion feature, so as to generate a fixed-length global feature vector.

[0222] The calculation unit is configured to input the global feature vector into a fully connected layer, and output a phosphorylation probability prediction value through a Sigmoid activation function, so as to generate a phosphorylation probability prediction result.

[0223] Optionally, in some embodiments, the device can specifically further include a visualization module, which specifically includes:

[0224] a marking unit configured to mark the candidate sites with phosphorylation probability prediction values higher than a preset threshold as high-confidence phosphorylation sites based on the phosphorylation probability prediction results, and generate a corresponding visual distribution map;

[0225] an association unit configured to mark biological pathways or disease mechanisms associated with the high-confidence phosphorylation sites through a protein function annotation database.

[0226] The specific limitations of the multi-modal phosphorylation site prediction device can refer to the limitations of the multi-modal phosphorylation site prediction method described above, which will not be repeated here. Each module in the multi-modal phosphorylation site prediction device described above can be realized by software, hardware, and combinations thereof, in whole or in part. The above-mentioned modules can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0227] The multi-modal phosphorylation site prediction device provided in the embodiment fuses the multi-modal features of the protein sequence language model and the three-dimensional structure graph convolution network, combines the dual information of protein sequence and structure for prediction, improves the accuracy and comprehensiveness of phosphorylation site prediction, and takes into account the computational efficiency and bioinformatics interpretability, solving the problems of low prediction efficiency and insufficient accuracy caused by traditional prediction methods relying only on sequence information or structure information.

[0228] In addition, the embodiment of the present application also provides an electronic device, as shown in Figure 6 The electronic device structure involved in the embodiment of the present application is shown in the figure, and specifically:

[0229] The electronic device can include a processor 301 with one or more processing cores, a memory 302 with one or more computer readable storage media, a power supply 303, and an input unit 304, etc. Those skilled in the art can understand that Figure 6 The electronic device structure shown in the figure does not constitute a limitation on the electronic device, and can include more or fewer components than the figure, or combine certain components, or different component arrangements. Among them:

[0230] The processor 301 is the control center of the electronic device, connects all parts of the electronic device through various interfaces and lines, executes various functions of the electronic device and processes data by running or executing software programs and / or modules stored in the memory 302 and calling data stored in the memory 302, thereby overall monitoring the electronic device. Optionally, the processor 301 can include one or more processing cores; preferably, the processor 301 can integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface and application program, etc., and the modem processor mainly processes wireless communication. It can be understood that the above-mentioned modem processor can also not be integrated into the processor 301.

[0231] The memory 302 can be used to store software programs and modules, and the processor 301 executes various functions and applications by running the software programs and modules stored in the memory 302, thereby executing the multi-modal phosphorylation site prediction method. The memory 302 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application program required by a function (such as a sound playing function, an image playing function, etc.), etc.; the data storage area can store data created according to the use of the electronic device, etc. In addition, the memory 302 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other volatile solid-state storage device. Accordingly, the memory 302 can also include a memory controller to provide access for the processor 301 to the memory 302.

[0232] The electronic device also includes a power supply 303 for powering various components, and preferably the power supply 303 can be logically connected to the processor 301 through a power management system, thereby realizing functions such as management of charging, discharging and power consumption management through the power management system. The power supply 303 can also include one or more direct current or alternating current power supplies, a recharging system, a power failure detection circuit, a power converter or inverter, a power state indicator, etc.

[0233] The electronic device can also include an input unit 304, which can be used to receive input digital or character information, and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0234] Although not shown, the electronic device can also include a display unit, etc., which will not be described here. Specifically, in the present embodiment, the processor 301 in the electronic device will load the executable file corresponding to the process of one or more application programs into the memory 302 according to the following instructions, and run the application program stored in the memory 302 by the processor 301, thereby realizing various functions, as follows:

[0235] extracting a subsequence of a candidate phosphorylation site from a protein sequence to be predicted, and generating sequence features of the subsequence through a deep learning model; constructing a graph structure based on three-dimensional structure information of the candidate phosphorylation site, and extracting structure features of the candidate site according to the graph structure through a graph neural network; performing multi-modal feature fusion on the sequence features and the structure features to generate fused features; and predicting a phosphorylation probability of the candidate site based on the fused features to obtain a phosphorylation probability prediction result.

[0236] The specific implementation of each operation above can be referred to the foregoing embodiments, which will not be described here again.

[0237] The embodiments of the present application fuse the multi-modal features of the protein sequence language model and the three-dimensional structure graph convolutional network, combine the dual information of protein sequence and structure for prediction, improve the accuracy and comprehensiveness of the phosphorylation site prediction, and meanwhile, take into account the calculation efficiency and bioinformatics interpretability, and solve the problems of low prediction efficiency and insufficient accuracy caused by the traditional prediction method which only relies on sequence information or structure information.

[0238] Those skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructions, or controlled by instructions related to hardware, which can be stored in a computer readable storage medium and loaded and executed by a processor.

[0239] To this end, the embodiments of the present application provide a storage medium, which stores a plurality of instructions. The instructions can be loaded by a processor to execute the steps in any one of the multi-modal phosphorylation site prediction methods provided by the embodiments of the present application. For example, the instructions can execute the following steps:

[0240] extracting a subsequence of a candidate phosphorylation site from a protein sequence to be predicted, and generating sequence features of the subsequence through a deep learning model; constructing a graph structure based on three-dimensional structure information of the candidate phosphorylation site, and extracting structure features of the candidate site according to the graph structure through a graph neural network; performing multi-modal feature fusion on the sequence features and the structure features to generate fused features; and predicting a phosphorylation probability of the candidate site based on the fused features to obtain a phosphorylation probability prediction result.

[0241] The specific implementation of each operation above can be referred to the foregoing embodiments, which will not be described here again.

[0242] The storage medium can include a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0243] Due to the instructions stored in the storage medium, the steps in any one of the multi-modal phosphorylation site prediction methods provided in the embodiments of the present application can be performed, and thus the beneficial effects that can be achieved by any one of the multi-modal phosphorylation site prediction methods provided in the embodiments of the present application can be achieved. Details are shown in the foregoing embodiments, and will not be described here.

[0244] The foregoing has described in detail a multi-modal phosphorylation site prediction method, device, equipment and medium provided in the embodiments of the present application. The principles and implementation manners of the present application are described by applying specific examples in this paper. The foregoing description of the embodiments is only used to help understand the method of the present application and its core idea. Meanwhile, for those skilled in the art, the specific implementation manners and application ranges will be changed according to the idea of the present application. In summary, the content of the specification should not be understood as a limitation of the present application.

Claims

1. A multimodal phosphorylation site prediction method, characterized in that: The steps include: Extracting subsequences of candidate phosphorylation sites from the protein sequence to be predicted, and generating sequence features of the subsequences through a deep learning model; Constructing a graph structure based on the three-dimensional structural information of the candidate phosphorylation site, and extracting structural features of the candidate site according to the graph structure through a graph neural network; Performing multimodal feature fusion on the sequence feature and the structural feature to generate a fusion feature; The phosphorylation probability of the candidate site is predicted based on the fusion feature to obtain a phosphorylation probability prediction result.

2. The multimodal phosphorylation site prediction method according to claim 1, wherein The step of extracting a subsequence of a candidate phosphorylation site from the protein sequence to be predicted and generating sequence features of the subsequence using a deep learning model includes: Obtaining the protein sequence to be predicted; Taking the candidate phosphorylation site as the center, the protein sequence to be predicted is intercepted according to a preset window length to generate a subsequence of the candidate phosphorylation site; Performing length normalization on the subsequence of the candidate phosphorylation site, including completing or truncating to a preset sequence length; The length-normalized subsequence is input into a pre-trained protein language model, and the model parameters are adjusted using low-rank adaptive technology to output the sequence embedding representation corresponding to the subsequence as the sequence feature.

3. The multimodal phosphorylation site prediction method according to claim 1, wherein The step of constructing a graph structure based on the three-dimensional structural information of the candidate phosphorylation sites, and extracting structural features of the candidate sites according to the graph structure through a graph neural network, includes: Obtaining three-dimensional structural information of the candidate phosphorylation site; Calculating the spatial distances between adjacent amino acid residues based on the three-dimensional structural information of the candidate phosphorylation sites to generate an adjacency matrix; Taking the candidate phosphorylation site as the central node, extracting a local subgraph containing multi-hop neighboring nodes through breadth-first search; The local subgraph is input into the graph attention network, the node features are aggregated through the multi-layer self-attention mechanism of the graph attention network, and skip connections are introduced to retain shallow network information, so as to generate the structural embedding representation corresponding to the local subgraph as the structural feature of the candidate site.

4. The multimodal phosphorylation site prediction method according to claim 1, wherein The performing multimodal feature fusion on the sequence feature and the structural feature to generate a fusion feature includes: Mapping the sequence features and structural features to feature spaces of the same dimension through linear projection layers respectively; The fused features are generated by performing cross-attention interaction with the projected sequence features and structural features.

5. The multimodal phosphorylation site prediction method according to claim 4, characterized in that: After generating the fusion features, the method further includes: Performing multi-head self-attention calculation on the fused features; Perform residual connection and layer normalization on the fusion features calculated by multi-head self-attention to generate optimized fusion features; The optimized fusion features are input into a feedforward neural network for nonlinear transformation to obtain fusion features with enhanced feature expression.

6. The multimodal phosphorylation site prediction method according to claim 1, characterized in that: The step of predicting the phosphorylation probability of the candidate site based on the fusion feature to obtain a phosphorylation probability prediction result includes: Performing weighted pooling based on attention weights on the fused features to generate a global feature vector of fixed length; The global feature vector is input into the fully connected layer, and the phosphorylation probability prediction value is output through the Sigmoid activation function to generate a phosphorylation probability prediction result.

7. The multimodal phosphorylation site prediction method according to claim 1, wherein: The method further comprises: Based on the phosphorylation probability prediction results, candidate sites with phosphorylation probability prediction values ​​higher than a preset threshold are marked as high-confidence phosphorylation sites, and a corresponding visual distribution map is generated; The biological pathways or disease mechanisms associated with the high-confidence phosphorylation sites are marked using the protein function annotation database.

8. A multimodal phosphorylation site prediction device, characterized in that: include: A sequence feature module is used to extract subsequences of candidate phosphorylation sites from the protein sequence to be predicted and generate sequence features of the subsequences through a deep learning model; A structural feature module is used to construct a graph structure based on the three-dimensional structural information of the candidate phosphorylation site, and extract the structural features of the candidate site according to the graph structure through a graph neural network; A feature fusion module, configured to perform multimodal feature fusion on the sequence feature and the structural feature to generate a fusion feature; A prediction module is used to predict the phosphorylation probability of the candidate site based on the fusion feature to obtain a phosphorylation probability prediction result.

9. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the multimodal phosphorylation site prediction method according to any one of claims 1 to 7 are implemented.

10. A storage medium, characterized in that: A computer program is stored which can be loaded by a processor and executes the multimodal phosphorylation site prediction method according to any one of claims 1 to 7.

Images