Protein subcellular localization method and system based on gated multi-mode routing

Abstract

The invention discloses a protein subcellular localization method and system based on gating multi-mode routing, and relates to the technical field of protein subcellular localization, and the method comprises the steps: obtaining a protein image and a protein sequence; the method comprises the following steps: respectively adopting a ConvNeXt-V2 network and an ESM-2 model to carry out feature extraction on a protein image and a protein sequence to obtain image features and sequence features, and then based on a Sparsemax gating network, obtaining multi-modal fusion features; according to the STRING database and the subcellular compartment distance matrix, a weighted adjacency matrix is constructed, a protein map is obtained, then in combination with multi-modal fusion features, through a positioning perception neural network, aggregated neighbor features are obtained, and in combination with geometric prototype decoding and hierarchical constraint loss, multi-label prediction is carried out, and an optimized prediction result is obtained. According to the method, the capability of capturing complex protein features is improved, the problem of insufficient single-modal information is solved, and the accuracy of positioning prediction is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of protein subcellular localization, and particularly relates to a protein subcellular localization method and system based on gated multi-modal routing. BACKGROUND

[0002] Protein subcellular localization is crucial for understanding protein function, analyzing cell biology mechanisms, and disease diagnosis and drug development. Accurate determination of the specific location of proteins in cells can provide key clues to reveal the biological processes in which proteins are involved. Traditional protein subcellular localization methods mainly rely on experimental techniques such as immunofluorescence microscopy, proteomics analysis, etc. However, these methods have limitations such as high cost, long time consumption, and low throughput, which are difficult to meet the demand for large-scale protein analysis in the post-genomic era.

[0003] With the rapid development of deep learning technology, computational protein subcellular localization methods have gradually become a research hotspot. Existing computational methods mostly use protein sequence information or single modal data for prediction. However, due to the influence of multiple factors such as sequence, structure, and interaction with other proteins, single modal information is difficult to fully capture the complex characteristics, resulting in limited prediction accuracy. Although some studies attempt to integrate multi-modal data, they often use fixed feature fusion strategies, which cannot adaptively select the optimal fusion method for different sample characteristics, reducing the generalization ability of the model.

[0004] In addition, proteins do not exist in isolation in cells, and their interaction relationships and spatial structures of subcellular compartments have important influence on localization. However, existing methods rarely consider protein-protein interaction networks and subcellular structure information in the model. Therefore, how to effectively integrate multi-modal data, dynamically select feature fusion strategies, and combine protein-protein interaction relationships to achieve high-precision subcellular localization is a key problem to be solved in the current protein subcellular localization research field. SUMMARY

[0005] The purpose of the present application is to provide a protein subcellular localization method and system based on gated multi-modal routing, which improves the ability to capture the complex characteristics of proteins by using a multi-modal dynamic fusion strategy, constructing a protein graph, and introducing a hierarchical constraint loss, solves the problem of insufficient single modal information, and further improves the accuracy of localization prediction.

[0006] To achieve the above purpose, the present application provides a protein subcellular localization method based on gated multi-modal routing, comprising the following steps:

[0007] S1, obtaining a protein image and a protein sequence;

[0008] S2, using a ConvNeXt-V2 network to extract features of the protein image to obtain image features; using an ESM-2 model to extract features of the protein sequence to obtain sequence features;

[0009] S3, based on a Sparsemax gating network, performing multi-modal feature fusion on the image features and the sequence features to obtain multi-modal fusion features; the Sparsemax gating network comprises a bidirectional cross-modal self-attention module, a hierarchical double-flow collaborative module, and a channel gating modulation module;

[0010] S4, constructing a weighted adjacency matrix according to a STRING database and a subcellular compartment distance matrix to obtain a protein graph;

[0011] S5, according to the multi-modal fusion features and the protein graph, performing feature propagation through a positioning perception neural network to obtain aggregated neighbor features; the positioning perception neural network comprises a topological convolution layer, a compartment attention layer, and a path aggregation layer;

[0012] S6, according to the aggregated neighbor features, combining geometric prototype decoding and hierarchical constraint loss to perform multi-label prediction to obtain an optimized prediction result.

[0013] Preferably, in S2, the ConvNeXt-V2 network is used to extract features of the protein image to obtain image features, specifically comprising:

[0014] Based on the ConvNeXt-V2 network, the protein image is sequentially subjected to convolution operation, normalization processing, and activation operation to extract features of the protein image to obtain initial image features; the initial image features comprise fourth layer image features and eighth layer image features; the fourth layer image features have abstract semantic information; the eighth layer image features have detailed information;

[0015] After the eighth layer image features are up-sampled through deconvolution operation, the eighth layer image features are normalized with the fourth layer image features to obtain the image features.

[0016] Preferably, in S2, the ESM-2 model is used to extract features of the protein sequence to obtain sequence features, specifically comprising:

[0017] The protein sequence is input into the ESM-2 model, and information is encoded through a Transformer structure to obtain a sequence feature matrix;

[0018] A first weight matrix is obtained; the first weight matrix is a weight matrix learnable by the Transformer structure;

[0019] The learnable weight matrix is multiplied with the sequence feature matrix, activated by a nonlinear activation function, and then multiplied element by element with the sequence feature matrix to obtain the sequence feature; the nonlinear activation function is a Softmax function, a GELU function, and a Sigmoid function.

[0020] Preferably, in S3, the image features and the sequence features are subjected to multi-modal feature fusion based on the Sparsemax gating network to obtain multi-modal fusion features, specifically including:

[0021] The image features are subjected to global average pooling based on the Sparsemax gating network to obtain global features of the image features;

[0022] The sequence features are subjected to global maximum pooling based on the Sparsemax gating network to obtain key feature information of the sequence features;

[0023] The global features and the key feature information are spliced, then sequentially subjected to second weight matrix transformation and nonlinear activation function activation, and then subjected to third weight matrix transformation, and finally subjected to Sparsemax function calculation to obtain a gating weight vector;

[0024] When the gating weight vector corresponds to the activation weight of the bidirectional cross-modal self-attention module, a bidirectional cross-modal self-attention method is used to fuse the image features and the sequence features to obtain bidirectional cross-modal fusion features;

[0025] When the gating weight vector corresponds to the activation weight of the hierarchical dual-flow collaborative module, a hierarchical dual-flow collaborative method is used to fuse the image features and the sequence features to obtain dual-flow collaborative fusion features;

[0026] When the gating weight vector corresponds to the activation weight of the channel gating modulation module, a channel gating modulation method is used to fuse the image features and the sequence features to obtain channel gating modulation fusion features;

[0027] According to the activation weight of the bidirectional cross-modal self-attention module, the activation weight of the hierarchical dual-flow collaborative module, and the activation weight of the channel gating modulation module corresponding to the gating weight vector respectively, the bidirectional cross-modal fusion features, the dual-flow collaborative fusion features, and the channel gating modulation fusion features are weighted and summed to obtain multi-modal fusion output features.

[0028] Preferably, if the gating weight vector corresponds to the activation weight of the bidirectional cross-modal self-attention module, a bidirectional cross-modal self-attention method is used to fuse the image features and the sequence features to obtain multi-modal fusion features, specifically including:

[0029] obtaining a first projection matrix, a second projection matrix and a third projection matrix; the first projection matrix, the second projection matrix and the third projection matrix are all learnable projection matrices;

[0030] transforming the image feature into a query matrix according to the first projection matrix; transforming the sequence feature into a key matrix and a value matrix according to the second projection matrix and the third projection matrix respectively;

[0031] calculating an image-to-sequence attention score according to the query matrix and the key matrix, normalizing the image-to-sequence attention score by using a Softmax function, and then multiplying the image-to-sequence attention score by the value matrix to obtain a cross-modal attention output from the image to the sequence;

[0032] calculating a sequence-to-image attention score according to the key matrix and the query matrix, normalizing the sequence-to-image attention score by using a nonlinear activation function, and then multiplying the sequence-to-image attention score by the image feature to obtain a cross-modal attention output from the sequence to the image;

[0033] performing adaptive pooling processing on the cross-modal attention output from the image to the sequence and the cross-modal attention output from the sequence to the image respectively, then adding them, and then performing layer normalization processing on the addition result to obtain a bidirectional cross-modal fusion feature.

[0034] Preferably, if the gating weight vector corresponds to the activation weight of the channel gating modulation module, a channel gating modulation method is used to perform multi-modal feature fusion on the image feature and the sequence feature to obtain a multi-modal fusion feature, which specifically includes:

[0035] performing a global max-pooling operation on the sequence feature, then sequentially performing multilayer perceptron transformation and nonlinear activation function activation to obtain a channel gating vector;

[0036] element-wise multiplying the channel gating vector and the image feature to obtain a channel gating modulation fusion feature.

[0037] Preferably, in S4, a weighted adjacency matrix is constructed according to the STRING database and the subcellular compartment distance matrix to obtain a protein graph, which specifically includes:

[0038] obtaining a protein-protein interaction score from the STRING database;

[0039] defining a subcellular compartment distance matrix;

[0040] calculating an edge weight according to the protein-protein interaction score and the subcellular compartment distance matrix, constructing a weighted adjacency matrix, and obtaining a protein graph.

[0041] Preferably, in S5, an aggregated neighbor feature is obtained by performing feature propagation through a positional perception neural network according to the multi-modal fusion feature and the protein graph, which specifically includes:

[0042] Based on the protein graph, the multi-modal fusion features are propagated through a topological convolution layer to obtain next layer multi-modal fusion features;

[0043] The attention weights of the multi-modal node pair to the neighbor nodes are calculated through the compartment attention mechanism;

[0044] The aggregated neighbor features are obtained by combining the next layer multi-modal fusion features and the attention weights of the multi-modal node pair to the neighbor nodes through the path aggregation module.

[0045] Preferably, according to the aggregated neighbor features, the multi-label prediction is performed by combining the geometric prototype decoding and the hierarchical constraint loss to obtain the optimized prediction result, specifically including:

[0046] According to the aggregated neighbor features, the prediction probability of the protein belonging to the subcellular compartment is calculated by using the geometric prototype decoding, and the true probability is obtained;

[0047] According to the prediction probability and the true probability, the binary cross-entropy loss is calculated;

[0048] According to the parent-child pairs in the subcellular compartment hierarchy, the difference between the prediction probability of the child compartment and the prediction probability of the parent compartment is calculated to obtain the hierarchical constraint loss;

[0049] According to the binary cross-entropy loss and the hierarchical constraint loss, a loss function is constructed, and multi-label prediction is performed to obtain the optimized prediction result.

[0050] The application also provides a protein subcellular localization system based on a gated multi-modal routing, comprising:

[0051] An acquisition module is configured to acquire protein images and protein sequences;

[0052] An extraction module is configured to extract features of the protein images by using a ConvNeXt-V2 network to obtain image features, and extract features of the protein sequences by using an ESM-2 model to obtain sequence features;

[0053] A feature fusion module is configured to fuse multi-modal features of the image features and the sequence features based on a Sparsemax gating network to obtain multi-modal fusion features; the Sparsemax gating network comprises a bidirectional cross-modal self-attention module, a hierarchical double-flow collaborative module and a channel gating modulation module;

[0054] A graph construction module is configured to construct a weighted adjacency matrix based on a STRING database and a subcellular compartment distance matrix to obtain a protein graph;

[0055] The aggregation module is used for obtaining aggregated neighbor features by performing feature propagation through a location-aware neural network according to the multi-modal fusion features and the protein graph; the location-aware neural network comprises a topological convolution layer, a compartment attention layer and a path aggregation layer;

[0056] The prediction module is used for performing multi-label prediction by combining geometric prototype decoding and hierarchical constraint loss according to the aggregated neighbor features, so as to obtain an optimized prediction result.

[0057] In summary, compared with the prior art, the protein subcellular localization method and system based on the gated multi-modal routing have the following beneficial effects:

[0058] (1) The multi-modal fusion strategy of the multi-modal fusion features is obtained based on the Sparsemax gated network, the feature representation capability is improved, the multi-modal adaptive fusion strategy can dynamically match the optimal feature fusion mode according to different sample characteristics, the capture capability of the complex features of the protein is significantly improved, the problem of insufficient single-modal information is effectively solved, and the accuracy of the protein subcellular localization prediction is improved.

[0059] (2) The protein graph is constructed, the protein interaction relationship and the subcellular spatial structure information are fully mined, multi-label prediction is performed by combining geometric prototype decoding and hierarchical constraint loss, the method has biological rationality, and the accuracy of the protein subcellular localization prediction is further improved.

[0060] The technical method of the present application will be further described in detail below by means of the accompanying drawings and examples. BRIEF DESCRIPTION OF DRAWINGS

[0061] Figure 1 The flowchart of the protein subcellular localization method based on the gated multi-modal routing of the present application;

[0062] Figure 2 The principle structure diagram of the protein subcellular localization method based on the gated multi-modal routing of the present application;

[0063] Figure 3 The principle structure diagram of the protein subcellular localization method based on the gated multi-modal routing of the present application;

[0064] Figure 4 The module diagram of the protein subcellular localization system based on the gated multi-modal routing of the present application. DETAILED DESCRIPTION

[0065] The technical method of the present application is further illustrated by the accompanying drawings and examples. It should be noted that the relative arrangement, numerical expression and numerical value of the components and steps set forth in these examples do not limit the scope of the present application unless otherwise specified.

[0066] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the application or its application or uses.

[0067] Techniques, systems, and devices known to those of ordinary skill in the relevant art can not be discussed in detail, but should be considered part of the specification where appropriate.

[0068] In all examples shown and discussed herein, any specific value should be interpreted as merely illustrative and not as a limitation. Thus, other examples of the exemplary embodiments can have different values.

[0069] Unless otherwise defined, technical terms or scientific terms used in the present application should be understood as having their usual meaning to those skilled in the art.

[0070] As shown in Figure 1 , the present application provides a protein subcellular localization method based on gated multi-modal routing, and the principle structure diagram of the method is shown in Figure 2 . Figure 1 and Figure 2 , a protein subcellular localization method based on gated multi-modal routing, specifically comprising:

[0071] Step S1, obtaining a protein image and a protein sequence.

[0072] Step S2, using a ConvNeXt-V2 network to extract features of the protein image to obtain image features, and using an ESM-2 model to extract features of the protein sequence to obtain sequence features.

[0073] The specific content of step S2 includes:

[0074] Based on the ConvNeXt-V2 network, the protein image is sequentially subjected to convolution operation, normalization processing and activation operation to extract features of the protein image to obtain initial image features. The initial image features include the 4th layer image features and the 8th layer image features. The 4th layer image features have abstract semantic information, and the 8th layer image features have detailed information.

[0075] Further, the expression of the initial image features is:

[0076]

[0077] wherein, V is an initial image feature, ConvNeXt-V2(·) is an image feature extraction network, I is a protein image, is a real set, C is a feature channel number, which is related to the number of convolution kernels in the network; H is the height of the picture, and W is the width of the picture.

[0078] After the 8th layer image feature is up-sampled through the deconvolution operation, the size is consistent with the 4th layer image feature, and then the layer normalization processing is performed with the 4th layer image feature, the image feature is obtained. In order to fuse the features of different levels and enhance the expression of image structure information, the feature maps of the 4th layer image feature and the 8th layer image feature in the ConvNeXt-V2 network are processed.

[0079] Further, the expression of the image feature is:

[0080]

[0081] wherein, V fuse is an image feature, LayerNorm(·) is a layer normalization processing, is the 4th layer image feature, is the 8th layer image feature, and Deconv(·) is a deconvolution operation.

[0082] The protein sequence is input into the ESM-2 model, information coding is performed through the Transformer structure, and a sequence feature matrix is obtained, the dimension of which is

[0083] Further, the expression of the sequence feature matrix is:

[0084]

[0085] wherein, S is a sequence feature matrix, ESM-2(·) is a sequence feature coding operation, Seq is a protein sequence, and L is a protein sequence length.

[0086] A first weight matrix is obtained. The first weight matrix is a weight matrix that can be learned by the Transformer structure.

[0087] After the learnable weight matrix is multiplied with the sequence feature matrix, the Softmax function is activated, and then the sequence feature of the focused key functional area is obtained by element-by-element multiplication with the sequence feature matrix. In order to focus on the key functional area in the sequence, the key functional area focusing operation is introduced.

[0088] Further, the expression of the sequence feature is:

[0089] S key=Softmax(W1S)⊙S;

[0090] Among them, S key is the sequence feature, Softmax(■) is the Softmax function, W1 is the first weight matrix, and ⊙ is the element-by-element multiplication.

[0091] Step S3: Based on the Sparsemax gating network, multimodal feature fusion is performed on the image features and sequence features to obtain multimodal fusion features. The Sparsemax gating network includes: a bidirectional cross-modal self-attention module, a hierarchical two-stream collaboration module, and a channel gating modulation module.

[0092] The specific contents of step S3 include:

[0093] Based on the Sparsemax gating network, the image features are globally averaged pooled, the spatial dimensions are compressed into vectors, the global features of the image are extracted, and the global features of the image features are obtained.

[0094] Based on the Sparsemax gating network, the sequence features are globally max-pooled to obtain the key feature information in the sequence and obtain the key feature information of the sequence features.

[0095] The global features and key feature information are spliced, then transformed by the second weight matrix and activated by the GELU function, and then transformed by the third weight matrix. Finally, the gating weight vector is calculated by the Sparsemax function, whose dimension is 3, corresponding to the selection probability of the bidirectional cross-modal self-attention module, the hierarchical two-stream collaborative module and the channel gating modulation module.

[0096] Furthermore, the expression of the gate weight vector is:

[0097] g=Sparsemax(W3GELU(W2[GAP(V fuse )||max(S)])

[0098] Among them, g is the gated weight vector, Sparsemax(■) is the Sparsemax function, W3 is the third weight matrix, GELU(■) is the GELU function, GAP(■) is the global average pooling operation, and max(■) is the global maximum pooling operation.

[0099] When the gate weight vector corresponds to the activation weight of the bidirectional cross-modal self-attention module, the bidirectional cross-modal self-attention method is used to perform multimodal feature fusion on the image features and sequence features to obtain bidirectional cross-modal fusion features. Further, the specific contents include:

[0100] The first projection matrix, the second projection matrix and the third projection matrix are all learnable projection matrices.

[0101] The image feature is converted into a query matrix according to the first projection matrix, and the sequence feature is converted into a key matrix and a value matrix according to the second projection matrix and the third projection matrix respectively. Further, the expression of the query matrix is:

[0102] Q v =V fuse W q ;

[0103] Wherein, Q V is the query matrix, and Wq is the first projection matrix.

[0104] The expression of the key matrix is:

[0105] K s =SW k ;

[0106] Wherein, K s is the key matrix, and W k is the second projection matrix.

[0107] The expression of the value matrix is:

[0108] V s =SW v ;

[0109] Wherein, V s is the key matrix, and W V is the third projection matrix.

[0110] Wherein, the attention score from the image to the sequence is calculated according to the query matrix and the key matrix, the attention score from the image to the sequence is normalized by using a Softmax function, and then the value matrix is multiplied to obtain the cross-modal attention output from the image to the sequence.

[0111] Further, the expression of the cross-modal attention output from the image to the sequence is:

[0112]

[0113] Wherein, h v2s is the cross-modal attention output from the image to the sequence, K s T is the transpose matrix of the key matrix, and d k is the key vector dimension.

[0114] According to the key matrix and the query matrix, the sequence-to-image attention score is calculated, the sequence-to-image attention score is normalized by using a Softmax function, and then multiplied by the image feature to obtain a sequence-to-image cross-modal attention output.

[0115] Further, an expression of the sequence-to-image cross-modal attention output is:

[0116]

[0117] wherein h s2v is the sequence-to-image cross-modal attention output, Q V T is a transpose matrix of the query matrix.

[0118] The image-to-sequence cross-modal attention output and the sequence-to-image cross-modal attention output are respectively subjected to adaptive pooling processing, and then added after adjusting the dimensions, and then subjected to layer normalization processing to obtain a bidirectional cross-modal fusion feature.

[0119] Further, an expression of the bidirectional cross-modal fusion feature is:

[0120] h BCMA =LayerNorm(h v2s +AdaPool(h s2v ));

[0121] wherein h BCMA is the bidirectional cross-modal fusion feature, and AdaPool(■) is adaptive pooling processing.

[0122] When the gating weight vector corresponds to the activation weight of the hierarchical dual-flow collaborative module, a hierarchical dual-flow collaborative method is used to perform multi-modal feature fusion on the image feature and the sequence feature to obtain a dual-flow collaborative fusion feature. The hierarchical dual-flow collaborative method divides the image feature and the sequence feature according to levels, for example, into shallow features and deep features, and realizes collaborative fusion of multi-scale information by designing a specific interaction mechanism, such as multi-level feature alignment and fusion operation, so as to obtain the dual-flow collaborative fusion feature.

[0123] When the gating weight vector corresponds to the activation weight of the channel gating modulation module, a channel gating modulation method is used to perform multi-modal feature fusion on the image feature and the sequence feature to obtain a channel gating modulation fusion feature. Further, the specific content includes:

[0124] The sequence feature is subjected to global maximum pooling operation, and then sequentially subjected to multi-layer perceptron transformation and nonlinear activation function activation to obtain a channel gating vector. The dimension of the channel gating vector is consistent with the channel number of the image feature. Further, an expression of the channel gating vector is:

[0125] g c =σ(MLP(GMP(S)));

[0126] Among them, g c is the channel gating vector, σ(■) is the Sigmoid activation function, and MLP(■) is the multi-layer perceptron transformation.

[0127] The channel gating vector is multiplied element-by-element with the image feature to achieve weighted modulation of the image feature channel and obtain the channel gating modulation fusion feature. Furthermore, the expression of the channel gating modulation fusion feature is:

[0128]

[0129] Among them, V mod It is the channel gating modulation fusion feature.

[0130] According to the activation weights of the bidirectional cross-modal self-attention module, the activation weights of the hierarchical two-stream collaborative module, and the activation weights of the channel gated modulation module corresponding to the gating weight vectors, the bidirectional cross-modal fusion features, the two-stream collaborative fusion features, and the channel gated modulation fusion features are weighted and summed to obtain the multimodal fusion output features. Furthermore, the expression of the multimodal fusion output features is:

[0131]

[0132] Among them, h fused is the multimodal fusion output feature, g k is the weight of bidirectional cross-modal fusion feature, dual-stream collaborative fusion feature and channel gated modulation fusion feature, k = 1, 2, 3, h expertk It is a comprehensive representation of bidirectional cross-modal fusion features, dual-stream collaborative fusion features and channel-gated modulation fusion features.

[0133] Step S4: construct a weighted adjacency matrix based on the STRING database and the subcellular compartment distance matrix to obtain a protein map.

[0134] The specific contents of step S4 include:

[0135] Protein interaction scores were obtained from the STRING database.

[0136] According to the symmetric matrix defined by prior knowledge, the subcellular compartment distance matrix is ​​defined, where the symmetric matrix is ​​M∈R K×K , where M is a symmetric matrix and K is the number of rows and columns of the symmetric matrix, which is used to describe the distance relationship between different subcellular compartments.

[0137] According to the protein interaction scores and the subcellular compartment distance matrix, the edge weights are calculated and the weighted adjacency matrix is ​​constructed to obtain the protein graph. Furthermore, the expression of the edge weight is:

[0138]

[0139] Among them, ω ij is the edge weight, score STRING is the protein interaction score, d sc (i, j) is the distance relationship between different subcellular compartments, C i is the ith subcellular compartment, C j is the jth subcellular compartment, From this formula, we can see that the edge weight is positively correlated with the protein interaction score and negatively correlated with the compartment distance, and only when the subcellular compartment C where the two proteins are located i and C j The edge weight is calculated only when there is an intersection, otherwise the weight is 0.

[0140] Furthermore, the expression of the weighted adjacency matrix is:

[0141]

[0142] Among them, A ij is the weighted adjacency matrix, threshold is the set threshold, and otherwse is other conditions except the first condition.

[0143] When protein i and protein j are different and the edge weight ω ij When it is greater than the set threshold, the weighted adjacency matrix corresponding to position A ij ω ij , indicating that there is a connection between the two proteins; otherwise A ij A value of 0 indicates the absence of connectivity, resulting in a complete protein interaction graph.

[0144] The principle structure diagram of steps 5 and 6 is as follows: Figure 3 As shown, the details are as follows:

[0145] Step S5: Based on the multimodal fusion features and the protein graph, feature propagation is performed through a location-aware neural network to obtain aggregated neighbor features. The location-aware neural network includes: a topological convolution layer, a compartmental attention layer, and a path aggregation layer.

[0146] The specific contents of step S5 include:

[0147] Based on the protein graph, the multi-modal fusion features are propagated through the topological convolution layer to obtain the updated next layer node feature matrix, i.e. the next layer multi-modal fusion features. Further, the expression of the next layer multi-modal fusion features is:

[0148] H (l+1) = σ(D -1 / 2 A ij D -1 / 2 H (l) Θ);

[0149] wherein H (l+1) is the next layer multi-modal fusion features, D is the degree matrix, H (l) is the current node feature matrix, and Θ is the weight matrix.

[0150] For the current node feature matrix H (l) , firstly, the weighted adjacency matrix A ij is symmetrically normalized by the degree matrix D, and then the normalized weighted adjacency matrix is multiplied by the node feature matrix and the learnable weight matrix Θ, and then the activation function σ is used to obtain the updated next layer node feature matrix H (l+1) .

[0151] The attention weight of the multi-modal node to the neighbor node is calculated by the compartment attention mechanism. The specific process is that for the node m and its neighbor node n, the feature vectors h m and h n are spliced, and then transformed by the learnable parameter vector a, and then the LeakyReLU function is used, and then the Softmax function is used to normalize all neighbor nodes of the node m to obtain the attention weight α mn of the node m to the neighbor node n. Further, the expression of the attention weight of the multi-modal node to the neighbor node is:

[0152]

[0153] wherein α mn is the attention weight of the multi-modal node to the neighbor node, exp(■) is the exponential function, LeakyRELU(■) is the LeakyReLU function, a T is the transpose vector of the learnable parameter vector, h m is the mth feature vector, h n is the nth feature vector, N sc (m) is the neighbor set of the node m in the subcellular compartment, and h k is the kth feature vector.

[0154] The path aggregation module is used for combining the next layer multi-modal fusion features and the attention weight of the neighbor node of the multi-modal node to aggregate the features of the neighbor node, so as to obtain the aggregated neighbor features. The path aggregation module is used for comprehensively considering the multiple path information of the protein. The module can aggregate the features on different paths based on the attention mechanism, fuse the node feature with the aggregated features from the neighbor node, further enhance the expression ability of the node feature, and obtain more comprehensive protein feature information.

[0155] In step S6, multi-label prediction is performed according to the aggregated neighbor features, combined with geometric prototype decoding and hierarchical constraint loss, to obtain an optimized prediction result.

[0156] The specific content of step S6 includes:

[0157] According to the aggregated neighbor features, the geometric prototype decoding is used to calculate the prediction probability of the protein belonging to the subcellular compartment, and the real probability is obtained. Specifically, for the feature vector h of the protein and the prototype vector e of the subcellular compartment c c , the cosine similarity cos(h,e c ) of the two is calculated, then multiplied by the L2 norm of the feature vector and the learnable scaling factor γ, and finally mapped to a probability value p between 0 and 1 through the Sigmoid function, representing the prediction probability of the protein belonging to the subcellular compartment c. Further, the calculation formula of the prediction probability of the protein belonging to the subcellular compartment is:

[0158] p(y c =1|h)=σ(γ·cos(h,e c )·||h||2);

[0159] Wherein, p(y c =1|h) is the prediction probability of the protein feature h belonging to the compartment c, y c is the real label of whether the protein belongs to the compartment c, h is the feature vector of the protein, γ is the learnable scaling factor, and cos(h,e c ) is the cosine similarity.

[0160] According to the prediction probability and the real probability, the binary cross-entropy loss is calculated.

[0161] According to the parent-child pairs in the hierarchical structure of the subcellular compartment, the difference between the prediction probability of the child compartment and the prediction probability of the parent compartment is calculated to obtain the hierarchical constraint loss. Further, the expression of the hierarchical constraint loss is:

[0162]

[0163] Wherein, L hierFor the hierarchical constraint loss, (p, c) is a parent-child pair in the subcellular compartment hierarchy, p is the parent compartment, c is the child compartment, max(■) is the maximum function, P c is the predicted probability of the child compartment c. p is the predicted probability of the parent compartment p.

[0164] According to the binary cross-entropy loss and the hierarchical constraint loss, a loss function is constructed, and multi-label prediction is performed to obtain an optimized prediction result. For each subcellular compartment b, the binary cross-entropy loss BCE between the predicted probability and the true label y b is calculated, and the binary cross-entropy losses of all compartments are added, and then the hierarchical constraint loss L hier is added multiplied by a weight parameter λ to obtain the final loss function L. Further, the optimized prediction result expression is:

[0165]

[0166] where L is the final loss function, b is the bth subcellular compartment, BCE(■) is the binary cross-entropy loss function, λ is the weight parameter, μ is the L2 regularization coefficient, y b is the true probability, i.e., the true label, is the predicted probability.

[0167] A protein subcellular localization system based on a gated multi-modal routing, as shown in Figure 4 , comprises:

[0168] An acquisition module is configured to acquire a protein image and a protein sequence.

[0169] An extraction module is configured to extract features from the protein image using a ConvNeXt-V2 network to obtain image features, and extract features from the protein sequence using an ESM-2 model to obtain sequence features.

[0170] A feature fusion module is configured to perform multi-modal feature fusion on the image features and the sequence features based on a Sparsemax gating network to obtain multi-modal fusion features. The Sparsemax gating network comprises a bidirectional cross-modal self-attention module, a hierarchical dual-flow collaborative module, and a channel gating modulation module.

[0171] A graph construction module is configured to construct a weighted adjacency matrix based on a STRING database and a subcellular compartment distance matrix to obtain a protein graph.

[0172] An aggregation module is configured to perform feature propagation through a localization-aware neural network based on the multi-modal fusion features and the protein graph to obtain aggregated neighbor features. The localization-aware neural network comprises a topological convolution layer, a compartment attention layer, and a path aggregation layer.

[0173] a prediction module, configured to perform multi-label prediction according to the aggregated neighbor features, combined with geometric prototype decoding and hierarchical constraint loss, to obtain an optimized prediction result.

[0174] Finally, it should be noted that the above examples are only used to illustrate the technical method of the present application, but not to limit it. Although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical method of the present application can still be modified or equivalently replaced, and these modifications or equivalent replacements should not make the modified technical method deviate from the spirit and scope of the technical method of the present application.

Claims

1. A protein subcellular localization method based on gated multimodal routing, characterized in that: The following steps are involved: S1. Obtain protein images and protein sequences; S2, using ConvNeXt-V2 network to extract features from protein images and obtain image features; The ESM-2 model is used to extract features from protein sequences and obtain sequence features; S3, based on the Sparsemax gating network, multimodal feature fusion is performed on image features and sequence features to obtain multimodal fusion features; The Sparsemax gating network includes: a bidirectional cross-modal self-attention module, a hierarchical two-stream collaboration module and a channel gating modulation module; S4. Based on the STRING database and the subcellular compartment distance matrix, a weighted adjacency matrix was constructed to obtain a protein map; S5. Based on the multimodal fusion features and the protein graph, feature propagation is performed through a location-aware neural network to obtain aggregated neighbor features; the location-aware neural network includes: a topological convolution layer, a compartment attention layer, and a path aggregation layer; S6. Based on the aggregated neighbor features, combined with geometric prototype decoding and hierarchical constraint loss, multi-label prediction is performed to obtain optimized prediction results.

2. A protein subcellular localization method based on gated multimodal routing according to claim 1, characterized in that: S2 uses the ConvNeXt-V2 network to extract features from protein images and obtain image features, including: Based on the ConvNeXt-V2 network, the protein image is subjected to feature extraction through convolution, normalization, and activation operations to obtain initial image features. The initial image features include: 4th layer image features and 8th layer image features. The 4th layer image features contain abstract semantic information, and the 8th layer image features contain detailed information. After upsampling the 8th layer image features through deconvolution operation, they are normalized with the 4th layer image features to obtain image features.

3. A protein subcellular localization method based on gated multimodal routing according to claim 1, characterized in that: S2 uses the ESM-2 model to extract features from protein sequences and obtain sequence features, including: The protein sequence is input into the ESM-2 model, and the information is encoded through the Transformer structure to obtain the sequence feature matrix; Obtain a first weight matrix; the first weight matrix is ​​a learnable weight matrix of the Transformer structure; The learnable weight matrix is ​​multiplied by the sequence feature matrix, activated by a nonlinear activation function, and then multiplied element-by-element with the sequence feature matrix to obtain sequence features; the nonlinear activation function is a Softmax function, a GELU function, and a Sigmoid function.

4. A method for protein subcellular localization based on gated multimodal routing according to claim 1, characterized in that: In S3, based on the Sparsemax gating network, multimodal feature fusion is performed on image features and sequence features to obtain multimodal fusion features, including: Based on the Sparsemax gating network, the image features are globally averaged pooled to obtain the global features of the image features; Based on the Sparsemax gating network, the sequence features are globally pooled to obtain the key feature information of the sequence features; The global features and key feature information are spliced, then activated by the second weight matrix transformation and nonlinear activation function, and then transformed by the third weight matrix. Finally, the gated weight vector is calculated by the Sparsemax function. When the gate weight vector corresponds to the activation weight of the bidirectional cross-modal self-attention module, the bidirectional cross-modal self-attention method is used to perform multimodal feature fusion on the image features and sequence features to obtain bidirectional cross-modal fusion features; When the gating weight vector corresponds to the activation weight of the hierarchical two-stream collaborative module, the hierarchical two-stream collaborative method is used to perform multimodal feature fusion on the image features and sequence features to obtain the two-stream collaborative fusion features; When the gating weight vector corresponds to the activation weight of the channel gating modulation module, the channel gating modulation method is used to perform multimodal feature fusion on the image features and sequence features to obtain the channel gating modulation fusion feature; According to the activation weights of the bidirectional cross-modal self-attention module, the activation weights of the hierarchical two-stream collaborative module and the activation weights of the channel gating modulation module corresponding to the gating weight vectors, the bidirectional cross-modal fusion features, the two-stream collaborative fusion features and the channel gating modulation fusion features are weighted and summed to obtain the multimodal fusion output features.

5. A method for protein subcellular localization based on gated multimodal routing according to claim 4, characterized in that: If the gate weight vector corresponds to the activation weight of the bidirectional cross-modal self-attention module, the bidirectional cross-modal self-attention method is used to perform multimodal feature fusion on the image features and sequence features to obtain multimodal fusion features, specifically including: Obtain a first projection matrix, a second projection matrix, and a third projection matrix; the first projection matrix, the second projection matrix, and the third projection matrix are all learnable projection matrices; According to the first projection matrix, the image features are converted into a query matrix; according to the second projection matrix and the third projection matrix, the sequence features are converted into a key matrix and a value matrix respectively; According to the query matrix and the key matrix, the image-to-sequence attention score is calculated. The image-to-sequence attention score is normalized using the Softmax function and then multiplied by the value matrix to obtain the image-to-sequence cross-modal attention output; According to the key matrix and the query matrix, the sequence-to-image attention score is calculated. The sequence-to-image attention score is normalized using a nonlinear activation function and then multiplied by the image feature to obtain the sequence-to-image cross-modal attention output; The image-to-sequence cross-modal attention output and the sequence-to-image cross-modal attention output are adaptively pooled and then added. After addition, they are layer-normalized to obtain bidirectional cross-modal fusion features.

6. A method for protein subcellular localization based on gated multimodal routing according to claim 4, characterized in that: If the gating weight vector corresponds to the activation weight of the channel gating modulation module, the channel gating modulation method is used to perform multimodal feature fusion on the image features and sequence features to obtain multimodal fusion features, specifically including: Perform a global maximum pooling operation on the sequence features, and then sequentially activate them through a multi-layer perceptron transformation and a nonlinear activation function to obtain a channel gating vector; The channel gating vector is multiplied element-wise with the image feature to obtain the channel gating modulation fusion feature.

7. The method for protein subcellular localization based on gated multimodal routing according to claim 1, characterized in that: In S4, a weighted adjacency matrix is ​​constructed based on the STRING database and the subcellular compartment distance matrix to obtain a protein map, which specifically includes: Obtain protein interaction scores from the STRING database; Define the subcellular compartment distance matrix; According to the protein interaction scores and the subcellular compartment distance matrix, edge weights are calculated and a weighted adjacency matrix is ​​constructed to obtain a protein graph.

8. The method for protein subcellular localization based on gated multimodal routing according to claim 1, characterized in that: In S5, based on the multimodal fusion features and protein graph, feature propagation is performed through the location-aware neural network to obtain aggregated neighbor features, specifically including: Based on the protein graph, the multimodal fusion features are propagated through the topological convolution layer to obtain the next layer of multimodal fusion features; Through the compartmental attention mechanism, the attention weight of the multimodal node to the neighboring nodes is calculated; Through the path aggregation module, the feature aggregation is performed by combining the next layer of multimodal fusion features and the attention weights of the multimodal nodes on the neighbor nodes to obtain the aggregated neighbor features.

9. The method for protein subcellular localization based on gated multimodal routing according to claim 1, characterized in that: In S6, multi-label prediction is performed based on aggregated neighbor features, combined with geometric prototype decoding and hierarchical constraint loss to obtain optimized prediction results, including: Based on the aggregated neighbor features, geometric prototype decoding is used to calculate the predicted probability that the protein belongs to a subcellular compartment and obtain the true probability at the same time; Calculate the binary cross entropy loss based on the predicted probability and the true probability; According to the parent-child pairs in the subcellular compartment hierarchy, the difference between the predicted probability of the child compartment and the predicted probability of the parent compartment is calculated to obtain the hierarchical constraint loss; According to the binary cross entropy loss and hierarchical constraint loss, a loss function is constructed, and multi-label prediction is performed to obtain optimized prediction results.

10. A protein subcellular localization system based on gated multimodal routing, characterized in that: include: Acquisition module, used to acquire protein images and protein sequences; The extraction module is used to extract features from protein images using the ConvNeXt-V2 network to obtain image features; and is used to extract features from protein sequences using the ESM-2 model to obtain sequence features; The feature fusion module is used to perform multimodal feature fusion on image features and sequence features based on the Sparsemax gating network to obtain multimodal fusion features; The Sparsemax gating network includes: a bidirectional cross-modal self-attention module, a hierarchical two-stream collaboration module and a channel gating modulation module; A graph construction module is used to construct a weighted adjacency matrix based on the STRING database and the subcellular compartment distance matrix to obtain a protein graph; An aggregation module is configured to propagate features based on the multimodal fusion features and the protein graph through a location-aware neural network to obtain aggregated neighbor features; the location-aware neural network includes a topological convolution layer, a compartmental attention layer, and a path aggregation layer; The prediction module is used to perform multi-label prediction based on aggregated neighbor features, combined with geometric prototype decoding and hierarchical constraint loss to obtain optimized prediction results.

Images