A method for calculating the association between diseases and RNA based on heterogeneous graph transformers
By employing a computational method based on heterogeneous graph transformers, combined with multilayer HGT networks and autoencoder modules, the limitations of existing technologies in predicting miRNA-disease associations are overcome, achieving more efficient association prediction and improving the model's accuracy and stability.
Patent Information
- Application Number
- CN202311185012.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-09-13
- Publication Date
- 2026-01-06
- Estimated Expiration
- 2043-09-13
AI Technical Summary
Existing similarity measurement and machine learning methods have limitations in predicting the association between miRNAs and diseases, and are unable to effectively capture heterogeneous nodes and different distribution relationships in complex graph data.
A computational method based on heterogeneous graph transformers was adopted. By combining a multi-layer HGT network and an autoencoder module with attention and multi-head attention mechanisms, a network for the association between miRNAs and diseases was constructed. The model was optimized using the cross-entropy loss function to predict the association between miRNAs and diseases.
It improves the accuracy and stability of miRNA-disease association prediction, significantly outperforming existing methods, and enhances the model's generalization and information capture capabilities.
Smart Images
Figure CN117238366B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of disease prediction technology, specifically a method for calculating the association between diseases and RNA based on a heterogeneous graph transformer. Background Technology
[0002] MicroRNAs (miRNAs) are short, non-coding, single-stranded RNAs, approximately 19–25 nucleotides in length, transcribed from genes. miRNAs regulate protein and gene expression levels by binding to the 3' untranslated region of target miRNAs. Furthermore, mounting evidence suggests that miRNAs play crucial roles in various biological processes, including cell proliferation, differentiation, apoptosis, and aging, and are closely related to the development of many human diseases. For example, miR-15 and miR-16 contribute to chronic lymphocytic leukemia. Additionally, upregulation of let-7 and miR-143 / miR-145 can lead to cancer, while axl, as a receptor, is regulated by miR-34a and miR-199a / b and downregulated in solid cancer cells. Therefore, identifying diseases associated with miRNAs can better help us understand the mechanisms of various diseases.
[0003] Based on the assumption that "functionally similar miRNAs are more correlated with diseases characterized by similar phenotypes," methods using similarity measurements to predict correlations have been developed. Jiang et al. predicted the association between miRNAs and diseases by constructing functional networks of miRNAs and human phenotype networks. Shi et al. focused on the association between disease genes and miRNAs in protein-protein interaction (PPI) networks and proposed a random walk computation method. Xu et al. proposed a prediction method that integrates disease gene associations and miRNA target interactions. The HDMP model predicts the association between miRNAs and diseases based on the functional similarity of miRNAs and the distribution of disease-related miRNAs in their k neighbors. Furthermore, the MDHGI model fully utilizes matrix factorization to integrate similarity information into heterogeneous networks of miRNAs and diseases. Due to the limitations of similarity scores in similarity-based methods, researchers have begun to consider machine learning-based models to predict miRNA-disease associations. Studies have shown that machine learning has achieved good results in this area. For example, the PBMDA model constructs a heterogeneous graph consisting of three interconnected subgraphs and uses a depth-first search algorithm to obtain scores related to miRNA-disease associations. Xu et al. constructed a heterogeneous miRNA targeting dysregulation network (MTDN) and used support vector machines to identify positive associations between miRNAs and diseases. Furthermore, Chen et al. proposed a model called LRSSLMDA, which uses Laplacian regularization and sparse subspace learning to learn the associations between miRNAs and diseases. Chen et al. used a random forest classifier and computed miRNA-disease correlation scores based on an autoencoder. While machine learning saves time, it also has some limitations. Therefore, a method based on a heterogeneous graph transformer to compute disease-RNA associations was proposed. Summary of the Invention
[0004] To address the shortcomings of existing technologies, this invention provides a method for calculating the association between diseases and RNA based on a heterogeneous graph transformer, thus solving the problems mentioned in the background section.
[0005] To achieve the above objectives, the present invention provides the following technical solution: a method for calculating the association between diseases and RNA based on a heterogeneous graph transformer, comprising the following steps:
[0006] S1: Dataset Construction: miRNA-disease association data were downloaded using the Human MicroRNA Disease Database;
[0007] S2: Multi-view similarity measurement module construction: Building a network between miRNAs and diseases, using their similarity as the basis for connecting their relationships:
[0008] (1)
[0009] (2)
[0010] in, and These represent the adjacency matrices of different miRNA-miRNA association networks. Similarly, and These represent the adjacency matrices of different disease-disease association networks.
[0011] S3: Autoencoder Module Construction: Data generated using Python's random module represents random vectors encoding miRNAs and diseases. Each vector is input into a separate autoencoder model, and the randomly generated vectors are encoded as low-dimensional embeddings to capture the basic features of the original data. The embedding process is defined as follows:
[0012] (3)
[0013] (4)
[0014] in and These are randomly generated vectors, representing the similarity matrix between miRNA and disease, respectively;
[0015] S4: Optimize the heterogeneous graph transformer module:
[0016] First, it is necessary to calculate the mutual attention between the source node d and the target node m, using an attention-based graph neural network:
[0017] (5)
[0018] in Represent the neighbors of the target node m, and Let represent all edges from source node d to target node m. In the Hierarchical Graph Transformer (HGT) model, a layer is represented by l. Attention-based graph neural networks rely on three basic operators: attention mechanism, message passing, and aggregation. The attention mechanism calculates the relevance score of each source node relative to a given target node. Message passing extracts information from source nodes and constructs messages that capture the relationships between them. Finally, target-specific aggregation uses the attention scores as weights to combine messages from neighboring nodes. To improve the performance of GNNs, these operators need to be carefully designed and customized according to specific application domains. Compared to GAT, the heterogeneous mutual attention mechanism can compute the mutual attention between target node m and all its neighboring nodes, which may exist in different distributions. This mechanism is particularly useful for tasks involving graphs with heterogeneous nodes and different distributions. This computation is based on their meta-relationships, i.e. Three aspects;
[0019] A novel mechanism for calculating the attention score between source and target nodes in a graph is proposed. The target node is transformed into a query vector, and the source node into a key vector. To achieve a balance between parameter sharing and relation-specific attributes, the weight matrix of the interaction operator is decomposed into three components: the projection of source node d, the projection of edge e, and the projection of target node m. Multi-head attention is used to compute the attention score for each edge. of -head attention score;
[0020] (6)
[0021] (7)
[0022] (8)
[0023] (9)
[0024] S41: For the i-th attention... Using linear projection Project the source node d onto the i-th key vector. in Indicates the number of heads of interest. This represents the dimension of the vector for each attention head. Note The linear projection is organized according to the type of the source node d, which ensures that each type of node has a different linear projection, effectively capturing distribution differences. For the target node m, a linear projection is used. Project it onto the i-th query vector middle;
[0025] S42: This is for calculating the query vector. and Key vector) A significant characteristic of heterogeneous graphs is that, for a given pair of node types, there can be multiple types of edges (relationships), for example... and Unlike a regular transformer that computes the dot product between the query vector and the key vector, we perform a transformation for each type of... The edge matrix uses a single edge-based matrix. .
[0026] S44: Calculate a weighted value for each attention head and concatenate them to create an attention vector for each pair of nodes. Then, collect all attention vectors from neighboring nodes and apply the softmax function to normalize their values to one. This process is calculated as follows:
[0027] (10)
[0028] In addition to calculating mutual attention, an additional technique is introduced to propagate information from the source node to the target node. To overcome the challenges posed by different types of nodes and varying edge distributions, meta-edge relationships are integrated into the message passing process. For multi-head message computation of a pair of nodes, the following definition is provided:
[0029] (11)
[0030] (12)
[0031] (13)
[0032] To retrieve the i-th message header The first step is to use linear projection. To obtain the i-th information vector Next, we will use a matrix. Merge edge dependencies, and finally collect all message headers to calculate the message HGT for each pair of nodes;
[0033] To transmit information from source node d to target node m, a technique called heterogeneous multi-head attention is employed. After calculating the attention vector of each target node using the softmax function in formula (6), these vectors can be used as weights to obtain a weighted average of messages from the corresponding source nodes. This effectively aggregates relevant information, and the newly generated vectors for the target nodes are represented as follows:
[0034] (14)
[0035] By using this aggregation process, information can be collected from nearby nodes with different feature distributions and used to update the target node; the final step involves mapping the target node's vector back to its nodes. The distribution corresponding to a specific type is indexed by the node type, which is achieved through a series of steps, including applying a linear projection to the updated vector and then using a residual connection as...
[0036] (15)
[0037] After performing the above steps, the first node of the target node m is obtained. Output of each HGT layer By stacking these blocks in layers, HGT is able to generate a representation with a uniform height for each node. ;
[0038] By employing a novel meta-relation-based approach, the ability of ML-HGT to capture information from complex graphs is further improved. The proposed method model is formulated as follows:
[0039] (16)
[0040] in Indicates the HGT's first In layer e, which is the feature matrix of the edge, the same operation as described above is performed on the source node d, embedding the encoding matrix into the HGT (Hierarchical Graph Transformer) framework, and performing four layers of HGT operations to output the feature encoding;
[0041] S5: Construct the inner product decoder module: To obtain better results, the output codes of each round of HGT operation are concatenated as the final code;
[0042] (17)
[0043] in and These represent the final miRNA and disease feature codes, respectively. The coding matrix is decoded through multiplication to predict a new miRNA disease sensitivity association matrix. The final prediction score is calculated using the following formula:
[0044] (18)
[0045] (19)
[0046] in This represents the final predicted association matrix between miRNAs and diseases.
[0047] The parameters are denoted as learnable. Finally, we successfully predicted the susceptibility to miRNA diseases using a multilayer heterogeneous graph transformer with random autoencoders.
[0048] To optimize the model, the cross-entropy loss function is applied to calculate the loss during model training:
[0049] (20)
[0050] in and These represent the positive and negative association datasets for all miRNAs and disease pairs, respectively. The true label of the correlation matrix is marked as 1 or 0.
[0051] Preferably, in step S1: Dataset construction: the step of downloading miRNA-disease association data using the Human MicroRNA Disease Database: the version of the MicroRNA Disease Database is HMDD v3.2, which is an updated database containing extensive associations between human miRNAs and diseases.
[0052] Preferably, in step S2: the construction of the multi-view similarity measurement module, MA and DA represent the similarity matrices of miRNA and disease, respectively.
[0053] Preferably, in step S1: Dataset construction: the step of downloading miRNA-disease association data using the Human MicroRNA Disease Database: from the HMDD v3.2 database, 12,446 experimentally confirmed associations between 853 miRNAs and 591 diseases were selected. Known miRNA-disease associations were marked as positive samples, while the remaining samples were marked as negative samples, indicating that there was no association between miRNAs and diseases.
[0054] Preferably, in step S2: the construction of the multi-view similarity measurement module, the functional similarity, sequence similarity, and Gaussian interaction spectrum kernel similarity matrix of miRNAs, as well as the semantic similarity, target-based disease similarity, and Gaussian interaction spectrum kernel similarity matrix of diseases are used to construct three different miRNA-miRNA networks and three different disease networks. These matrices are used as adjacency matrices to establish connections between miRNAs and diseases based on their similarity.
[0055] Preferably, in step S3: the autoencoder module is constructed by using random vectors representing miRNAs and disease codes generated by the Python random module, inputting each vector into a separate autoencoder model, encoding the randomly generated vectors as low-dimensional embeddings to capture the basic features of the original data, and in the step of the autoencoder model being a fully connected layer.
[0056] This invention provides a method for calculating the association between diseases and RNA based on a heterogeneous graph transformer, which has the following beneficial effects:
[0057] This method for calculating disease-RNA associations based on heterogeneous graph transformers (HGTs) predicts miRNA-disease associations using the VRMHMD computational method. VRMHMD adds a random autoencoding process to multi-mode encoding, encodes two sets of data, and extracts the codes through two different multi-layer HGT networks. The output codes from each layer of the HGT are concatenated as the final code. An attention mechanism is used to fuse the two sets of codes, and matrix multiplication decoding is performed to predict a new miRNA disease sensitivity association matrix. Attached Figure Description
[0058] Figure 1 This is a schematic diagram of the steps and structure of the present invention;
[0059] Figure 2 This is a diagram illustrating the impact of different hidden channels on the performance of the proposed HGT model in this invention.
[0060] Figure 3 The present invention VRMHMD utilizes a multi-head attention mechanism to provide it with a more powerful representation learning graph;
[0061] Figure 4 This is a diagram illustrating the impact of changing the number of layers h in the model on the results of this invention.
[0062] Figure 5 The dimension of the random vector in this invention significantly affects information extraction and model performance.
[0063] Figure 6 For the ablation test of this invention Figure 1 ;
[0064] Figure 7 For the ablation test of this invention Figure 2 ;
[0065] Figure 8 For the ablation test of this invention Figure 3 ;
[0066] Figure 9 These are test diagrams for the six models of this invention. Detailed Implementation
[0067] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.
[0068] Please see Figures 1 to 9 This invention provides a technical solution: a method for calculating the association between diseases and RNA based on a heterogeneous graph transformer, comprising the following steps:
[0069] S1: Dataset Construction: miRNA-disease association data were downloaded using the Human MicroRNA Disease Database;
[0070] S2: Multi-view similarity measurement module construction: Building a network between miRNAs and diseases, using their similarity as the basis for connecting their relationships:
[0071] (1)
[0072] (2)
[0073] in, and These represent the adjacency matrices of different miRNA-miRNA association networks. Similarly, and These represent the adjacency matrices of different disease-disease association networks.
[0074] S3: Autoencoder Module Construction: Data generated using Python's random module represents random vectors encoding miRNAs and diseases. Each vector is input into a separate autoencoder model, and the randomly generated vectors are encoded as low-dimensional embeddings to capture the basic features of the original data. The embedding process is defined as follows:
[0075] (3)
[0076] (4)
[0077] in and These are randomly generated vectors, representing the similarity matrix between miRNA and disease, respectively;
[0078] S4: Optimize the heterogeneous graph transformer module:
[0079] First, it is necessary to calculate the mutual attention between the source node d and the target node m, using an attention-based graph neural network:
[0080] (5)
[0081] in Represent the neighbors of the target node m, and Let represent all edges from source node d to target node m. In the Hierarchical Graph Transformer (HGT) model, a layer is represented by l. Attention-based graph neural networks rely on three basic operators: attention mechanism, message passing, and aggregation. The attention mechanism calculates the relevance score of each source node relative to a given target node. Message passing extracts information from source nodes and constructs messages that capture the relationships between them. Finally, target-specific aggregation uses the attention scores as weights to combine messages from neighboring nodes. To improve the performance of GNNs, these operators need to be carefully designed and customized according to specific application domains. Compared to GAT, the heterogeneous mutual attention mechanism can compute the mutual attention between target node m and all its neighboring nodes, which may exist in different distributions. This mechanism is particularly useful for tasks involving graphs with heterogeneous nodes and different distributions. This computation is based on their meta-relationships, i.e. Three aspects;
[0082] A novel mechanism for calculating the attention score between source and target nodes in a graph is proposed. The target node is transformed into a query vector, and the source node into a key vector. To achieve a balance between parameter sharing and relation-specific attributes, the weight matrix of the interaction operator is decomposed into three components: the projection of source node d, the projection of edge e, and the projection of target node m. Multi-head attention is used to compute the attention score for each edge. of -head attention score;
[0083] (6)
[0084] (7)
[0085] (8)
[0086] (9)
[0087] S41: For the i-th attention... Using linear projection Project the source node d onto the i-th key vector. in Indicates the number of heads of interest. This represents the dimension of the vector for each attention head. Note The linear projection is organized according to the type of the source node d, which ensures that each type of node has a different linear projection, effectively capturing distribution differences. For the target node m, a linear projection is used. Project it onto the i-th query vector middle;
[0088] S42: This is for calculating the query vector. and Key vector) A significant characteristic of heterogeneous graphs is that, for a given pair of node types, there can be multiple types of edges (relationships), for example... and Unlike a regular transformer that computes the dot product between the query vector and the key vector, we perform a transformation for each type of... The edge matrix uses a single edge-based matrix. .
[0089] S44: Calculate a weighted value for each attention head and concatenate them to create an attention vector for each pair of nodes. Then, collect all attention vectors from neighboring nodes and apply the softmax function to normalize their values to one. This process is calculated as follows:
[0090] (10)
[0091] In addition to calculating mutual attention, an additional technique is introduced to propagate information from the source node to the target node. To overcome the challenges posed by different types of nodes and varying edge distributions, meta-edge relationships are integrated into the message passing process. For multi-head message computation of a pair of nodes, the following definition is provided:
[0092] (11)
[0093] (12)
[0094] (13)
[0095] To retrieve the i-th message header The first step is to use linear projection. To obtain the i-th information vector Next, we will use a matrix. Merge edge dependencies, and finally collect all message headers to calculate the message HGT for each pair of nodes;
[0096] To transmit information from source node d to target node m, a technique called heterogeneous multi-head attention is employed. After calculating the attention vector of each target node using the softmax function in formula (6), these vectors can be used as weights to obtain a weighted average of messages from the corresponding source nodes. This effectively aggregates relevant information, and the newly generated vectors for the target nodes are represented as follows:
[0097] (14)
[0098] By using this aggregation process, information can be collected from nearby nodes with different feature distributions and used to update the target node; the final step involves mapping the target node's vector back to its nodes. The distribution corresponding to a specific type is indexed by the node type, which is achieved through a series of steps, including applying a linear projection to the updated vector and then using a residual connection as...
[0099] (15)
[0100] After performing the above steps, the first node of the target node m is obtained. Output of each HGT layer By stacking these blocks in layers, HGT is able to generate a representation with a uniform height for each node. ;
[0101] By employing a novel meta-relation-based approach, the ability of ML-HGT to capture information from complex graphs is further improved. The proposed method model is formulated as follows:
[0102] (16)
[0103] in Indicates the HGT's first In layer e, which is the feature matrix of the edge, the same operation as described above is performed on the source node d, embedding the encoding matrix into the HGT (Hierarchical Graph Transformer) framework, and performing four layers of HGT operations to output the feature encoding;
[0104] S5: Construct the inner product decoder module: To obtain better results, the output codes of each round of HGT operation are concatenated as the final code;
[0105] (17)
[0106] in and These represent the final miRNA and disease feature codes, respectively. The coding matrix is decoded through multiplication to predict a new miRNA disease sensitivity association matrix. The final prediction score is calculated using the following formula:
[0107] (18)
[0108] (19)
[0109] in This represents the final predicted association matrix between miRNAs and diseases. The parameters are denoted as learnable. Finally, we successfully predicted the susceptibility to miRNA diseases using a multilayer heterogeneous graph transformer with random autoencoders.
[0110] To optimize the model, the cross-entropy loss function is applied to calculate the loss during model training:
[0111] (20)
[0112] in and These represent the positive and negative association datasets for all miRNAs and disease pairs, respectively. The true label of the correlation matrix is marked as 1 or 0.
[0113] Further, S1: Dataset Construction: Steps for downloading miRNA-disease association data using the Human MicroRNA Disease Database: The version of the MicroRNA Disease Database is HMDD v3.2, which is an updated database containing extensive associations between human miRNAs and diseases.
[0114] Furthermore, in step S2: the construction of the multi-view similarity measurement module, MA and DA represent the similarity matrices of miRNA and disease, respectively.
[0115] Further, S1: Dataset Construction: Steps for downloading miRNA-disease association data using the Human MicroRNA Disease Database: From the HMDD v3.2 database, 12,446 experimentally confirmed associations between 853 miRNAs and 591 diseases were selected. Known miRNA-disease associations were marked as positive samples, while the remaining samples were marked as negative samples, indicating that there was no association between miRNAs and diseases.
[0116] Furthermore, in step S2: the multi-view similarity measurement module construction step uses the functional similarity, sequence similarity, and Gaussian interaction spectrum kernel similarity matrix of miRNAs, as well as the semantic similarity, target-based disease similarity, and Gaussian interaction spectrum kernel similarity matrix of diseases to construct three different miRNA-miRNA networks and three different disease networks. These matrices are used as adjacency matrices to establish connections between miRNAs and diseases based on their similarity.
[0117] Furthermore, S3: The autoencoder module is constructed: using random vectors representing miRNAs and disease codes generated by Python's random module, each vector is input into a separate autoencoder model, and the randomly generated vectors are encoded as low-dimensional embeddings to capture the basic features of the original data in the step of the autoencoder model being a fully connected layer.
[0118] To evaluate the effectiveness of the proposed model, 5-CV and 10-CV experiments were performed on a standard dataset. Negative miRNA disease sensitivities were randomly generated in equal numbers to match positive samples. These were then divided into five equal-sized subsets, each used as the test set once, while the other four subsets were used for training. This process was repeated five times to ensure reliable results. Seven metrics were used to evaluate the model's performance: Area Under the Receiver Operating Characteristic (AUC), Precision-Recall (AUC), Accuracy, Precision, Recall, F1 Score, and Specificity. These metrics were calculated using specific formulas (21-25).
[0119]
[0120] Example:
[0121] First, positive samples showing known miRNA-disease correlations between 853 miRNAs and 591 diseases were collected. To ensure dataset balance, an equal number of negative samples were randomly selected for samples labeled 0. The data was then split into training and test sets, with the latter comprising 20% of the dataset. During training, 5x cross-validation was used to tune model parameters and architecture, while the best-performing model was independently tested. To achieve optimal generalization performance, the dimension of the hidden layer output encoding was set to 512, the number of Hierarchical Graph Transformer (HGT) layers was set to 2, the dimension of the random autoencoder vector was set to 256, and the number of attention heads in the HGT model was set to 8. Furthermore, dropout (p=0.5) was implemented by randomly ignoring some neurons during training to address potential overfitting. All experiments were repeated multiple times to ensure the reliability of the results.
[0122] In this experiment, the model of the present invention was compared with eight existing methods based on individual tests, including NIMCGCN, MMGCN, ERMDA, HGANMDA, AGAEMD, MINIMDA, MAGCN and AMHMDA.
[0123] 1. The NIMCGCN (Geographed Convolutional Network) extracts features from similarity networks. It then uses the NIMC model to generate a complete association matrix.
[0124] 2. MMGCN uses a multi-view GCN encoder, a multi-channel attention mechanism, and a CNN combiner to predict the association between miRNAs and diseases;
[0125] 3. ERMDA proposes a resampling strategy to construct multiple balanced training subsets derived from various data sources to obtain feature representations. Finally, it uses a soft-voting ensemble method to predict the correlation between miRNAs and diseases according to the proposed strategy.
[0126] 4. HGANMDA utilizes node and semantic-level attention mechanisms to learn the meaning of neighboring nodes and metapaths, thereby reconstructing the association between miRNAs and diseases;
[0127] 5. AGAEMD uses a node-level attention-based autoencoder to aggregate relevant information in the miRNA disease network, which is then used to reconstruct the miRNA disease association network;
[0128] 6. MINIMDA constructs different multimodal networks and learns the embedding representations of miRNAs and diseases by fusing hybrid high-order neighborhood information from these networks. This method can infer the correlation between miRNAs and diseases;
[0129] 7. MAGCN uses a combination of GCN and CNN with multi-channel attention mechanism to learn the established lncRNA-miRNA and miRNA disease network, and then reconstructs miRNA disease associations through a bilinear decoder;
[0130] 8. AMHMDA uses a similarity network and a graph convolutional network with supernodes to fuse attention mechanisms to construct heterogeneous supergraphs, allowing the use of attention mechanisms to predict miRNA disease associations.
[0131] To evaluate predictive performance, a 5-fold cross-validation experiment was conducted on the dataset. Under the same experimental conditions, all methods were compared with the best recommended parameters of the study. The comparison of the 5-CV experimental results can be seen in Table 1. The AUC results show that VEMHMD significantly outperforms other methods, exceeding them by 7.36% (NIMCGCN), 3.6% (MMGCN), 5.1% (ERMDA), 4.16% (HGANMDA), 4.11% (AGAEMD), 3.77% (MINIMDA), 4.36% (MAGCN), and 2.59% (AMHMDA). Furthermore, VEMHMD's average AUPRC score was 0.9625, which was 6.99% (NIMCGC), 3.05% (MMGCN), 4.52% (ERMDA), 3.72% (HGANMDA), 3.39% (AGAEMD), 2.75% (MINIMDA), 3.57% (MAGCN), and 2.14% (AMHMDA) higher than other methods. Table 1 shows the results for various performance metrics, such as F1 score, accuracy, recall, and precision. Among these metrics, VEMHMD scored the highest, at 0.9134, 0.911, 0.9389, and 0.8894, respectively.
[0132] Table 1 5-CV
[0133] 1. Parameter sensitivity analysis:
[0134] The VRMHMD model includes the following parameters: 1. the dimension of the hidden layer output encoding; 2. the number of heads in the multi-head attention mechanism; 3. the number of layers in the HGT; and 4. the dimension of the variational autoencoder. To evaluate its predictive performance, experiments were conducted on a benchmark dataset using 5x cross-validation.
[0135] 2. Dimensions of hidden layer output encoding:
[0136] The study investigated the impact of different hidden channels on the performance of the proposed HGT model, and the results are as follows: Figure 2 As shown, when the number of hidden channels is set to 512, the AUC, AUPR, Acc, and F1 scores achieve the best performance scores.
[0137] 3. Pay attention to the number of heads in the bullish mechanism:
[0138] VRMHMD utilizes a multi-head attention mechanism to provide it with more powerful representation learning capabilities. For example... Figure 3 As shown, optimal performance is achieved when the number of attention heads reaches eight. The results indicate that increasing the number of attention heads can improve the performance of the HGT model within a certain range.
[0139] 4. Number of layers in HGT:
[0140] Evaluate the impact of changing the number of layers h in the model on the results, such as Figure 4 As shown, it was observed that optimal performance was achieved when the number of layers reached two. Based on these experiments, the dimension of the random vector was set to 256, the number of hidden channels to 512, and the number of heads in the multi-head attention mechanism to 4. This configuration yielded optimal performance in terms of evaluation metrics. These results provide important insights for the design selection of HGT models.
[0141] Dimension of variational autoencoders
[0142] The dimensionality of a random vector significantly affects information extraction and model performance. For example... Figure 5 As shown, different evaluation metrics scores were observed for random vectors of different dimensions. To ensure the robustness of the model and the best predictive performance of the feature information, the feature dimension of all nodes was standardized to 256.
[0143] The VRMHMD model proposed in this invention comprises four basic modules: multi-view similarity measurement, random encoding, multi-layer HGT framework, and connection decoding. To evaluate the impact of each component on the results, five cross-validation experiments were conducted on a benchmark dataset after removing each module from the AMHMDA model. Specifically, the following six models were tested and compared:
[0144] 1. VRMHMD Model: VRMHMD is a computational framework based on Heterogeneous Graph Transformer (HGT) for predicting the association between miRNAs and disease susceptibility. It utilizes an autoencoder, attention-based multi-turn output encoding, and multi-view similarity for measurement.
[0145] 2. VRMHMD-LDE: This model uses linear layers for decoding. A multi-layered HGT framework captures the complex interactions between miRNAs and disease through an attention-based information transfer mechanism. Finally, based on the linear decoding module, a linear transformation is applied to the final hidden state of the HGT framework to generate the final prediction.
[0146] 3. VRMHMD NBias: The VRMHMD NBias model does not use random coding. It relies solely on multi-view similarity measurements and the HGT framework to predict the association between miRNAs and diseases.
[0147] 4. VRMHMD-HAN: This model uses HAN as the central encoder, leverages heterogeneous attention mechanisms to learn the features of different types of nodes, and models their complex relationships. The VRMHMD-HAN model also retains the random encoding and decoding modules, replacing the MHGT framework with a simple HAN backbone.
[0148] 5. VRMHMD-NL: This model removes the first linear layer from the decoder. It generates the final prediction directly from the last hidden state of the HGT framework without any additional linear transformation.
[0149] 6. VRMHMD Fout: This model retains the random coding module and MHGT framework, using the last layer of HGT as the final decoding vector.
[0150] Figure 9 The results show that VRMHMD consistently outperforms other models in terms of ROC, PRC, Acc, and f1 scores. Experimental results demonstrate that using randomly generated vectors improves the robustness of the autoencoder, enhancing model stability and generalization ability. Furthermore, VRMHMD outperforms VRMHMD-NBias and VRMHMD-HAN when processing heterogeneous graph data. This is because VRMHMD introduces a self-attention mechanism in the converter model to better capture complex dependencies between nodes. Conversely, VRMHMD-NBias can only consider isomorphic relationships, and VRMHMD-HAN can only establish attentional connections between two layers of graphs, making it difficult to effectively handle multi-layered heterogeneous graphs. It should be noted that VRMHMD-HAN is more flexible and accurate than VRMHMD-NL, and exhibits better performance. Moreover, VRMHMD is generally more suitable than VRMHMD-LDE during decoding.
[0151] The above are merely preferred embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.
Claims
1. A method for computing disease-RNA associations based on a heterogeneous graph transformer, the method comprising: The method comprises the following steps: S1: dataset construction: download the miRNA-disease association data from the human microRNA disease database; S2: multi-view similarity measurement module construction: build a network between miRNAs and diseases, and use their similarities as the basis for connecting their relationships: (1) (2) wherein, and A and A represent the adjacency matrices of different miRNA-miRNA association networks, respectively, and A and A represent the adjacency matrices of different disease-disease association networks, respectively, S3: autoencoder module construction: use the data generated by the Python random module to represent the random carriers of miRNA and disease encoding, input each carrier into a separate autoencoder model, and encode the randomly generated vector as a low-dimensional embedding to capture the basic features of the original data. The embedding process is defined as follows: , (3) ， (4) wherein and are randomly generated vectors representing the miRNA and disease similarity matrices, respectively; S4: optimize the heterogeneous graph transformer module: First, the mutual attention between the source node d and the target node m needs to be calculated, and the edge is denoted as e. The attention-based graph neural network is as follows: （5） wherein denotes the neighbors of the target node m, in equation (5) denotes all edges from source nodes d to target node m, in the Hierarchy Graph Transformer (HGT) model, the layers are denoted by l, the attention-based graph neural network relies on three basic operators: attention mechanism, message passing and aggregation, the attention mechanism calculates the relevance score of each source node with respect to a given target node, the message passing extracts information from the source nodes and constructs messages that capture the relationship between them, finally, the target-specific aggregation will combine the messages of neighboring nodes using the attention scores as weights, in order to improve the performance of GNN, these operators need to be carefully designed and customized according to specific application domains; compared with GAT, the heterogeneous mutual attention mechanism can calculate the mutual attention between the target node m and all its neighbor nodes, which may exist in different distributions, this mechanism is particularly useful for tasks involving graphs with heterogeneous nodes and different distributions, this calculation is based on their meta-relationships, i.e. three aspects; A new mechanism for calculating attention scores is proposed, which converts the target node into a query vector and the source node into a key vector. In order to achieve parameter sharing and balance between relationship-specific properties, the weight matrix of the interaction operator is decomposed into three components: the projection of the source node d, the projection of the edge e, and the projection of the target node m. Multi-head attention is used to calculate the attention score of each edge -head attention score , (6) , (7) , (8) , (9) S41: For the i-th attention , a linear projection is used to project the source node d into the i-th key vector, where is the dimension of d, denotes the dimension size of the vector for each attention head, denotes the number of attention heads, and it is noted that the linear is organized according to the type of the source node d, ensuring that each type of node has a different linear projection, effectively capturing the distribution difference, and for the target node m, a linear projection is used to project it into the i-th query vector . S42: Calculate the similarity between Query vector and Key vector A significant feature of heterogeneous graphs is that for a given pair of node types, there are multiple edges (relationships) for each type of edge matrix using a separate edge-based matrix , represents the dimension size of the vector of each attention head, represents the number of attention heads; S43: calculate a weighted value for each attention head and connect them to create a note vector for each pair of nodes, then collect all the attention vectors from adjacent nodes and apply the softmax function to normalize their values to one. The process is calculated as follows: (10) In addition to computing mutual attention, an additional technique is introduced to propagate information from source nodes to target nodes, integrating meta-edge relations into the message passing process, defined as follows for multi-head message computation for a pair of nodes: , (11) , (12) (13) To retrieve the ith message header , the first step is to use linear projection to obtain the ith information vector , next, use the matrix to merge edge dependencies, and finally collect all the message headers to calculate the message HGT of each pair of nodes; In order to transmit information from the source node d to the target node m, a heterogeneous multi-head attention technique is adopted. After using the softmax function in formula (6) to calculate the attention vector of each target node, these vectors are used as weights to obtain the weighted average of the messages from the corresponding source nodes. This can effectively aggregate relevant information and update the vector representation of the target node as follows: (14) By using this aggregation process, information is collected from nearby nodes with different feature distributions and used to update the target node; The final step involves mapping the target node's vector back to its node The corresponding specific type distribution indexed by the node type is mapped by a series of steps including applying a linear projection to the new vector and then using a residual connection as (15) After performing the above steps, the output of the HGT layer of the target node m is obtained By superimposing the layers of these blocks, HGT is able to generate a representation that is unified in height for each node ; By adopting a new meta-relation-based method, the ability of ML-HGT to capture complex graph information is further improved. The formula of the proposed method model is as follows: , (16) wherein represents the first layer, e is the edge feature matrix, the same operation as described above is performed on the source node d, the encoding matrix is embedded into the HGT (hierarchical graph transformer) framework, and a four-layer HGT operation is performed to output the feature encoding; S5: build an inner product decoder module: in order to obtain better results, connect the output encoding of each round of HGT operation as the final encoding; (17) R representing the final encoding corresponding to m and d nodes, through learnable weights weighting the final encoding to obtain a weighted feature encoding: (18) where and represent the final miRNA and disease signature encodings, respectively, and the final prediction score is computed by decoding the encoding matrix through multiplication to predict a new miRNA-disease susceptibility association matrix, with the final prediction score computed as: (19) wherein represents the final predicted association matrix between miRNAs and diseases, To optimize the model, a cross-entropy loss function is applied to calculate the loss during model training: (20) wherein and represent the positive and negative association datasets for all miRNA and disease pairs, respectively, denotes the true labels of the association matrix, labeled as 1 or 0.
2. The method for calculating disease-RNA association based on heterogeneous graph transformer according to claim 1, characterized in that: The S1: dataset construction: download the miRNA-disease association data from the human microRNA disease database step: the microRNA disease database is HMDD v3.
2.
3. The method of claim 1, wherein the method is based on a heterogeneous graph transformer to compute disease-RNA associations. In the S2: multi-view similarity measurement module construction step, MA and DA represent the similarity matrices of miRNAs and diseases, respectively.
4. The method for calculating disease-RNA association based on heterogeneous graph transformer according to claim 1, characterized in that: In the S1: dataset construction: download the miRNA-disease association data from the human microRNA disease database step: from the HMDD v3.2 database, select 12446 experimental verified associations between 853 miRNAs and 591 diseases. Known miRNA-disease associations are labeled as positive samples, while the remaining samples are labeled as negative samples, indicating that there is no association between miRNAs and diseases.
5. The method of claim 1, wherein the method is based on a heterogeneous graph transformer to compute disease-RNA associations. The S2: multi-view similarity measure module constructs the steps of using the functional similarity of miRNA, sequence similarity and Gaussian interaction spectrum kernel similarity matrix, and the semantic similarity of disease, target-based disease similarity and Gaussian interaction spectrum kernel similarity matrix to construct three different miRNA-miRNA networks and three different disease networks, which are used as adjacency matrix, based on their similarity between miRNA and disease.
6. The method of claim 1, wherein the method is based on a heterogeneous graph transformer to compute disease-RNA associations. The S3: autoencoder module constructs: the data generated by the Python random module represents the random carrier of miRNA and disease coding, each carrier is input into a separate autoencoder model, and the randomly generated vector is encoded as a low-dimensional embedding to capture the basic features of the original data.