CircRNA-miRNA interaction prediction method based on graph fusion network

Abstract

The invention provides a circRNA-miRNA interaction prediction method based on a graph fusion network. The circRNA-miRNA interaction prediction method comprises the following steps: acquiring sequence data of target circRNA and miRNA; utilizing a pre-trained natural language processing model to respectively extract semantic feature vectors; using the semantic features as node attributes, and jointly constructing a network diagram with the known interaction relationship; inputting the network graph with the attributes into a multi-level graph fusion network, capturing a local structure and global dependency by the network through a graph convolution layer and then fusing and refining through the graph convolution layer to obtain a node fusion representation; and finally predicting the interaction probability through a classifier. According to the method, sequence semantics and network structure information are effectively fused, the defects of feature loss, global information shortage, difficulty in multi-source fusion and the like of a traditional method are overcome, and the prediction accuracy is remarkably improved.

Description

Technical Field

[0001] This invention relates to the field of bioinformatics processing technology, specifically providing a method for predicting circRNA–miRNA interactions based on graph fusion networks. Background Technology

[0002] In recent years, an increasing number of studies have shown that circRNAs can influence the occurrence and development of various diseases through interactions with miRNAs. circRNAs can bind to and inhibit the activity of specific miRNAs, making them potential therapeutic targets and providing new strategies for disease intervention. For example, some circRNAs can regulate the expression of downstream genes by adsorbing specific miRNAs, thereby affecting key signaling pathways; others regulate tumor invasion and metastasis through competitive binding with miRNAs. Therefore, in-depth research on the diversity of circRNA–miRNA interactions and their potential mechanisms is of great value for understanding their clinical significance and assessing their potential as biomarkers or drug targets. With the rapid development of computing technology, it has become possible to construct computational models for studying molecular interactions. Computational methods aim to utilize multi-source information of molecules to characterize their relationship networks from different perspectives and achieve the prediction of circRNA–miRNA interactions. However, accurately predicting such interactions still faces three major challenges: (1) simplification of molecular property expression; (2) insufficient consideration of global structural information; and (3) difficulty in fusing multi-source features.

[0003] First, the inherent properties of molecules are often simplified to the similarity between similar molecules, which weakens the semantic information within the sequence. Existing methods typically use variational graph autoencoders to learn nonlinear features from adjacency and similarity matrices to characterize the complex relationships between similar molecules. However, these methods still suffer from uncertainty and ignore the true semantic information of the molecules themselves, making the obtained features insufficient to fully reflect complex molecular interaction patterns. By constructing similarity-based heterogeneous networks, structural features between molecules can be extracted more comprehensively, but as the network size increases, the role of structural features is amplified, making the prediction of isolated nodes in the network difficult. Incorporating the intrinsic features of molecules can alleviate this problem and improve the overall performance of the model. Second, many existing methods do not fully consider the influence of global structure on target nodes, but rely mainly on local structure, thus limiting predictive capabilities. Denoising autoencoders can reduce noise and enhance local topological information in sparse networks, thereby obtaining more robust feature representations. However, when dealing with highly complex networks, these methods may filter out some global structural information, thus affecting the overall feature expression. Random walks in heterogeneous networks can capture local structural features under different patterns, but focusing only on local perspectives can easily overlook important global patterns, thus affecting prediction accuracy. Therefore, effectively integrating global structural information into the learning process is crucial. Finally, due to the heterogeneity of multi-source information, how to fully integrate this data to maximize its potential remains a significant challenge. Combining natural language processing techniques with graph embedding methods and fusing multi-source features through fully connected strategies can yield more comprehensive molecular representations. Constructing knowledge graphs and analyzing molecules from multiple perspectives also helps improve feature representation, but traditional fully connected fusion methods lack adaptability to multi-source information, which may cause important features from some sources to be overlooked, thereby weakening the model's expressive power. Summary of the Invention

[0004] To achieve the above objectives, in a first aspect, the present invention provides a method for predicting circRNA–miRNA interactions based on graph fusion networks, comprising: Obtain the target circRNA sequence and the target miRNA sequence; Using a pre-trained natural language processing model, the circRNA sequence and miRNA sequence are encoded respectively, and the corresponding semantic feature vectors are extracted. Specifically, the first encoder trained based on the masked language model is used to extract features from the circRNA sequence, and the second encoder trained based on the substitution marker detection task is used to extract features from the miRNA sequence. Based on the known circRNA-miRNA interaction relationship, an interaction network with circRNA and miRNA as nodes is constructed, and the extracted semantic feature vectors are used as the initial attribute features of the corresponding nodes, thus forming a network graph with initial node features. The network graph with initial node features is input into a multi-level graph fusion network. The multi-level graph fusion network first uses the initial node features as input, and aggregates the first-order neighbor information of each node through a first graph convolutional layer to generate a first-layer node representation containing local topology. Using the first-layer node representation as input, it performs multi-head self-attention calculation for all nodes in the graph through a graph Transformer layer to generate a second-layer node representation containing global dependencies. Using the second-layer node representation as input, it performs feature fusion and refinement through a second graph convolutional layer, outputting the final node fusion representation. Based on the final node fusion representations corresponding to the circRNA and the miRNA, a classifier is used to calculate the probability that there is an interaction between the circRNA and the miRNA, and a prediction is made based on this probability.

[0005] Preferably, the first encoder trained based on the masked language model is a BERT model architecture; when extracting semantic features of circRNA, a first marker and a last marker are added to the beginning and end of the input sequence, respectively, and the hidden state vector corresponding to the first marker in the last Transformer encoder layer is used as the semantic feature vector of the circRNA. The second encoder trained based on the substitution marker detection task is an ELECTRA model architecture. When extracting miRNA semantic features, the discriminator of the ELECTRA model architecture is used to encode the input sequence, and the feature vectors of all output positions are averaged and pooled. The resulting pooled vector is used as the semantic feature vector of the miRNA.

[0006] Preferably, both the first graph convolutional layer and the second graph convolutional layer aggregate neighborhood information in the following way: The adjacency matrix of the interaction network Adding self-loops yields the adjacency matrix after adding self-loops. : in, Represents the identity matrix; Using the degree matrix to add the adjacency matrix after the self-loop Normalization is performed to obtain the normalized adjacency matrix. : in, Degree matrix; Based on normalized adjacency matrix With input node feature matrix Perform graph convolution operations: in, For trainable weight matrix, Represents the coefficient.

[0007] Preferably, the graph Transformer layer employs a scaled dot product multi-head self-attention mechanism, and the calculation process includes: The input node features are mapped to a query matrix through a linear transformation. Key matrix Sum matrix ; Calculate the scaled dot product attention score: in, Indicates the dimension of the key matrix; Multiple attention heads are computed in parallel, and the outputs of each attention head are concatenated and then subjected to a linear transformation to obtain the final output.

[0008] Preferably, the multi-level graph fusion network has a fixed layer dimension setting: the first graph convolutional layer maps the node feature dimension to 64 dimensions; the graph Transformer layer maintains 64-dimensional features; the second graph convolutional layer maps the node feature dimension to 32 dimensions and serves as the final node fusion representation.

[0009] Preferably, the classifier is a LightGBM gradient boosting decision tree model; the interaction prediction step specifically includes: splicing the 32-dimensional fusion representations of the target nodes to form a 64-dimensional feature vector, inputting it into the trained LightGBM classifier, and outputting a prediction probability value between 0 and 1; when the prediction probability value is greater than the set classification threshold, it is determined that there is an interaction, otherwise it is determined that there is no interaction.

[0010] Preferably, the method further includes a step of jointly training the multi-level graph fusion network and the classifier, wherein: The training objective is to minimize the binary cross-entropy loss between the predicted result and the true label; A five-fold cross-validation strategy was used to divide the training and validation sets to evaluate the model's generalization performance. Use the Adam optimizer to optimize hyperparameters to obtain the optimal combination of learning rate and number of training epochs.

[0011] Preferably, after the graph fusion representation learning step, a model interpretability analysis step is also included: The output node representations of the first graph convolutional layer, the graph Transformer layer, and the second graph convolutional layer are dimensionality-reduced using the t-distributed random neighborhood embedding algorithm and visualized in a two-dimensional plane to show the distribution and clustering of local structures, global dependencies, and fusion features captured by the model in each layer.

[0012] Secondly, the present invention provides a circRNA–miRNA interaction prediction system based on graph fusion networks, comprising: The sequence data acquisition module is used to acquire the target circRNA sequence and the target miRNA sequence; A semantic feature extraction module is used to encode the circRNA sequence and miRNA sequence respectively using a pre-trained natural language processing model to extract the corresponding semantic feature vectors. The semantic feature extraction module includes: a first encoder unit trained based on a masked language model for processing circRNA sequences; and a second encoder unit trained based on a substitution marker detection task for processing miRNA sequences. The interaction network construction module is used to construct an interaction network with circRNA and miRNA as nodes based on the known circRNA-miRNA interaction relationship, and to use the extracted semantic feature vectors as the initial attribute features of the corresponding nodes, thereby forming a network graph with initial node features; The graph fusion network module is used to input the network graph with initial node features into a multi-level graph fusion network. The multi-level graph fusion network first uses the initial node features as input, and aggregates the first-order neighbor information of each node through a first graph convolutional layer to generate a first-layer node representation containing local topology. Using the first-layer node representation as input, it performs multi-head self-attention calculation for all nodes in the graph through a graph Transformer layer to generate a second-layer node representation containing global dependencies. Using the second-layer node representation as input, it performs feature fusion and refinement through a second graph convolutional layer, outputting the final node fusion representation. The interaction prediction module is used to calculate the probability that there is an interaction between the circRNA and the miRNA by a classifier based on the final node fusion representation corresponding to the circRNA and the miRNA, and make a prediction based on the probability.

[0013] Thirdly, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the circRNA–miRNA interaction prediction method based on graph fusion networks as described in any one of claims 1 to 8.

[0014] The beneficial effects of this invention are as follows: 1. By extracting semantic information from circRNA and miRNA sequences through a semantic feature extraction module, and balancing local and global structural signals through a multi-level graph fusion network, the synergistic representation of molecular attributes and network topology is achieved. This two-layer design effectively overcomes the limitations of traditional fusion strategies in processing multi-source heterogeneous biological data, enabling the model to more fully capture complex molecular association features.

[0015] 2. Significantly improved prediction accuracy: By fusing sequence semantic features with graph structure information and adopting a multi-level graph neural network architecture of "local-global-re-fusion", the model can capture the complex patterns of molecular interactions more comprehensively and at a deeper level.

[0016] 3. By employing pre-trained language models such as BERT and ELECTRA to extract deep semantic features from long-sequence circRNAs and short-sequence miRNAs respectively, the problem of simplification and loss of molecular attribute information in traditional methods is avoided. By introducing a graph Transformer layer, long-range dependencies between any nodes in the graph can be directly modeled, effectively capturing global structural information ignored by traditional graph convolutional networks. Through the designed graph fusion network, guided by the graph structure, semantic features are organically fused with local and global topological features, rather than simply splicing them together, thus solving the problem of difficult fusion of multi-source heterogeneous features. Attached Figure Description

[0017] The disclosure of this invention will become more readily understood with reference to the accompanying drawings. It will be readily understood by those skilled in the art that these drawings are for illustrative purposes only and are not intended to limit the scope of protection of this invention. Furthermore, similar numbers in the drawings are used to denote similar components, wherein: Figure 1 This is a flowchart of a graph fusion network-based circRNA–miRNA interaction prediction method according to an embodiment of the present invention. Figure 2This is a comparison of multiple experimental results for one embodiment of the present invention. (a) The average AUC of the method under different combinations of learning rate (L) and training epochs (E). (b) Comparison results between the method (T) and the semantic feature extraction module (S), the multi-level graph fusion network (G), and the model with Transformer removed (WT). (c) Comparison results between the method and DeepWalk, LINE, and SDNE. (d) Comparison results between the method and the fully connected fusion strategy. (e) Performance comparison of the method with state-of-the-art models.

[0018] Figure 3 Visualization and interpretability analysis of the method according to one embodiment of the present invention.

[0019] Figure 4 This is a case study of disease-related CMI prediction using the method according to an embodiment of the present invention. (a) Visual analysis of the disease contribution of circRNA and miRNA. (b) Volcano plot of miRNA distribution in hepatocellular carcinoma (HCC). (c) Differential expression analysis of circRNA in HCC samples. (d) Differential expression analysis of miRNA in HCC samples. Detailed Implementation

[0020] Some embodiments of the present invention will now be described with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0021] Example 1 like Figure 1-4 As shown, this invention provides a method for predicting circRNA–miRNA interactions based on graph fusion networks, comprising: Step S1: Obtain the target circRNA sequence and the target miRNA sequence.

[0022] In this embodiment, three datasets—CircBank, CMI-9905, and CMI-2767—were used to evaluate the performance of the proposed method. The CircBank dataset, sourced from the CircBank database, selected high-confidence interaction pairs and consists of 2,115 circRNAs, 821 miRNAs, and 9,589 interactions. The CMI-9905 dataset, constructed by Wang et al., integrates information from databases such as CircBank and circR2Cancer, containing 2,346 circRNAs, 962 miRNAs, and 9,905 interactions, all derived from literature, experimental validation, and CircBank data resources. Sequence information for the corresponding circRNAs and miRNAs was supplemented to the two datasets to improve the completeness of model training. Furthermore, a novel experimental validation dataset, CMI-2767, was constructed. This dataset integrates information from multiple databases, including LncACTdb 3.0, CircNet 2.0, and RNAInter v4.0, as well as relevant literature reports. The CMI-2767 dataset contains 1,649 circRNAs, 1,032 miRNAs, and 2,767 experimentally supported interactions.

[0023] To construct a balanced training set, the same number of negative samples as the number of positive samples were randomly generated, and all negative samples did not overlap with any known positive samples. Multiple evaluation metrics, including Acc., Precision, Recall, Spec., and F1 score (F1), were used to comprehensively evaluate the model's performance. The formulas for calculating these evaluation metrics are as follows: MACROBUTTON MTPlaceRef \ MERGEFORMAT MACROBUTTON MTPlaceRef \ MERGEFORMAT MACROBUTTON MTPlaceRef \ MERGEFORMAT MACROBUTTON MTPlaceRef \ MERGEFORMAT MACROBUTTON MTPlaceRef \ MERGEFORMAT In addition, the area under the receiver operating characteristic (AUC) curve and the area under the precision-recall curve (AUPR) curve were used as important metrics for evaluating classification performance. All evaluation metrics were calculated based on 5-fold cross-validation (5-fold CV) to comprehensively assess the performance of this method.

[0024] Step S2: Using a pre-trained natural language processing model, the circRNA sequence and miRNA sequence are encoded respectively to extract the corresponding semantic feature vectors; wherein, the first encoder trained based on the masked language model is used to extract features from the circRNA sequence, and the second encoder trained based on the substitution marker detection task is used to extract features from the miRNA sequence.

[0025] In one embodiment, the first encoder trained based on the masked language model is a BERT model architecture; when extracting semantic features of circRNA, a first marker and a last marker are added to the beginning and end of the input sequence, respectively, and the hidden state vector corresponding to the first marker in the last Transformer encoder layer is used as the semantic feature vector of the circRNA. The second encoder trained based on the substitution marker detection task is an ELECTRA model architecture. When extracting miRNA semantic features, the discriminator of the ELECTRA model architecture is used to encode the input sequence, and the feature vectors of all output positions are averaged and pooled. The resulting pooled vector is used as the semantic feature vector of the miRNA.

[0026] Specifically, BERT's bidirectional encoding mechanism can accurately model the contextual dependencies in long circRNA sequences; while ELECTRA processes short sequences through a generator-discriminator framework. The generator constructs pseudo-samples similar to real miRNA sequences, and the discriminator enhances the feature extraction process by distinguishing between real and fake samples.

[0027] circRNA semantic feature extraction: To address the significant variation in circRNA sequence length, shorter sequences are first padded to ensure consistent input sequence length across the entire dataset. Then, each circRNA sequence is encoded using a tokenizer and converted into a tensor conforming to the BERT input format. A start tag [CLS] and a end tag [SEP] are added to the sequence, where [CLS] identifies the sequence start and [SEP] delineates sequence boundaries, ensuring the model can correctly recognize and process contextual information within the sequence. During training, 15% of the input tokens are randomly masked, requiring the model to recover the masked true values. The cross-entropy loss function is used to measure the difference between the predicted and true values. Finally, the [CLS] vector extracted from the last hidden state of BERT is used as the contextual semantic representation of the circRNA for subsequent feature modeling.

[0028] miRNA semantic feature extraction: ELECTRA employs a substitution-based pre-training strategy: it uses a lightweight generator network to generate plausible alternatives for partial input tokens, thereby constructing a "broken" input sequence. ELECTRA consists of two Transformer-based neural networks, namely the generator... With discriminator For generators The input token sequence is represented as follows After encoding, a context-dependent vector representation can be obtained. .set up For the position of the token to be replaced, then at position The probability of generating a specific token is calculated as follows: MACROBUTTON MTPlaceRef \ MERGEFORMAT For discriminator Its task is to be in position Determine the token at this position It determines whether the token is real or a fake token generated by the generator. This determination is performed through a sigmoid output layer, calculated as follows: MACROBUTTON MTPlaceRef \ MERGEFORMAT The final layer of the discriminator generates a corresponding contextualized vector representation for each token. By averaging these token vectors, a global feature representation of the miRNA sequence can be obtained.

[0029] Step S3: Based on the known circRNA-miRNA interaction relationship, construct an interaction network with circRNA and miRNA as nodes, and use the extracted semantic feature vectors as the initial attribute features of the corresponding nodes to form a network graph with initial node features.

[0030] Step S4: Input the network graph with initial node features into a multi-level graph fusion network; the multi-level graph fusion network first uses the initial node features as input, aggregates the first-order neighbor information of each node through a first graph convolutional layer, and generates a first-layer node representation containing local topology; using the first-layer node representation as input, performs multi-head self-attention calculation for all nodes in the graph through a graph Transformer layer, and generates a second-layer node representation containing global dependencies; using the second-layer node representation as input, performs feature fusion and refinement through a second graph convolutional layer, and outputs the final node fusion representation.

[0031] In this embodiment, the first layer is a GCN layer, used to extract local neighborhood information; the second layer is a Transformer layer, used to capture global structural relationships; and the third layer is a GCN layer, used to further fuse local and global features to achieve bidirectional information integration. For the first GCN layer, the adjacency matrix of the interaction network is denoted as... The molecular semantic feature matrix is ,in As input to the model.

[0032] To ensure that nodes retain their own information during feature aggregation, the adjacency matrix is... Add a self-loop. The calculation method is as follows: MACROBUTTON MTPlaceRef \ MERGEFORMAT in, ; The adjacency matrix after adding self-loops is normalized using the degree matrix to reduce the imbalance in feature aggregation caused by differences in node connectivity. This normalization step ensures more balanced and stable information propagation between nodes during graph convolution. Its operation is defined as follows: MACROBUTTON MTPlaceRef \ MERGEFORMAT in, Degree matrix; Subsequently, the weight matrix is ​​randomly initialized. The loss function is minimized through iterative updates during training. The output of the first layer is represented as... This representation integrates the neighborhood information of the node, and its calculation method is as follows: MACROBUTTON MTPlaceRef \ MERGEFORMAT in, Represents the coefficient.

[0033] For the graph Transformer layer, a multi-head attention mechanism is used to compute the attention scores between nodes. Input node features This will be mapped to a query matrix through a linear transformation. Key matrix Sum matrix The corresponding linear transformations are respectively given by the weight matrix. , and Implementation. The conversion process is defined as follows: MACROBUTTON MTPlaceRef \ MERGEFORMAT MACROBUTTON MTPlaceRef \ MERGEFORMAT MACROBUTTON MTPlaceRef \ MERGEFORMAT The dimension of the key matrix is ​​denoted as .along with The increase of the query matrix AND key matrix The dot product can produce excessively large values. When these values ​​are processed by the softmax function, their gradients may become extremely small, thus affecting the effective training of the model. To alleviate the vanishing gradient problem, the attention score is scaled to stabilize the gradient. Here, a dot product attention mechanism is used to calculate the attention weights. The normalized attention weights are applied to... This generates the information to be passed to the target node. The attention score is calculated as follows: MACROBUTTON MTPlaceRef \ MERGEFORMAT The outputs of each attention head are concatenated and then subjected to a linear transformation to obtain the final multi-head attention output: MACROBUTTONMTPlaceRef \ MERGEFORMAT in, This is the weight matrix used in the output linear transformation.

[0034] To further optimize the relationship between nodes, the third convolutional layer fuses the local and global information extracted from the first two layers. The weight matrix is ​​learnable, and its feature update method is formalized as follows: MACROBUTTON MTPlaceRef \ MERGEFORMAT During model training, the model parameters are learned by minimizing the mean squared error (MSE) loss function, which is defined as follows: MACROBUTTON MTPlaceRef \ MERGEFORMAT in, The model represents the first Predicted features of each node, Indicates the first The true characteristics of each node This represents the number of samples.

[0035] Step S5: Based on the final node fusion representations corresponding to the circRNA and the miRNA, calculate the probability of an interaction between the circRNA and the miRNA using a classifier, and make a prediction based on this probability.

[0036] Furthermore, graph fusion networks integrate semantic and structural features at both local and global levels. During model training, the learning rate and the number of training epochs are two key hyperparameters affecting performance. Therefore, a systematic experimental analysis was conducted on three datasets: CMI-9905, CircBank, and CMI-2767, to explore the impact of different hyperparameter settings on model performance. Specifically, while keeping other parameters constant, the learning rate and the number of training epochs were adjusted within the intervals [0.00002, 0.001] and [110, 600], respectively. For each hyperparameter combination, model performance was evaluated based on the average AUC value of 5-fold cross-validation (5-fold CV).

[0037] The experimental results shown in Figure 2a indicate that the optimal hyperparameter combinations for each of the three datasets are as follows: [0.00014, 370] for CMI-9905, [0.00036, 128] for CircBank, and [0.00008, 210] for CMI-2767. Under these optimal settings, the proposed method achieves average AUCs of 0.9150, 0.9576, and 0.9032 on the corresponding datasets, respectively.

[0038] Furthermore, to evaluate the contribution of each module in this method to the prediction performance, ablation experiments were conducted on multiple datasets. Specifically, the semantic feature extraction module and the multi-level graph fusion network were used independently for node representation learning and prediction on the CMI-9905 and CircBank datasets. The CMI-2767 dataset was not included in this ablation experiment due to the lack of available sequence information. In addition, to evaluate the role of the Transformer layer in capturing global structural features in the multi-level graph fusion network, this layer was removed on all three datasets and the model was retrained. The experimental results are shown in Figure 2b.

[0039] Ablation experiments show that multi-level graph fusion networks can effectively capture structural information between nodes, but the model performance slightly decreases due to the lack of representation of inherent node features. In contrast, the semantic feature extraction module can fully extract the semantic features of nodes, but it cannot model graph structural information, resulting in a significant performance reduction. Furthermore, removing the Transformer layer also leads to a decrease in model performance, indicating that global structural features play a crucial role in modeling node relationships. In summary, this method achieves a more comprehensive node representation by fusing molecular semantics, local structure, and global structure, thereby improving prediction performance.

[0040] Furthermore, unlike traditional graph embedding methods that rely solely on structural features, this method integrates multi-source information. To validate its advantages, this method is compared with several commonly used graph embedding methods, including DeepWalk, LINE, and SDNE. DeepWalk captures graph structural features through random walks, LINE focuses on modeling relationships between neighboring nodes, while SDNE extracts complex nonlinear relationships and global structural information based on a deep learning framework. Experimental results on three datasets under five-fold cross-validation are shown below. Figure 2 As shown in c. The results show that this method has a significant advantage in prediction performance.

[0041] To evaluate the performance of different feature fusion strategies, the model employing the fully connected fusion strategy was compared with our proposed method. Due to the lack of molecular sequence information in CMI-2767, the comparison was only conducted on the CMI-9905 and CircBank datasets. Figure 2d shows the experimental results of the two fusion strategies on these two datasets. The results show that our proposed method using the GCN fusion strategy outperforms the model using the fully connected fusion strategy on multiple evaluation metrics.

[0042] To further demonstrate the performance advantages of this method, it was compared with a series of state-of-the-art models on the CMI-9905 and CircBank datasets, including CMASG, JSNDCMI, BCMCMI, BioKG-CMI, BEROLECMI, and RBNE-CMI. The average AUC of five-fold cross-validation was used as the evaluation metric for all models, and the comparison results are shown in Figure 2e.

[0043] Experimental results demonstrate that our proposed method outperforms existing state-of-the-art models on both the CMI-9905 and CircBank datasets. This further proves the effectiveness of our method in capturing complex molecular features and predicting unknown interactions. Compared to other methods, our approach constructs a higher-quality network representation by fusing semantic and structural features, thereby deepening our understanding of molecular interaction mechanisms.

[0044] By mapping nodes to a low-dimensional space and visualizing them using t-SNE, the operating mechanism of this method can be analyzed more comprehensively. Figure 3 The visualization results show the node feature representations learned by this method on three different datasets.

[0045] In the first layer, this method utilizes GCN to aggregate information from neighboring nodes, resulting in a relatively dispersed but clearly clustered spatial distribution. This result demonstrates that the model can effectively capture local structural features through neighborhood aggregation. In the second layer, this method extracts information from global nodes through the Transformer's self-attention mechanism, thus making the node clustering structure more compact. Some previously dispersed nodes begin to merge, reflecting the Transformer's ability to model long-range dependencies and improving the global consistency of node representations. In the third layer, the model again applies GCN to fuse the information extracted from the first two layers. Compared to the first two layers, the node distribution in this layer maintains certain clustering characteristics while exhibiting clearer boundaries and structures, indicating that the model can better integrate local and global features through information fusion, thereby enhancing the representational power of nodes.

[0046] Furthermore, it is noteworthy that the node distribution in the CMI-2767 dataset is significantly more discrete than that in the CMI-9905 and CircBank datasets. This is primarily because CMI-2767 lacks molecular sequence information, making it impossible to extract semantic features from molecules. In contrast, the first-layer inputs of CMI-9905 and CircBank are derived from the intrinsic semantic features of molecules. Therefore, even relying solely on available structural information, this method can still achieve a certain degree of effective node embedding on the CMI-2767 dataset.

[0047] Case Study: Since the discovery of miRNAs by Victor Ambros, Gary Ruvkun, and their team, their regulatory role in diseases has become a major focus of biomedical research. Over the past decade, most PubMed studies related to circRNAs have emphasized their function as miRNA "sponges," participating in gene regulation by adsorbing miRNAs, particularly in complex tumorigenesis mechanisms. Furthermore, increasing research indicates that circRNAs have the potential to serve as biomarkers and therapeutic targets, and several biotechnology companies are actively advancing clinical research on circRNA-based disease intervention strategies. To objectively evaluate the effectiveness of this method in identifying potential biological experimental candidates, a disease-related heterogeneous network was constructed based on CMI-2767, and independent CMI predictions were performed. Specifically, disease information corresponding to circRNAs and miRNAs in CMI-2767 was collected from multiple authoritative databases, including CircR2DiseaseV2.0, circAtlas 3.0, circRNADisease V2.0, and HMDD v4.0. This method differs from previous CMI prediction methods that primarily rely on existing interactions; instead, it predicts based on circRNA-disease and miRNA-disease relationships. The top 20 predicted CMIs are shown in Table 1, of which 17 pairs have been supported by literature or experimental validation.

[0048] To further understand how this method evaluates the disease contribution of circRNA and miRNA in heterogeneous networks, several prediction results were visualized and analyzed. Specifically, the contributions of circRNA and miRNA to the disease were used as x and y coordinates, respectively, and mapped to a low-dimensional space for analysis. As shown in Figure 4a, true positive samples are significantly far from the origin, exhibiting strong discriminative power; while true negative samples cluster near the origin, showing low difference from other samples in the feature space. This indicates that negative samples exhibit similar distribution patterns in high-dimensional space, making them more difficult to distinguish accurately in downstream prediction tasks.

[0049] Hepatocellular carcinoma (HCC) is the most common type of primary liver cancer, accounting for 75%-85% of all liver cancer cases. circRNA expression profiles of HCC samples were obtained from the GEO database, and differential expression analysis was performed using the limma package in R. As shown in Figure 4c, Table 1 contains three pairs of HCC-related circRNAs, two of which (hsa_circ_0000284 and hsa_circ_0000567) were validated in the expression data. Previous studies have shown that hsa_circ_0000284 and hsa_circ_0000567 inhibit HCC cell proliferation and migration by interacting with hsa-miR-124 and hsa-miR-421, respectively.

[0050] Furthermore, the miRNA expression profiles of HCC samples were analyzed. The volcano plot in Figure 4b shows the distribution of significantly upregulated (red) and downregulated (blue) miRNAs, while the differential analysis in Figure 4d further reveals the aberrant expression of hsa-miR-421. This result further validates the function of hsa_circ_0000567 as a sponge for hsa-miR-421, potentially inhibiting HCC development through its associated pathways. This finding provides a potential key molecular target for targeted therapeutic strategies in HCC.

[0051] Table 1. Top 20 candidate interactions predicted by this method

[0052] Example 2 like Figure 1As shown, this invention provides a circRNA–miRNA interaction prediction system based on graph fusion networks, comprising: a sequence data acquisition module for acquiring target circRNA sequences and target miRNA sequences; a semantic feature extraction module for encoding the circRNA sequences and miRNA sequences respectively using a pre-trained natural language processing model to extract corresponding semantic feature vectors; the semantic feature extraction module includes: a first encoder unit trained based on a masked language model for processing circRNA sequences; a second encoder unit trained based on a substitution marker detection task for processing miRNA sequences; and an interaction network construction module for constructing an interaction network with circRNA and miRNA as nodes based on known circRNA–miRNA interaction relationships, and using the extracted semantic feature vectors as the initial attribute features of the corresponding nodes, thereby… A network graph with initial node features is formed; a graph fusion network module is used to input the network graph with initial node features into a multi-level graph fusion network; the multi-level graph fusion network first uses the initial node features as input, and aggregates the first-order neighbor information of each node through a first graph convolutional layer to generate a first-layer node representation containing local topology; using the first-layer node representation as input, multi-head self-attention calculation of all nodes in the graph is performed through a graph Transformer layer to generate a second-layer node representation containing global dependencies; using the second-layer node representation as input, feature fusion and refinement are performed through a second graph convolutional layer to output the final node fusion representation; an interaction prediction module is used to calculate the probability that there is an interaction between the circRNA and the miRNA through a classifier based on the final node fusion representation corresponding to the circRNA and the miRNA, and make a prediction based on the probability.

[0053] Example 3 The present invention provides a computer-readable storage medium having a computer program stored thereon, characterized in that, when the computer program is executed by a processor, it implements the circRNA–miRNA interaction prediction method based on graph fusion network.

[0054] The technical solution of the present invention has been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the original technical features, and the technical solutions resulting from these changes or substitutions will all fall within the scope of protection of the present invention.

Claims

1. A method for predicting circRNA–miRNA interactions based on graph fusion networks, characterized in that, include: Obtain the target circRNA sequence and the target miRNA sequence; Using a pre-trained natural language processing model, the circRNA sequence and miRNA sequence are encoded respectively, and the corresponding semantic feature vectors are extracted. Specifically, the first encoder trained based on the masked language model is used to extract features from the circRNA sequence, and the second encoder trained based on the substitution marker detection task is used to extract features from the miRNA sequence. Based on the known circRNA-miRNA interaction relationship, an interaction network with circRNA and miRNA as nodes is constructed, and the extracted semantic feature vectors are used as the initial attribute features of the corresponding nodes, thus forming a network graph with initial node features. The network graph with initial node features is input into a multi-level graph fusion network. The multi-level graph fusion network first uses the initial node features as input, and aggregates the first-order neighbor information of each node through a first graph convolutional layer to generate a first-layer node representation containing local topology. Using the first-layer node representation as input, it performs multi-head self-attention calculation for all nodes in the graph through a graph Transformer layer to generate a second-layer node representation containing global dependencies. Using the second-layer node representation as input, it performs feature fusion and refinement through a second graph convolutional layer, outputting the final node fusion representation. Based on the final node fusion representations corresponding to the circRNA and the miRNA, a classifier is used to calculate the probability that there is an interaction between the circRNA and the miRNA, and a prediction is made based on this probability.

2. The method according to claim 1, characterized in that, The first encoder trained based on the masked language model is a BERT model architecture; when extracting semantic features of circRNA, a first marker and a last marker are added to the beginning and end of the input sequence, respectively, and the hidden state vector corresponding to the first marker in the last Transformer encoder is used as the semantic feature vector of the circRNA. The second encoder trained based on the substitution marker detection task is an ELECTRA model architecture. When extracting miRNA semantic features, the discriminator of the ELECTRA model architecture is used to encode the input sequence, and the feature vectors of all output positions are averaged and pooled. The resulting pooled vector is used as the semantic feature vector of the miRNA.

3. The method according to claim 1, characterized in that, Both the first and second graph convolutional layers use the following method to aggregate neighborhood information: The adjacency matrix of the interaction network Adding self-loops yields the adjacency matrix after adding self-loops. : in, Represents the identity matrix; Using the degree matrix to add the adjacency matrix after the self-loop Normalization is performed to obtain the normalized adjacency matrix. : in, Degree matrix; Based on normalized adjacency matrix With input node feature matrix Perform graph convolution operations: in, For trainable weight matrix, Represents the coefficient.

4. The method according to claim 1, characterized in that, The graph Transformer layer employs a scaled dot product multi-head self-attention mechanism, and the calculation process includes: The input node features are mapped to a query matrix through a linear transformation. Key matrix Sum matrix ; Calculate the scaled dot product attention score: in, Indicates the dimension of the key matrix; Multiple attention heads are computed in parallel, and the outputs of each attention head are concatenated and then subjected to a linear transformation to obtain the final output.

5. The method according to claim 1, characterized in that, The multi-level graph fusion network has a fixed layer dimension setting: the first graph convolutional layer maps the node feature dimension to 64 dimensions; the graph Transformer layer maintains 64-dimensional features; The second graph convolutional layer maps the node feature dimensions to 32 dimensions and serves as the final node fusion representation.

6. The method according to claim 1, characterized in that, The classifier is a LightGBM gradient boosting decision tree model; the interaction prediction step specifically includes: splicing the 32-dimensional fusion representations of the target nodes to form a 64-dimensional feature vector, inputting it into the trained LightGBM classifier, and outputting a prediction probability value between 0 and 1; when the prediction probability value is greater than the set classification threshold, it is determined that there is an interaction, otherwise it is determined that there is no interaction.

7. The method according to claim 1, characterized in that, The method further includes a step of jointly training the multi-level graph fusion network and the classifier, wherein: The training objective is to minimize the binary cross-entropy loss between the predicted result and the true label; A five-fold cross-validation strategy was used to divide the training and validation sets to evaluate the model's generalization performance. Use the Adam optimizer to optimize hyperparameters to obtain the optimal combination of learning rate and number of training epochs.

8. The method according to claim 1, characterized in that, Following the graph fusion representation learning step, a model interpretability analysis step is also included: The output node representations of the first graph convolutional layer, the graph Transformer layer, and the second graph convolutional layer are dimensionality-reduced using the t-distributed random neighborhood embedding algorithm and visualized in a two-dimensional plane to show the distribution and clustering of local structures, global dependencies, and fusion features captured by the model in each layer.

9. A circRNA–miRNA interaction prediction system based on graph fusion networks, characterized in that, include: The sequence data acquisition module is used to acquire the target circRNA sequence and the target miRNA sequence; The semantic feature extraction module is used to encode the circRNA and miRNA sequences using a pre-trained natural language processing model, respectively, to extract the corresponding semantic feature vectors; The semantic feature extraction module includes: a first encoder unit trained based on a masked language model for processing circRNA sequences; and a second encoder unit trained based on a substitution marker detection task for processing miRNA sequences. The interaction network construction module is used to construct an interaction network with circRNA and miRNA as nodes based on the known circRNA-miRNA interaction relationship, and to use the extracted semantic feature vectors as the initial attribute features of the corresponding nodes, thereby forming a network graph with initial node features; The graph fusion network module is used to input the network graph with initial node features into a multi-level graph fusion network. The multi-level graph fusion network first uses the initial node features as input, and aggregates the first-order neighbor information of each node through a first graph convolutional layer to generate a first-layer node representation containing local topology. Using the first-layer node representation as input, it performs multi-head self-attention calculation for all nodes in the graph through a graph Transformer layer to generate a second-layer node representation containing global dependencies. Using the second-layer node representation as input, it performs feature fusion and refinement through a second graph convolutional layer, outputting the final node fusion representation. The interaction prediction module is used to calculate the probability that there is an interaction between the circRNA and the miRNA by a classifier based on the final node fusion representation corresponding to the circRNA and the miRNA, and make a prediction based on the probability.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the circRNA–miRNA interaction prediction method based on graph fusion networks as described in any one of claims 1 to 8.

Images