Multi-channel attention mechanism lncRNA-miRNA association prediction method

Abstract

The invention discloses an lncRNA-miRNA association prediction method of a multi-channel attention mechanism, and relates to an lncRNA-miRNA association prediction method. The method aims to solve the problem that more effective and robust feature representation may not be learned due to the fact that feature representation of multiple data sources cannot be properly fused and similarity and difference between samples cannot be learned through a contrast learning method in an existing method. According to the method, multi-source data of lncRNA and miRNA is obtained, and multiple similarity matrixes and coding matrixes are obtained through calculation; in combination with a graph convolutional network and a Transform, capturing information from a local angle and a global angle; fusing feature representations of different data sources by using a multi-channel attention mechanism; a contrast learning method is introduced to further optimize feature representation, so that the consistency between different modal features of the same lncRNA or miRNA is ensured, and the discrimination capability of the model is enhanced; in the final prediction stage, KAN is utilized to obtain a prediction score associated with lncRNA-miRNA, and the parameter and calculation efficiency are further improved.

Description

Technical Field

[0001] The present invention belongs to the field of bioinformatics, and in particular relates to a lncRNA-miRNA association prediction method. Background Art

[0002] Non-coding RNAs (ncRNAs) cannot encode proteins, but they play an indispensable role in many life processes. Among them, long non-coding RNAs (lncRNAs) and microRNAs (miRNAs), as two typical ncRNAs, play an important role in cancer development, gene regulation, and cell metabolism. lncRNA is about 200 nucleotides long. As the most abundant RNA in organisms, it is widely involved in various cellular processes such as protein folding and cell differentiation. MiRNA is a small ncRNA with a length of about 22 nucleotides. It indirectly regulates gene expression by attaching to lncRNA, adjusts the function of lncRNA, and cooperates with lncRNA to complete the regulatory process. More and more evidence shows that the association relationship between lncRNA and miRNA (LMIs) helps to discover some potential regulatory relationships. Therefore, studying the potential relationship between lncRNA and miRNA can provide new insights into our understanding of the biological functions of lncRNA and miRNA and the pathogenesis of complex diseases.

[0003] However, the potential association between lncRNA and miRNA discovered through traditional biological experiments is time-consuming, costly, and difficult to label large amounts of data. Therefore, the key to solving this problem is to use computational methods to predict the association between lncRNA and miRNA on a large scale. Previous computational methods can be roughly divided into two categories: network-based methods and sequence-based methods. Network-based methods generally use a variety of information about lncRNA and miRNA to calculate the multiple similarities of lncRNA and miRNA, further construct a bipartite network, and finally use various machine learning models to extract features from the bipartite network and infer the LMIs score. Sequence-based methods only use lncRNA and miRNA sequence information, encode it, and then use deep learning models such as convolutional neural networks to extract deep features and predict the probability of LMIs. Although these methods have been proven to be effective, there are still problems such as slightly low prediction performance due to improper methods of combining multiple data sources and reliance on known LMIs, which limits the generalization ability of the model. Summary of the invention

[0004] The purpose of the present invention is to solve the problem that existing methods fail to properly fuse feature representations of multiple data sources and fail to learn similarities and differences between samples through comparative learning methods, resulting in the possibility of not learning more effective and robust feature representations.

[0005] A lncRNA-miRNA association prediction method based on a multi-channel attention mechanism comprises the following steps:

[0006] S1: Based on the sequence information, expression profile information and lncRNA-miRNA association information of lncRNA and miRNA, the sequence similarity LS, expression profile similarity LE and GIP core similarity LG corresponding to lncRNA and miRNA are obtained, and then three similarity matrices corresponding to lncRNA and miRNA are obtained;

[0007] Fill the lncRNA sequence and miRNA sequence to a fixed length, segment the lncRNA and miRNA sequences using three different step lengths, encode the segmented sequences, and encode the corresponding codes of all segmentation combinations corresponding to each segmentation step length as a whole to obtain a coding matrix. Three coding matrices are obtained for the three segmentation step lengths, and the coding matrices corresponding to the three different step length segmentation structures of the lncRNA sequence and miRNA sequence are obtained respectively;

[0008] S2: Use the P nearest neighbor graph method to process the three similarity matrices corresponding to lncRNA and miRNA, and obtain the corresponding sparse similarity matrices respectively; use GCN to process the sparse similarity matrices respectively, and obtain the feature representation L of lncRNA based on the results of GCN of the three similarity matrices of lncRNA GCN Based on the three similarity matrices of miRNA, the feature representation M of miRNA is obtained by GCN. GCN ;

[0009] S3: Based on the encoding matrix corresponding to the three different length segmentation structures of the lncRNA sequence, the embedding layer and Transformer are used to obtain the features of the lncRNA and construct the feature representation L of the lncRNA trans ; For the encoding matrices corresponding to the three different length segmentation structures of miRNA sequences, the same method is used to obtain the miRNA feature representation M trans ;

[0010] S4: Use the channel attention mechanism to represent the feature L separately GCN And the feature representation L trans Weighted fusion Use the channel attention mechanism to represent the feature M GCN and feature representation M trans Weighted fusion

[0011] S5: Use contrastive learning to capture the information interaction between GCN and Tranformer to obtain the final lncRNA feature representation and final miRNA signature representation

[0012] S6: Representation of lncRNA features obtained in S5 Splice to get L * , miRNA signature expression Splice to get M * , use KAN to get the final prediction score y.

[0013] Furthermore, the sequence similarity LS:

[0014]

[0015] Where i and j represent the i-th and j-th lncRNA or miRNA, respectively. Function L obtains the length of the lncRNA or miRNA sequence. Function lev calculates the Levenshtein distance between sequence i and sequence j.

[0016] Furthermore, the expression profile similarity LE:

[0017]

[0018] Among them, p in and p jn represents the nth attribute value of the i-th and j-th lncRNA or miRNA expression profiles, respectively, N represents the number of related attributes, and p i and p j represent the average values ​​of the expression profiles of the i-th and j-th lncRNA or miRNA, respectively.

[0019] Furthermore, the GIP nuclear similarities are as follows:

[0020] The adjacency matrix of lncRNA-miRNA association data is represented as IM∈R nl×nm , where nl and nm represent the number of lncRNA and miRNA, respectively; the element IM in the adjacency matrix IM ij represents the relationship between the i-th lncRNA and the j-th miRNA. If the value in the matrix is ​​1, it means that the relationship is experimentally confirmed, otherwise the value is 0;

[0021] Using the matrix IM, we can obtain the GIP nuclear similarity LG between lncRNAs or miRNAs:

[0022] LG(i,j)=exp(-λ l||IP(i)-IP(j)|| 2 )

[0023] Where i and j represent the i-th and j-th lncRNA or miRNA, respectively, IP represents the interaction information between each lncRNA / miRNA and all miRNAs / lncRNAs obtained from each row of the adjacency matrix IM; λ l Indicates the bandwidth parameter.

[0024] Furthermore, in the process of filling the lncRNA sequence and miRNA sequence to a fixed length, the sequence is filled by using the copy-fill method:

[0025]

[0026] Where L represents the maximum length of all lncRNA sequences or miRNA sequences, l represents the initial length of the lncRNA sequence or miRNA sequence, and mod represents the remainder; when the remainder is 0, the sequence x1x2x3…x l It is copied an integer number of times to reach the maximum length L. When the remainder is c, the first c bases in the sequence need to be copied after being copied an integer number of times.

[0027] Furthermore, the specific process of obtaining the encoding matrix corresponding to the three different length segmentation structures of the lncRNA sequence and miRNA includes the following steps:

[0028] For the lncRNA sequence filled to a fixed length, the segmentation step size k = 2, 3, and 4 are taken. For the three segmentation step sizes, segmentation combinations with a length less than or equal to k are generated respectively, and each segmentation combination is mapped to a one-hot vector. Then, the one-hot vectors corresponding to all segmentation combinations corresponding to each segmentation step size are encoded as a whole to obtain an encoding matrix. Three encoding matrices are obtained for the three segmentation step sizes. For lncRNA, the three encoding matrices are respectively denoted as L k＝2 , L k＝3 , L k＝4 ;

[0029] For the miRNA sequence filled to a fixed length, the same method is used to obtain the three encoding matrices M of miRNA k＝2 、M k＝3 、M k＝4 .

[0030] Furthermore, the process of using the P nearest neighbor graph method to process the three similarity matrices corresponding to lncRNA and miRNA to obtain the corresponding sparse similarity matrices includes:

[0031] First, the P nearest neighbor graph method is used to process the similarity matrix corresponding to the sequence similarity matrix LS, and the weight matrix G is calculated by the following formula LS :

[0032]

[0033] in, G LS The elements in N P (l i ) and N P (l j ) represent the p nearest neighbor set of the i-th lncRNA and the p nearest neighbor set of the j-th lncRNA, respectively, and p is set to 5 in the present invention;

[0034] According to the weight matrix G LS , the sparse similarity matrix of lncRNA sequence similarity LS*=LS·G is calculated LS ;

[0035] In the same way, we obtained the sparse similarity matrix LE* of lncRNA expression profile similarity and the sparse similarity matrix LG* of GIP core similarity, as well as the sparse similarity matrix MS* of miRNA sequence similarity, the sparse similarity matrix ME* of miRNA expression profile similarity, and the sparse similarity matrix MG* of miRNA GIP core similarity.

[0036] Furthermore, S2 uses GCN to process the sparse similarity matrix respectively, and obtains the feature representation L of lncRNA based on the three similarity matrices of lncRNA through GCN. GCN Based on the three similarity matrices of miRNA, the feature representation M of miRNA is obtained by GCN. GCN The process includes:

[0037] When GCN is used to process LS*, the feature representation of a single node is:

[0038]

[0039] Among them, l i represents the i-th lncRNA, l ij represents the neighbor i of the i-th lncRNA in LS* j ,s i,j represents the similarity weight between the i-th lncRNA and its j-th neighbor in LS*, s i,i represents the similarity weight of the i-th lncRNA in LS*; W i is a learnable parameter that projects features into the latent space, and σ represents a nonlinear activation function;

[0040] The following processing is performed based on the node:

[0041] S'=I+S

[0042] Where I represents the identity matrix; S represents the sparse similarity matrix. For LS*, it represents LS*. For other cases, it corresponds to one of LE*, LG*, MS*, ME*, and MG*.

[0043] Based on a single node, we get the embedding representation of all nodes of lncRNA in the entire graph, and take the result L1 of the lth layer. (l) As the embedding of LS* after GCN processing;

[0044] In the same way, take the result L2 of LE* passing through the lth layer of GCN (l) As the processed embedding, take the result L3 of LG* passing through the lth layer of GCN (l) As the processed embedding, the feature representation L of lncRNA is obtained GCN ={L1 (l) , L2 (l) , L3 (l)};

[0045] For the sparse similarity matrix of miRNA, the result M1 of MS* after GCN is (l) As the processed embedding, ME* passes through GCN result M2 (l) As the processed embedding, MG* passes through GCN result M3 (l) As the processed embedding, the feature representation M of miRNA is obtained. GCN ={M1 (l) , M2 (l) , M3 (l)}.

[0046] Furthermore, the specific process of step S3 includes the following steps:

[0047] S31: The encoding matrix corresponding to the three different length segmentation structures of lncRNA sequence is represented as L k＝2 , L k＝3 , L k＝4 , the encoding matrix corresponding to the three different length segmentation structures of miRNA sequences is represented as M k＝2 、M k＝3 、M k＝4 , after the embedding layer, L k＝2 , L k＝3 , L k＝4 and M k＝2 、M k＝3 、M k＝4 Treat them as the same dimension, and set the number of columns to dmodel;

[0048] S32: Use the Transformer encoder to process the preprocessed matrix in S31:

[0049] First, use sine and cosine functions to implement position embedding. Use sine encoding at even positions and cosine encoding at odd positions. The position encoding equation is:

[0050]

[0051] Among them, pos represents the position index in the sequence, and a represents the dimension index;

[0052] The encoding matrix of lncRNA after embedding layer and position encoding is represented as L′ k＝2 , L′ k＝3 , L′ k＝4 ;

[0053] The encoding matrix of miRNA after embedding layer and position encoding is represented as M′ k＝2 , M′ k＝3 , M′ k＝4 ;

[0054] The encoding result is obtained by using the Transformer encoder, the encoding matrix L′ k＝2 The feature representation L′1 is obtained through the Transformer encoder;

[0055] In the same way, the encoding matrix L′ k＝3 After the Transformer encoder, the feature representation L′2 is obtained, and the encoding matrix L′ k＝4 After the Transformer encoder, the feature representation L′3 is obtained; then the feature representation of lncRNA is obtained as L trans ={L'1, L'2, L'3};

[0056] For the miRNA coding matrix, we get M′ k＝2 After the feature representation M′1 of the Transformer encoder, we get M′ k＝3 After the feature representation M′2 of the Transformer encoder, we get M′ k＝4 The feature representation of the Transformer encoder is M′3; the feature representation of miRNA is then obtained as M trans ={M'1, M'2, M'3}.

[0057] Furthermore, the specific process of S4 includes the following steps:

[0058] For L GCN ={L1 (l), L2 (l) , L3 (l)}, L1 (l) 、L2 (l) and L3 (l) Each is considered as a channel;

[0059] For the first channel L1 (l) , using the attention mechanism to calculate the attention weight of the channel

[0060]

[0061] Among them, F l ×dmodel is L1 (l) The size of It is L1 (l) The value at position (m, n) in the middle; δ represents the sigmoid activation function, σ represents the relu activation function, and W1 and W2 are training parameters;

[0062] L2 (l) and L3 (l) The same process is performed, and the corresponding Then we get the attention weight of the channel The feature tensor L GCN ={L1 (l) , L2 (l) , L3 (l)} and attention weight Corresponding multiplication, we get the weighted feature representation {l1 * , l2 * , l3 *}; The feature representation of lncRNA weighted fusion is obtained through convolution operation

[0063]

[0064] Among them, conv means applying convolution operation to three channels;

[0065] Using the same method, we get M GCN Feature representation after weighted fusion and L trans Feature representation after weighted fusion M trans Feature representation after weighted fusion

[0066] The beneficial effects of the present invention are:

[0067] The present invention proposes to use a multi-channel attention mechanism to fuse the feature representations of different data sources, assign higher weight values ​​to matrices with greater contributions, and effectively extract features from different data sources. The impact of data quality on model performance can be reduced, while improving the transparency and interpretability of feature representations, partially alleviating the problem of poor feature interpretability. The present invention comprehensively utilizes GCN and transformer, takes advantage of their efficient computing capabilities, and introduces KAN in the final prediction stage, achieving higher parameter efficiency and computing efficiency, thereby reducing the overall computational complexity.

[0068] In addition, the present invention proposes to use a contrastive learning method, which can effectively learn the similarities and differences between samples, enhance the discrimination and generalization capabilities of the model, and reduce dependence on known lncRNA-miRNA associations.

[0069] Therefore, the present invention can effectively learn more effective and robust feature representations, thereby improving prediction performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 This is a flowchart of lncRNA-miRNA association prediction based on a multi-channel attention mechanism;

[0071] Figure 2-Figure 4 It is a comparison diagram of ROC curves of the five-fold cross validation method of the present invention and the related lncRNA-miRNA association prediction model on three data sets (data set 1-data set 3);

[0072] Figure 5-Figure 7 It is a comparison diagram of the PR curves of the five-fold cross-validation method of the present invention and the related lncRNA-miRNA association prediction model on three data sets (data set 1-data set 3). DETAILED DESCRIPTION

[0073] In order to more clearly describe the present invention and introduce the purpose and technology of the present invention, the present invention will be further described below in combination with relevant data, drawings and examples. In particular, the description here is a part of the specific examples of the present invention, rather than limiting the scope of the invention.

[0074] Specific implementation method 1: Combination Figure 1 To explain this embodiment,

[0075] This embodiment is a lncRNA-miRNA association prediction method using a multi-channel attention mechanism, comprising the following steps:

[0076] S1: Collect and organize the sequence information, expression profile information and prior lncRNA-miRNA association information of lncRNA and miRNA; calculate the sequence similarity, expression profile similarity and GIP nuclear similarity of lncRNA and miRNA respectively based on the above data; fill the lncRNA sequence and miRNA sequence to a fixed length, and in order to capture information under different fields of view, use a step size of 2, a step size of 3, and a step size of 4 to segment the lncRNA and miRNA sequences, encode the segmented sequences, and obtain three encoding matrices of lncRNA sequences and miRNA respectively.

[0077] In this embodiment, the data to be obtained are as follows:

[0078] S11: lncRNA and miRNA sequence information was collected and sorted through https: / / lncipedia.org / and http: / / www.mirbase.org / index.shtm. Taking lncRNA as an example, the sequence similarity LS between lncRNAs was calculated using the following formula:

[0079]

[0080] Among them, i and j represent the i-th lncRNA and the j-th lncRNA respectively, the function L obtains the length of the lncRNA sequence, and the function lev calculates the Levenshtein distance between sequence i and sequence j, which represents the minimum number of single-character editing operations (insertion, deletion or replacement) required to convert one string into another string, and the number of operations is used as the score of the two sequences. The value of LS ranges from 0 to 1, and the closer the result is to 1, the more similar the two sequences are.

[0081] Similarly, the sequence similarity MS between miRNAs was calculated by the above method.

[0082] S12: The expression profile information of lncRNA and miRNA was collected and sorted through http: / / www.noncode.org / and http: / / www.microrna.org / m icrorna / home.do / . Taking lncRNA as an example, the expression profile similarity LE between lncRNAs was calculated by the following formula:

[0083]

[0084] Among them, p in and p jnThey represent the nth attribute value of the ith lncRNA expression profile and the nth attribute value of the jth lncRNA expression profile, respectively. N represents the number of related attributes, and p i and p j They represent the average value of the expression profile of the i-th lncRNA and the average value of the expression profile of the j-th lncRNA, respectively. The value of LE ranges from 0 to 1. The closer the result is to 1, the more similar the expression profiles of the two lncRNAs are. Similarly, the expression profile similarity ME between miRNAs is calculated by the above method.

[0085] S13: Collect and organize prior lncRNA-miRNA association data through http: / / bioinfo.life.hust.edu.cn / lncRNASNP#! / and define the adjacency matrix IM∈R nl×nm , where nl and nm represent the number of lncRNA and miRNA, respectively. The elements IM in the adjacency matrix IM ij It represents the relationship between the i-th lncRNA and the j-th miRNA. If the value in the matrix is ​​1, it means that the relationship is experimentally confirmed, otherwise the value is 0.

[0086] Using the matrix IM, taking lncRNA as an example, the GIP nuclear similarity LG between lncRNAs is calculated by the following formula:

[0087] LG(i,j)=exp(-λ l ||IP(i)-IP(j)|| 2 )

[0088] Where i and j represent the i-th lncRNA and j-th lncRNA, respectively, IP represents the interaction information between each lncRNA and all miRNAs obtained from each row of the adjacency matrix IM, and λ l represents the bandwidth parameter, which is calculated as:

[0089]

[0090] in, The kernel similarity bandwidth used to regularize the Gaussian interaction properties is set to 1.

[0091] Similarly, the GIP nuclear similarity MG between miRNAs was calculated by the above method.

[0092] S14: Collect and organize lncRNA and miRNA sequence information, and use the copy and fill method to fill the sequence. Taking lncRNA as an example, the sequence is filled by the following method:

[0093]

[0094] Where L represents the maximum length of all lncRNA sequences, l represents the initial length of the lncRNA sequence, and mod represents the remainder. When the remainder is 0, the sequence x1x2x3…x l It is copied an integer number of times to reach the maximum length L. When the remainder is c, the first c bases in the sequence need to be copied after being copied an integer number of times.

[0095] Similarly, miRNA sequences are padded to the maximum length M among all miRNA sequences.

[0096] S15: Split the padded sequence in S14, and treat each of the split subsequences as a split combination. Take the split step length k = 2, 3, 4, and generate split combinations with a length less than or equal to k for the three split step lengths, respectively, map each split combination to a one-hot vector, and then encode the one-hot vectors corresponding to all the split combinations corresponding to each split step length as a whole to obtain a coding matrix. Three coding matrices are obtained for the three split step lengths. Taking lncRNA as an example, the three coding matrices are denoted as L k＝2 , L k＝3 , L k＝4 .

[0097] The segmentation method corresponding to the segmentation step length: for example, if the step length is 2, all possible segmentation combinations are A, G, C, T (not an integer multiple, a single segmentation result is left after segmentation), and AA, AG, AC, AT, GA, GG, GC, GT, CA, CG, CC, CT, TA, TG, TC, TT, a total of 20 (i.e. 4+42); similarly, the segmentation combinations with a step length of 3 are 84 (i.e. 4+42+4 3 ), the number of combinations with a step length of 3 is 340 (i.e. 4+42+4 3 +4 4 ).

[0098] Similarly, the three encoding matrices M of miRNA are obtained k＝2 、M k＝3 、M k＝4 .

[0099] S2: Use the P nearest neighbor graph method to process the three similarity matrices in S1, so that the lncRNAs (or miRNAs) close to each other are close enough in the lncRNA space (or miRNA space) to obtain a sparse similarity matrix; use GCN to process the sparse similarity matrix to obtain the feature representation of lncRNA and the feature representation of miRNA; the specific process includes:

[0100] S21: The similarity matrices obtained in S11, S12, and S13 were processed using the P nearest neighbor graph method to construct affinity graphs LS*, LE*, and LG* of lncRNA and affinity graphs MS*, ME*, and MG* of miRNA.

[0101] Taking the sequence similarity matrix LS of lncRNA as an example, the weight matrix G is defined by the following formula: LS :

[0102]

[0103] in, G LS The elements in N P (l i ) and N P (l j ) represent the p nearest neighbor set of the i-th lncRNA and the p nearest neighbor set of the j-th lncRNA, respectively, and p is set to 5 in the present invention.

[0104] According to the weight matrix G LS , the sparse similarity matrix of lncRNA sequence similarity LS*=LS·G is calculated LS .

[0105] Similarly, according to the weight matrix G LE , the sparse similarity matrix of lncRNA expression profile similarity is calculated as LE*=LE·G LE , according to the weight matrix G LG , the sparse similarity matrix of lncRNA GIP nuclear similarity LG*=LG·G LE .

[0106] Similarly, the above method can be used to obtain the sparse similarity matrix MS* of miRNA sequence similarity, the sparse similarity matrix ME* of miRNA expression profile similarity, and the sparse similarity matrix MG* of miRNA GIP core similarity.

[0107] S22: Use GCN to process the sparse similarity matrices LS*, LE*, LG* of lncRNA and the sparse similarity matrices MS*, ME*, MG* of miRNA obtained in S21 respectively.

[0108] Taking the sparse similarity matrix LS* of lncRNA sequence similarity as an example, the feature representation of a single node is:

[0109]

[0110] Among them, l i represents the i-th lncRNA, represents the neighbor i of the i-th lncRNA in LS* j ,s i,j represents the similarity weight between the i-th lncRNA and its j-th neighbor in LS*, s i,i represents the similarity weight of the i-th lncRNA in LS*; W i is a learnable parameter that projects features into the latent space, and σ represents a nonlinear activation function.

[0111] In order to ensure the characteristics of the node itself, the node self-loop is also added, which is expressed as:

[0112] S'=I+S

[0113] Among them, I represents the unit matrix; S represents the sparse similarity matrix. Taking LS* as an example, it is LS*. When it is for other cases, it corresponds to one of LE*, LG*, MS*, ME*, and MG*.

[0114] After obtaining the propagation rule of a single node using GCN, the embedding of all nodes of lncRNA on the entire graph is expressed as:

[0115]

[0116] in, represents the node embedding of lncRNA obtained by the lth layer GCN on LS*, and the node embedding of the 0th layer is randomly initialized; D1 represents a diagonal matrix, D1(i,i)=∑ j S′(i,j).

[0117] Take the result of layer l, L1 (l) As the embedding of LS* after GCN processing;

[0118] In the same way, take the result L2 of the lth layer (l) As the embedding of LE* after GCN processing, take the result L3 of the lth layer (l) As the embedding of LG* after GCN processing, the feature tensor of lncRNA can be expressed as L GCN ={L1 (l) , L2 (l) , L3 (l)}.

[0119] Similarly, for the sparse similarity matrix of miRNA, take the result M1 of the lth layer (l) As the embedding of MS* after GCN processing, M2 (l) As the embedding of ME* after GCN processing, M3 (l)As the embedding of MG* after GCN processing, the feature tensor of miRNA can be expressed as M GCN ={M1 (l) , M2 (l) , M3 (l)}.

[0120] S3: Coding matrix L based on lncRNA in S15 k＝2 , L k＝3 , L k＝4 and the coding matrix M of miRNA k＝2 、M k＝3 、M k＝4 , using the embedding layer and Transformer to obtain the feature representation L of lncRNA trans ={L'1, L'2, L'3} and miRNA feature representation M trans ={M'1, M'2, M'3}. The specific process includes:

[0121] S31: After the embedding layer, the size of the lncRNA encoding matrix is ​​processed to the same dimension, and the number of columns is set to dmodel. The size of the miRNA encoding matrix is ​​processed to the same dimension, and the number of columns is set to dmodel.

[0122] S32: Use the Transformer encoder to process the preprocessed matrix in S31:

[0123] First, use sine and cosine functions to implement position embedding. Use sine encoding at even positions and cosine encoding at odd positions. The position encoding equation is:

[0124]

[0125] Among them, pos represents the position index in the sequence, and a represents the dimension index.

[0126] The encoding matrix of lncRNA after embedding layer and position encoding is represented as L′ k＝2 , L′ k＝3 , L′ k＝4 .

[0127] Similarly, the encoding matrix of miRNA after embedding layer and position encoding is represented as M′ k＝2 , M′ k＝3 , M′ k＝4 .

[0128] The encoder of the Transformer after position encoding is a stack of N identical layers, each of which consists of two modules: multi-head attention and feedforward network. Residual connection and layer normalization are used for each module.k＝2 For example, multi-head attention divides the scaled dot product attention into n heads, and each head is expressed as follows:

[0129]

[0130] MultiHeadAttention(L' k＝2 )=concat(h1,h2,···,h n )W

[0131] Among them, 1≤i≤n, n represents the number of heads, Q, K, V are the transformations of input X, d K is the characteristic dimension of K. h i represents the output of the i-th head, MultiHeadAttention(L′ k＝2 ) represents the encoding matrix L′ k＝2 The overall output of the multi-head attention. Next, use residual connections and layer normalization:

[0132] L” k＝2 =layerNorm(L' k＝2 +MultiHeadAttention(L' k＝2 ))

[0133] The feedforward network is a two-layer neural network, expressed as follows:

[0134] FFN(L” k＝2 )=ReLu(0,L” k＝2 W1+b1)W2+b2

[0135] Among them, w1 and w2 are weight matrices, b1 and b2 are bias terms. Next, use residual connections and layer normalization:

[0136] L'1 = layerNorm(L" k＝2 +FFN(L” k＝2 ))

[0137] L′1 is the encoding matrix L′ k＝2 Feature representation obtained through the Transformer encoder;

[0138] In the same way, the encoding matrix L′ k＝3 After the Transformer encoder, the feature representation L′2 is obtained, and the encoding matrix L′ k＝4 After the Transformer encoder, the feature representation L′3 is obtained, and the feature representation of lncRNA is L trans ={L'1, L'2, L'3}.

[0139] Similarly, for the coding matrix of miRNA, we get M′ k＝2 After the feature representation M′1 of the Transformer encoder, we get M′ k＝3 After the feature representation M′2 of the Transformer encoder, we get M′ k＝4 The feature representation after the Transformer encoder is M′3, and the feature representation of miRNA is M trans ={M'1, M'2, M'3}.

[0140] S4: Use the channel attention mechanism to represent the features obtained in S2 L GCN ={L1 (l) , L2 (l) , L3 (l)}、M GCN ={M1 (l) , M2 (l) , M3 (l)} and S3 get the feature representation L trans ={L'1, L'2, L'3}, M trans = {M'1, M'2, M'3} are weighted fused to obtain the feature representation of lncRNA after weighted fusion and And the feature representation after miRNA weighted fusion and

[0141] The feature tensor L of lncRNA obtained by GCN GCN ={L1 (l) , L2 (l) , L3 (l)} as an example, where L1 (l) 、L2 (l) and L3 (l) Each is considered as a channel. By compressing the feature tensor L of lncRNA GCN To generate channel statistics, for example, the first channel L1 (l) It is expressed as:

[0142]

[0143] Among them, F l ×dmodel is L1 (l) The size of It is L1 (l) The value at position (m, n) in the .

[0144] Use the attention mechanism to calculate the attention weight of the channel Channel 1 L1 (l) The attention weight is:

[0145]

[0146] Among them, δ represents the sigmoid activation function, σ represents the relu activation function, and W1 and W2 are training parameters.

[0147] L2 (l) and L3 (l) The same process is performed, and the corresponding The attention weight of the channel calculated by the attention mechanism

[0148] Feature tensor L GCN ={L1 (l) , L2 (l) , L3 (l)} and attention weight Corresponding multiplication, we get the weighted feature representation {l1 * , l2 * , l3 *}. Take the first channel L1 (l) For example, the calculation is:

[0149]

[0150] Through the convolution operation, the weighted three channel information is integrated together to obtain the weighted fusion feature representation of lncRNA. The formula is as follows:

[0151]

[0152] Among them, conv means applying a convolution operation to three channels.

[0153] Similarly, through the above steps, we get M GCN Feature representation after weighted fusion and L trans Feature representation after weighted fusion M trans Feature representation after weighted fusion

[0154] S5: Use contrastive learning to capture the information interaction between GCN and Tranformer, balance the information embedding between different modalities, and obtain the final lncRNA feature representation after contrastive learning of the weighted feature information obtained in S4. and final miRNA signature representation The loss function of this method is as follows:

[0155]

[0156] Where T represents the temperature coefficient, which is used to adjust the similarity scale in contrastive learning; i represents the sample, which represents lncRNA / miRNA; I represents the set of all samples; z i represents the feature vector of sample i. In this embodiment, it is the feature vector of sample i P(i) represents the positive sample set of sample i, including samples related to sample i, |P(i)| represents the size of P(i); z p represents the feature vector of the positive sample, which is the feature vector of sample i in this embodiment p is the identifier of the element in the reference set P(i). N(i) represents the negative sample set of sample i, which contains samples that are irrelevant to sample i; n represents the feature vector of negative samples. In this embodiment, it is the feature vector of all samples except sample i. The feature vectors of all samples except sample i n is an identifier that refers to an element in the set N(i).

[0157] S6: Representation of lncRNA features obtained in S5 Splice to get L * , miRNA signature expression Splice to get M * , use KAN to get the final prediction score y, calculated using the following formula:

[0158] y=σ(KAN(concat(L * ,M * )))

[0159] Where σ represents the sigmoid activation function. The KAN layer is specifically expressed as:

[0160] KAN=W base ·σ(x)+W spline ·B(x)

[0161] Where σ represents the SiLU activation function, B(x) represents the B-spline function, and W base and W spline Represent the base weight and spline weight respectively.

[0162] S7: Conduct comparative experiments and ablation experiments on the dataset, collect and organize the experimental results; use AUC (area under the ROC curve), AUPR (area under the PR curve), accuracy (ACC), recall rate (Recall), precision (Precision) and F1 score (F1-score) indicators to analyze the experimental results from multiple angles to evaluate the performance of the new method; analyze typical cases and evaluate the practical effect of the new method. Some evaluation indicators are calculated as follows:

[0163]

[0164] Among them, TP represents the number of correctly predicted lncRNA-miRNA associations, FP represents the number of incorrectly predicted lncRNA-miRNA associations, TN represents the number of correctly predicted non-lncRNA-miRNA associations, and FN represents the number of incorrectly predicted non-lncRNA-miRNA associations.

[0165] The recognition effect of the present invention is further described below with a group of specific experimental examples.

[0166] In order to verify the predictive performance of the invention, the present invention designs three data sets: Data set 1 includes 7500 positive samples and 7500 negative samples, that is, the ratio of positive and negative samples is 1:1; Data set 2 includes 1500 positive samples and 7500 negative samples, that is, the ratio of positive and negative samples is 1:5; Data set 3 includes 750 positive samples and 7500 negative samples, that is, the ratio of positive and negative samples is 1:10. The performance of the present invention and PmliPred, PmliPEMG, preMLI, PmliHFM, LncMirNet and RNAI-FRID related models on three data sets was compared using a five-fold cross-validation method. As shown in Tables 1, 2 and 3, the performance of the models on Data set 1 is better than that of other models. Although some indicators on Data sets 2 and 3 are slightly inferior to other models, the overall performance is better than other models, and the following is plotted. Figures 2 to 7 The ROC curve and PR curve shown in the figure indicate that the present invention can effectively predict the lncRNA-miRNA association relationship.

[0167] In order to verify the effectiveness of each component of the model, the present invention conducts ablation experiments without using sequence similarity, expression spectrum similarity, GIP similarity, encoding matrix with k=2, encoding matrix with k=3, encoding matrix with k=4, GCN, transformer, G channel attention fusion GCN processing results (replaced with summing and averaging), channel attention fusion transformer processing results (replaced with summing and averaging), contrastive learning, and KAN (replaced with MLP). The results are shown in Table 4, which shows that each component contributes to the improvement of model performance to varying degrees.

[0168] In order to verify the application effect of the present invention in actual cases, a specific lncRNA associated object prediction experiment and a specific miRNA associated object prediction experiment were performed respectively. As shown in Table 5, the present invention was used to predict the top 20 miRNAs with potential association with NONHSAT137542.2, of which 18 were verified. As shown in Table 6, the present invention was used to predict the top 20 lncRNAs with potential association with hsa-miR-590-3p, of which 14 were verified. The above results show that the present invention has good practical application value.

[0169] Table 1 Performance index results of the present invention and other related models in five-fold cross validation on dataset 1

[0170]

[0171] Table 2 Performance index results of the present invention and other related models in five-fold cross validation on dataset 2

[0172]

[0173] Table 3 Performance index results of the present invention and other related models in five-fold cross validation on dataset 3

[0174]

[0175] Table 4 Performance comparison of the present invention under different settings

[0176]

[0177] Table 5 Validation of the top 20 miRNAs predicted by the present invention to be associated with NONHSAT137542.2

[0178]

[0179] Table 6 Validation of the top 20 lncRNAs predicted by the present invention to be associated with hsa-miR-590-3p

[0180]

[0181] The detailed description of the embodiments of the present invention provided above is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present invention.

Claims

1. A lncRNA-miRNA association prediction method based on a multi-channel attention mechanism, characterized in that: The following steps are involved: S1: Based on the sequence information, expression profile information and lncRNA-miRNA association information of lncRNA and miRNA; The sequence similarity LS, expression profile similarity LE and GIP nuclear similarity LG corresponding to lncRNA and miRNA were obtained respectively, and then three similarity matrices corresponding to lncRNA and miRNA were obtained respectively; Fill the lncRNA sequence and miRNA sequence to a fixed length, segment the lncRNA and miRNA sequences using three different step lengths, encode the segmented sequences, and encode the corresponding codes of all segmentation combinations corresponding to each segmentation step length as a whole to obtain a coding matrix. Three coding matrices are obtained for the three segmentation step lengths, and the coding matrices corresponding to the three different step length segmentation structures of the lncRNA sequence and miRNA sequence are obtained respectively; S2: Use the P nearest neighbor graph method to process the three similarity matrices corresponding to lncRNA and miRNA, and obtain the corresponding sparse similarity matrices respectively; use GCN to process the sparse similarity matrices respectively, and obtain the feature representation L of lncRNA based on the results of GCN of the three similarity matrices of lncRNA GCN Based on the three similarity matrices of miRNA, the feature representation M of miRNA is obtained by GCN. GCN ; S3: Based on the encoding matrix corresponding to the three different length segmentation structures of the lncRNA sequence, the embedding layer and Transformer are used to obtain the features of the lncRNA and construct the feature representation L of the lncRNA trans ; For the encoding matrices corresponding to the three different length segmentation structures of miRNA sequences, the same method is used to obtain the miRNA feature representation M trans ; S4: Use the channel attention mechanism to represent the features L separately GCN And the feature representation L trans Weighted fusion Use the channel attention mechanism to represent the feature M GCN and feature representation M trans Weighted fusion S5: Use contrastive learning to capture the information interaction between GCN and Tranformer to obtain the final lncRNA feature representation and final miRNA signature representation S6: Representation of lncRNA features obtained in S5 Splice to get L * , miRNA signature expression Splice to get M * , use KAN to get the final prediction score y.

2. The lncRNA-miRNA association prediction method based on a multi-channel attention mechanism according to claim 1, characterized in that: The sequence similarity LS: Where i and j represent the i-th and j-th lncRNA or miRNA, respectively. Function L obtains the length of the lncRNA or miRNA sequence. Function lev calculates the Levenshtein distance between sequence i and sequence j.

3. The lncRNA-miRNA association prediction method based on a multi-channel attention mechanism according to claim 1, characterized in that: The expression profile similarity LE: Among them, p in and p jn represents the nth attribute value of the i-th and j-th lncRNA or miRNA expression profiles, respectively, N represents the number of related attributes, and p i and p j represent the average values ​​of the expression profiles of the i-th and j-th lncRNA or miRNA, respectively.

4. The lncRNA-miRNA association prediction method based on a multi-channel attention mechanism according to claim 1, characterized in that: The GIP nuclear similarities are as follows: The adjacency matrix of lncRNA-miRNA association data is represented as IM∈R nl×nm , where nl and nm represent the number of lncRNA and miRNA, respectively; the element IM in the adjacency matrix IM ij represents the relationship between the i-th lncRNA and the j-th miRNA. If the value in the matrix is ​​1, it means that the relationship is experimentally confirmed, otherwise the value is 0; Using the matrix IM, we can obtain the GIP nuclear similarity LG between lncRNAs or miRNAs: LG(i,j)=exp(-λ l ||IP(i)-IP(j)|| 2 ) Where i and j represent the i-th and j-th lncRNA or miRNA, respectively, IP represents the interaction information between each lncRNA / miRNA and all miRNAs / lncRNAs obtained from each row of the adjacency matrix IM; λ l Indicates the bandwidth parameter.

5. The lncRNA-miRNA association prediction method based on a multi-channel attention mechanism according to claim 1, characterized in that: In the process of filling lncRNA sequences and miRNA sequences to a fixed length, the copy-fill method is used to fill the sequences: Where L represents the maximum length of all lncRNA sequences or miRNA sequences, l represents the initial length of the lncRNA sequence or miRNA sequence, and mod represents the remainder; when the remainder is 0, the sequence x1x2x3…x l It is copied an integer number of times to reach the maximum length L. When the remainder is c, the first c bases in the sequence need to be copied after being copied an integer number of times.

6. The lncRNA-miRNA association prediction method of a multi-channel attention mechanism according to any one of claims 1 to 5, characterized in that: The specific process of obtaining the encoding matrix corresponding to the three different length segmentation structures of lncRNA sequences and miRNAs includes the following steps: For the lncRNA sequence filled to a fixed length, the segmentation step size k = 2, 3, and 4 are taken. For the three segmentation step sizes, segmentation combinations with a length less than or equal to k are generated respectively, and each segmentation combination is mapped to a one-hot vector. Then, the one-hot vectors corresponding to all segmentation combinations corresponding to each segmentation step size are encoded as a whole to obtain an encoding matrix. Three encoding matrices are obtained for the three segmentation step sizes. For lncRNA, the three encoding matrices are respectively denoted as L k＝2 , L k＝3 , L k＝4 ; For the miRNA sequence filled to a fixed length, the same method is used to obtain the three encoding matrices M of miRNA k＝2 、M k＝3 、M k＝4 .

7. The lncRNA-miRNA association prediction method of a multi-channel attention mechanism according to claim 6, characterized in that: The process of using the P nearest neighbor graph method to process the three similarity matrices corresponding to lncRNA and miRNA to obtain the corresponding sparse similarity matrices includes: First, the P nearest neighbor graph method is used to process the similarity matrix corresponding to the sequence similarity matrix LS, and the weight matrix G is calculated by the following formula LS : in, G LS The elements in N P (l i ) and N P (l j ) represent the p nearest neighbor set of the i-th lncRNA and the p nearest neighbor set of the j-th lncRNA, respectively, and p is set to 5 in the present invention; According to the weight matrix G LS , the sparse similarity matrix of lncRNA sequence similarity LS*=LS·G is calculated LS ; In the same way, we obtained the sparse similarity matrix LE* of lncRNA expression profile similarity and the sparse similarity matrix LG* of GIP core similarity, as well as the sparse similarity matrix MS* of miRNA sequence similarity, the sparse similarity matrix ME* of miRNA expression profile similarity, and the sparse similarity matrix MG* of miRNA GIP core similarity.

8. The lncRNA-miRNA association prediction method of a multi-channel attention mechanism according to claim 7, characterized in that: In S2, GCN is used to process the sparse similarity matrix, and the feature representation L of lncRNA is obtained based on the results of GCN of the three similarity matrices of lncRNA. GCN Based on the three similarity matrices of miRNA, the feature representation M of miRNA is obtained by GCN. GCN The process includes: When GCN is used to process LS*, the feature representation of a single node is: Among them, l i represents the i-th lncRNA, represents the neighbor i of the i-th lncRNA in LS* j ,s i,j represents the similarity weight between the i-th lncRNA and its j-th neighbor in LS*, s i,i represents the similarity weight of the i-th lncRNA in LS*; W i is a learnable parameter that projects features into the latent space, and σ represents a nonlinear activation function; The following processing is performed based on the node: S'=I+S Where I represents the identity matrix; S represents the sparse similarity matrix. For LS*, it represents LS*. For other cases, it corresponds to one of LE*, LG*, MS*, ME*, and MG*. Based on a single node, we get the embedding representation of all nodes of lncRNA in the entire graph, and take the result L1 of the lth layer. (l) As the embedding of LS* after GCN processing; In the same way, take the result L2 of LE* passing through the lth layer of GCN (l) As the processed embedding, take the result L3 of LG* passing through the lth layer of GCN (l) As the processed embedding, the feature representation L of lncRNA is obtained GCN ={L1 (l) , L2 (l) , L3 (l) }; For the sparse similarity matrix of miRNA, the result M1 of MS* after GCN is (l) As the processed embedding, ME* passes through GCN result M2 (l) As the processed embedding, MG* passes through GCN result M3 (l) As the processed embedding, the miRNA feature representation M is obtained. GCN ={M1 (l) , M2 (l) , M3 (l) }.

9. The lncRNA-miRNA association prediction method of a multi-channel attention mechanism according to claim 8, characterized in that: The specific process of step S3 includes the following steps: S31: The encoding matrix corresponding to the three different length segmentation structures of lncRNA sequence is represented as L k＝2 , L k＝3 , L k＝4 , the encoding matrix corresponding to the three different length segmentation structures of miRNA sequences is represented as M k＝2 、M k＝3 、M k＝4 , after the embedding layer, L k＝2 , L k＝3 , L k＝4 and M k＝2 、M k＝3 、M k＝4 Treat them as the same dimension, and set the number of columns to dmodel; S32: Use the Transformer encoder to process the preprocessed matrix in S31: First, use sine and cosine functions to implement position embedding. Use sine encoding at even positions and cosine encoding at odd positions. The position encoding equation is: Among them, pos represents the position index in the sequence, and a represents the dimension index; The encoding matrix of lncRNA after embedding layer and position encoding is represented as L′ k＝2 , L′ k＝3 , L′ k＝4 ; The encoding matrix of miRNA after embedding layer and position encoding is represented as M′ k＝2 , M′ k＝3 , M′ k＝4 ; The encoding result is obtained by using the Transformer encoder, the encoding matrix L′ k＝2 After the Transformer encoder, the feature representation L′1 is obtained; In the same way, the encoding matrix L′ k＝3 After the Transformer encoder, the feature representation L′2 is obtained, and the encoding matrix L′ k＝4 After the Transformer encoder, the feature representation L′3 is obtained; then the feature representation of lncRNA is obtained as L trans ={L'1, L'2, L'3}; For the miRNA coding matrix, we get M′ k＝2 After the feature representation M′1 of the Transformer encoder, we get M′ k＝3 After the feature representation M′2 of the Transformer encoder, we get M′ k＝4 The feature representation of the Transformer encoder is M′3; the feature representation of miRNA is then obtained as M trans ={M'1, M'2, M'3}.

10. The lncRNA-miRNA association prediction method of a multi-channel attention mechanism according to claim 9, characterized in that: The specific process of S4 includes the following steps: For L GCN ={L1 (l) , L2 (l) , L3 (l) }, L1 (l) 、L2 (l) and L3 (l) Each is considered as a channel; For the first channel L1 (l) , using the attention mechanism to calculate the attention weight of the channel Among them, F l ×dmodel is L1 (l) The size of It is L1 (l) The value at position (m, n) in the middle; δ represents the sigmoid activation function, σ represents the relu activation function, and W1 and W2 are training parameters; L2 (l) and L3 (l) The same process is performed, and the corresponding Then we get the attention weight of the channel The feature tensor L GVN ={L1 (l) , L2 (l) , L3 (l) } and attention weight Corresponding multiplication, we get the weighted feature representation {l1 * , l2 * , l3 * }; The feature representation of lncRNA weighted fusion is obtained through convolution operation Among them, conv means applying convolution operation to three channels; Using the same method, we get M GCN Feature representation after weighted fusion and L trans Feature representation after weighted fusion M trans Feature representation after weighted fusion

Images