MLP-Mixer-based ncRNA and protein interaction relevance prediction method

Abstract

The invention provides a method for predicting the interaction relevance between ncRNA (non-coding ribonucleic acid) and protein based on MLP-Mixer. The method comprises the following steps: firstly, coding a protein / ncRNA sequence by combining a k-mer method; inputting the sequence features into a hybrid network combined with an MLP-Mixer network so as to extract the sequence features; and finally, simulating interaction between nucleotide and amino acid through a double-sequence comparison attention module, and outputting the interaction to a classification part for classification. According to the method, the first-level sequence features, the second-level structure sequence features and the third-level structure features are comprehensively considered, a multi-layer perceptron mixer module is introduced and is combined with the convolutional neural network and the bidirectional long-short-term memory network, and therefore the long-distance dependency between the features recognized by the CNN can be captured. Meanwhile, complex interaction between nucleotide and amino acid is simulated through a double-sequence comparison attention module, so that fine correlation between ncRNA and a protein sequence is explored.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and specifically relates to a method for predicting the association between ncRNA and protein based on MLP-Mixer. Background Art

[0002] The interaction between RNA and protein plays a key role in physiological and pathological processes, involving multiple levels of regulation such as RNA synthesis, modification, transport, translation, and degradation. The regulatory function of RNA largely depends on its binding proteins (RBPs). Many proteins, such as RNA polymerase, transcription factors, and regulatory proteins, need to bind to RNA to exert their biological functions.

[0003] In the genome sequencing study conducted in 2001, it was found that only about 2% of RNA is responsible for encoding proteins, while up to 98% of RNA was identified as not participating in protein coding. This part of RNA is called non-coding RNA (ncRNAs for short). Research shows that the interaction between proteins and ncRNAs is an important link in gene expression regulation and various biological processes. Although ncRNAs do not directly encode proteins, they play a key role at the post-transcriptional level through interaction with proteins. This interaction can regulate the activity and stability of proteins, thereby affecting various biological processes within cells. These processes include key life activities such as translation, RNA splicing, chromatin remodeling, and gene regulation. Therefore, accurately predicting the interaction between ncRNAs and proteins is crucial for understanding the regulatory functions of ncRNAs, the pathogenesis of related diseases, and the development of new drugs and treatment methods.

[0004] After entering the 21st century, to reveal the molecular mechanism and structural basis of the binding between ncRNA and protein, a variety of high-throughput experimental techniques and structure analysis techniques have been developed. These high-throughput techniques, such as RIP-Chip, HITS-CLIP, PAR-CLIP, etc., can simultaneously detect the interactions between a large number of ncRNAs and proteins, and determine their binding sites and interaction strengths through steps such as cross-linking, immunoprecipitation, and sequencing. And the structure analysis techniques include methods such as X-ray crystallography, nuclear magnetic resonance, and cryo-electron microscopy. Although these techniques have provided valuable data and information for studying the binding between ncRNA and protein, there are still challenges such as difficult sample preparation, experimental condition limitations, and data noise and bias. Therefore, it is very meaningful to develop effective computational methods to predict the binding between ncRNA and protein.

[0005] With the rapid development of artificial intelligence in the 21st century, computational methods have made remarkable progress in predicting the interactions between ncRNAs and proteins. These methods can be roughly divided into two categories. One category is based on sequence features as input, and the other uses structural features as input. For methods based on sequence feature input, for example, IPMiner uses a Stacked Autoencoder to mine hidden features from the sequence composition features of protein and ncRNA sequences, and then inputs the learned hidden features into a random forest model for prediction; RPISeq predicts ncRPI only based on the sequences of ncRNAs and proteins, using two classifiers, support vector machine (SVM) and random forest (RF); CFRP extracts traditional k-mer features from ncRNA and protein pairs as basic features and constructs complex features through different types of non-linear transformations to predict ncRPI. Compared with sequence feature-based methods, structure feature-based ncRPI prediction methods provide a new perspective for the study of ncRNA-protein interactions. This method can provide richer data by capturing the interaction patterns of molecules in three-dimensional space. Therefore, it performs well in both prediction accuracy and generalization ability, mainly because structural features can provide details that sequence information cannot cover. In view of this, structure-based computational prediction methods have been deeply studied and applied. catRAPID calculates the secondary structure properties of protein and ncRNA sequences and calculates the binding propensity by combining physicochemical features such as hydrogen bonds, hydrophobicity, and van der Waals forces; RPI-Pred combines the sequence information and structural information of ncRNAs and proteins and uses a support vector machine model to predict whether protein-ncRNA pairs interact; LPI-BLS uses a Broad Learning System and inputs the results of multiple independent broad learning systems into a stacked ensemble classifier to predict the interaction between lncRNA and protein.

[0006] In recent years, deep learning, a subfield of artificial intelligence, has had a significant impact on the research of predicting ncRPI. BGFE represents protein sequences using a position-specific scoring matrix (PSSM), represents ncRNA sequences using a k-mer sparse matrix, and uses a stacked autoencoder (SAE) network for ncRPI prediction; RPITER improves the combined triple feature (CTF) encoding method and combines CNN and SAE to learn the k-mer features of RNA and protein sequences; EDLMFC combines a convolutional neural network to learn the dominant biological information and a Bi-LSTM network to capture the long-range dependencies between the features identified by CNN. These advanced methods not only improve the accuracy and efficiency of ncRPI prediction but also provide extremely valuable tools for research work in the fields of bioinformatics and systems biology.

[0007] Although existing methods have achieved remarkable results in predicting the interactions between ncRNAs and proteins, there is still room for improvement in prediction performance. Due to its fixed structure, traditional convolutional neural networks are difficult to capture long-range interactions between sequences. In contrast, the attention mechanism is better at dealing with long-range dependencies in protein sequences and ncRNA sequences. In addition, existing research methods often ignore the possible subtle associations between ncRNA and protein sequences and are not sufficient in extracting features from protein and ncRNA sequences. Summary of the Invention

[0008] In view of the defects that the current biological experimental methods for identifying the interactions between ncRNAs and proteins (ncRPI) are not only costly but also time-consuming, and the existing methods do not consider the subtle associations between ncRNA and protein sequences. In addition, due to the complex molecular interaction mechanism between ncRNAs and proteins and the limitations in the scale and diversity of the available datasets. Therefore, a method for predicting the association of ncRNA and protein interactions based on MLP-Mixer is provided. By using the Multi-Layer Perceptron Mixer (MLP-Mixer) network to predict ncRPI, it can accurately predict the interactions between ncRNAs and proteins.

[0009] To achieve the above object, the technical solutions adopted by the present invention are as follows:

[0010] A method for predicting the interaction correlation between ncRNA and protein based on MLP-Mixer. First, the protein / ncRNA sequence is encoded by the combined k-mer method; then, it is input into a hybrid network incorporating the MLP-Mixer network to extract sequence features; finally, the interaction between nucleotides and amino acids is simulated through a double-sequence comparison attention module, and the output is sent to the classification part for classification.

[0011] Furthermore, there are two independent CNN Layer modules in the hybrid network, which are respectively for ncRNA and protein data; each CNN Layer module consists of a one-dimensional convolutional layer (Conv1D), a max pooling layer (Maxpooling), and a batch normalization layer (Batch Normalization). As the convolutional filter moves along the protein sequence or ncRNA sequence, they can capture different combinations of amino acids or nucleotides, thereby generating a potential vector containing biological information.

[0012] Furthermore, the MLP-Mixer network is mainly composed of a Per-patch fully connected layer and Mixer Block; the Per-patch fully connected layer is responsible for generating a dense representation of the input features; during the processing, first, the input is converted into a feature matrix of a specific size, and then the Per-patch fully connected layer is used to expand the channel dimension of each patch to 128; subsequently, features are further extracted through the Mixer Block; the structure of the Mixer Block includes a fully connected layer, a global average pooling (GAP) layer, two MLP-Mixer blocks, and two layer normalization (LN) layers; the Mixer Block is defined as:

[0013] U S =X S +W 3 σ(W 2 LayerNorm(W 1 X S )) (1)

[0014]

[0015] In the formula, represents the input of the Mixer Block, and W 1 , W 2 , W 3 , W 4 , W 5 are the weight matrices of the fully connected layer; σ(·) represents the GELU activation function; LayerNorm is the layer normalization layer; and They are the outputs of the channel mixing perceptron and the spatial mixing perceptron respectively.

[0016] Furthermore, the dual-sequence comparison attention module is implemented by creating an attention matrix that represents the interaction between the two sequences in space and channel; then, the model calculates the average value of this matrix and applies the sigmoid function to obtain the final attention scores, which are then used to weight the original sequences to highlight the parts that the model deems important; the specific steps are as follows:

[0017] For the output R S and P L of the Mixer Block, they are converted into attention vectors Ra S and Pa L through a multi-layer perceptron;

[0018] Ra S = σ(W R ·R S + b) (3)

[0019] Pa L = σ(W P ·P L + b) (4)

[0020] In the formula, σ(·) is the non-linear activation function ReLU; and are weight matrices; b is the bias vector;

[0021] Then, an attention matrix A S,L representing the interaction between the two sequences in space and channel is created:

[0022] A S,L = σ(W A ·(Ra S + Pa L ) + b) (5)

[0023] In the formula, is the weight matrix;

[0024] After completing the above steps, the attention matrix is successfully constructed; subsequently, an average operation is performed on each dimension to obtain the ncRNA sequence attention matrix and the protein sequence attention matrix

[0025] A R = α(Mean(A, 2)) (6)

[0026] A P= α(Mean(A, 1)) (7)

[0027] Where α is the Sigmoid activation function used to map the attention score to the range (0, 1); Mean(A, axis) is used to obtain the average value of each row of A in dimension axis;

[0028] Subsequently, based on the ncRNA sequence attention matrix and the protein sequence attention matrix the feature matrices R A and P A are obtained:

[0029] R A = R S · 0.6 + R S e A R (8)

[0030] P A = P L · 0.6 + P L e A P (9)

[0031] Where e represents element-wise multiplication; R S and P L represent the outputs of the Mixer Block;

[0032] Finally, the feature matrices R A and P A are concatenated and fed into the classifier.

[0033] Furthermore, the steps of the ncRNA and protein interaction correlation prediction method based on MLP-Mixer are as follows:

[0034] First, the ncRNA, protein sequence features and their derived features, the high-order features extracted by the CNN layer, are fed into the Bi-LSTM network;

[0035] Then, the output of the network is unfolded and reshaped to an appropriate dimension to adapt to the input of the Mixer Layer, and the deeper feature representations of the protein sequence and ncRNA sequence are further refined through the MLP-Mixer network;

[0036] Subsequently, the complex interaction between nucleotides and amino acids is simulated in the Attention Moudule to discover the subtle associations between ncRNA and protein sequences, the corresponding attention scores are calculated, these attention scores are used to weight the original feature representations, and the weighted sum feature matrices are concatenated;

[0037] Finally, the concatenated feature representation is input into a classifier, which consists of two fully connected layers and a Softmax function. These layers jointly process the merged high-level features and finally generate a binary value as the basis for determining whether the objects interact with each other.

[0038] The present invention proposes a deep learning framework model (named RPI-HNBM) that integrates MLP-Mixer, CNN, Bi-LSTM, and a dual-sequence comparison attention module. This model not only utilizes the primary sequence features of ncRNAs and proteins but also integrates secondary and tertiary structure features. In addition, to effectively extract deep network features and accurately identify ncRNA-protein interaction pairs, the present invention uses MLP-Mixer to capture the long-range dependencies between protein and ncRNA sequences. Compared with the prior art, the beneficial effects of the present invention are as follows:

[0039] (1) The present invention first uses MLP-Mixer to capture the long-range dependencies between protein and ncRNA sequences extracted by CNN. This is the first effective application of MLP-Mixer in extracting protein and ncRNA sequence features in the field of ncRPI prediction.

[0040] (2) The present invention comprehensively considers primary sequence features, secondary structure sequence features, and tertiary structure features, and extracts the sequence features of proteins and ncRNAs through the combined k-mer method. In the architecture design of the model, RPI-HNBM introduces a multi-layer perceptron mixer (MLP-Mixer) module and combines it with a convolutional neural network (CNN) and a bidirectional long short-term memory network (Bi-LSTM), so as to capture the long-range dependencies between the features identified by CNN. At the same time, the present invention simulates the complex interactions between nucleotides and amino acids through a dual-sequence comparison attention module to discover the subtle associations between ncRNA and protein sequences.

[0041] (3) The present invention evaluates RPI-HNBM on three public datasets. The experimental results show that compared with other advanced prediction methods, RPI-HNBM exhibits excellent performance on most evaluation metrics, thereby demonstrating the effectiveness of the RPI-HNBM method in ncRPI prediction. Brief Description of the Drawings

[0042] Figure 1This is an overview of the RPI-HNBM model. The protein and non-coding RNA sequence features extracted by the CNN Layer capture the long-range dependencies of the input features from both the forward and backward directions through the Bi-LSTM network. After the output is unfolded, it is reshaped into an appropriate dimension to extract the protein sequence and non-coding RNA sequence features through the MLP-Mixer network, and the sequence comparison is performed through the dual-sequence comparison attention mechanism to calculate the attention scores. Subsequently, these attention scores are used to weight the original feature representations. Finally, the high-level features obtained after weighted summation are fused and then output to the fully connected layer for binary prediction using the Softmax activation function in the last layer.

[0043] Figure 2 This is the specific structure of the Mixer Layer module. Each Mixer Layer module consists of two FC layers, one GELU activation function, and two Dropout layers.

[0044] Figure 3 This is the comparison of the average performance results of each method in 5-fold cross-validation on the RPI1807 dataset.

[0045] Figure 4 This is the comparison of the average performance results of each method in 5-fold cross-validation on the NPInterv2.0 dataset.

[0046] Figure 5 This is the comparison of the average performance results of each method in 5-fold cross-validation on the RPI488 dataset. Specific implementation manner

[0047] In order to enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of this application. Obviously, the described embodiments are only a part of the embodiments of this application, rather than all the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of this application.

[0048] For a given ncRNA sequence L and the corresponding protein sequence S, the main task of the present invention is to develop an efficient mapping function F(L,S). The purpose of this function is to map these two sequences into an abstract space defined by a binary label Y, where "1" indicates the presence of an interaction between the two sequences, and "0" indicates the absence of an interaction. In view of this, the present invention proposes a method for predicting the correlation between ncRNA and protein interactions based on MLP-Mixer. First, the protein / ncRNA sequences are encoded by the combined k-mer method; then, they are input into a hybrid network incorporating an MLP-Mixer network to extract sequence features; finally, the interaction between nucleotides and amino acids is simulated through a double-sequence comparison attention module, and the output is sent to the classification section for classification.

[0049] Example 1

[0050] The present invention uses a total of three publicly available datasets to evaluate the performance of the method, namely RPI1807, NPInter v2.0, and RPI488. The RPI1807 dataset was created based on information in the Nucleic Acid Database (NDB) and the Protein-RNA Interface Database (PRIDB). By parsing these databases, a non-redundant RPI composite training dataset was constructed. It contains 1807 positive pairs (consisting of 1807 proteins and 1078 RNA strands) and 1436 negative pairs (containing 1436 proteins and 493 RNA strands).

[0051] The RPI488 dataset (Fan X N, Zhang S W. LPI-BLS: Predicting lncRNA–protein interactions with abroad learning system-based stacked ensemble classifier[J]. Neurocomputing, 2019, 370: 88-93.) was extracted from the Protein Data Bank (PDB). It consists of interaction pairs between ncRNAs and proteins with lengths exceeding 200. This dataset covers 488 ncRNAs-protein pairs, including 243 positive interaction pairs and 245 negative interaction pairs. Among them, the construction of the RPI1807 and RPI488 datasets follows the minimum atomic spacing criterion. This standard stipulates that if the distance between any pair of atoms between a protein and an RNA is below a given threshold, then the protein-RNA pair is classified as an interaction pair.

[0052] NPInter v2.0 is derived from the literature (Wang J, Zhao Y, Gong W, et al. EDLMFC: an ensemble deep learning framework with multi-scale features combination for ncRNA–protein interaction prediction[J]. BMC bioinformatics, 2021, 22: 1-19.) and extracted from the NPInter dataset. The NPInter dataset mainly covers experimentally verified physical interactions, and its judgment criterion does not rely on the measurement of the distance between atoms in the RNA-protein complex.

[0053] Given the shortage of negative samples in the NPInter dataset, referring to the method in the literature (Pan X, Fan Y X, Yan J, et al. IPMiner: hidden ncRNA-protein interaction sequential pattern mining with stacked autoencoder for accurate computational prediction[J]. BMC genomics, 2016, 17: 1-14.), by randomly combining and pairing ncRNAs and proteins in the positive samples, an equal number of non-interacting pairs were artificially constructed as negative samples in this way. The detailed dataset data is shown in Table 1.

[0054] Table 1 Statistics of ncRNA and protein sequence data on three public datasets

[0055]

[0056] The present invention uses the SPOT-RNA method to predict the secondary structure of ncRNAs. Since the SPOT-RNA method has a limitation on the sequence length, with a maximum input length of 500 nucleotides, all ncRNA sequences longer than 500 are excluded. The information of the preprocessed dataset is shown in Table 2. In addition, since the data distribution of the dataset RPI488 becomes overly unbalanced after excluding some ncRNAs, the method in the literature (Wang J, Zhao Y, Gong W, et al. EDLMFC: an ensemble deep learning framework with multi-scale features combination for ncRNA–protein interaction prediction[J]. BMC bioinformatics, 2021, 22: 1-19.) is used to divide it into 5 subsets, and the average value of their results is taken as the final result.

[0057] Table 2 Statistics of ncRNA and protein sequence data on the preprocessed dataset

[0058]

[0059] In the present invention, a total of four groups of features are screened for the training of the deep learning model, and these features are all derived from protein sequences and ncRNA sequences. The present invention will next elaborate on these feature sets in detail.

[0060] 1 Original ncRNAs sequences and protein sequences

[0061] For ncRNAs, the original ncRNAs sequences include their nucleotide arrangement order, which is the most basic biological information unit; for proteins, the amino acid sequence of the protein determines its linear structure, from which functional regions can be identified. The present invention uses a combined k-mer encoding method to represent the amino acid types in protein sequences and the ribonucleotide types in ncRNAs sequences.

[0062] 2 ncRNAs and protein secondary structures

[0063] In the present invention, the secondary structure information of ncRNAs is predicted using the SPOT-RNA method. In this method, the secondary structure of ncRNAs can be described by seven different structure types, each type represented by a specific character, specifically including: stem (S), hairpin loop (H), multi-loop (M), internal loop (I), bulge (B), external loop (X), and end (E).

[0064] In addition, the present invention obtains the secondary structure of proteins through the SPIDER3 method. Given that the secondary structure of protein sequences can be represented as three structural types: α-helix, β-sheet, and coil, it is represented using identifiers composed of three single characters.

[0065] 3 Protein Tertiary Structure

[0066] For the prediction of protein tertiary structure, the present invention also uses the SPIDER3 method to obtain it, specifically including: solvent accessible surface area (ASA), contact number (CN), upper hemisphere exposure (HSEα-up), and lower hemisphere exposure (HSEα-down).

[0067] 4 Interface Propensity

[0068] Interface propensity (IP) is the occurrence probability of specific residues at the protein-nucleic acid interface. The present invention uses the method of the literature (Li C H, Cao L B, Su J G, et al. A new residue-nucleotide propensity potential with structural information considered for discriminating protein-RNA docking decoys [J]. Proteins: Structure, Function, and Bioinformatics, 2012, 80(1): 14-24.) to obtain residue-nucleotide propensity.

[0069] To meet the requirements of deep learning models, ncRNAs and protein sequences must be converted into numerical vector forms. This step is crucial for processing biological sequence data because it allows algorithms to identify and learn the patterns and features of sequences.

[0070] The present invention uses the combined k-mer encoding method to extract the features of ncRNAs and protein sequences, that is, the frequency information from 1-mer to 3-mer is combined to represent the primary sequence and secondary structure information of proteins; the frequency information from 1-mer to 4-mer is combined to represent the primary sequence and secondary structure information of ncRNAs.

[0071] In protein classification, based on properties such as dipole moment and side-chain volume, the 20 standard amino acids can be classified into 7 groups: Group G1 (Alanine (A), Glycine (G), Valine (V)); Group G2 (Isoleucine (I), Leucine (L), Phenylalanine (F), Proline (P)); Group G3 (Tyrosine (Y), Methionine (M), Threonine (T), Serine (S)); Group G4 (Histidine (H), Asparagine (N), Glutamine (Q), Tryptophan (W)); Group G5 (Arginine (R), Lysine (K)); Group G6 (Aspartic acid (D), Glutamic acid (E)); Group G7 (Cysteine (C)). Therefore, each protein sequence can be represented by seven different letters. By calculating the frequencies of 3-mer, 2-mer, and 1-mer, the protein sequence can be converted into a digital vector containing 399 elements. At the same time, according to its structural characteristics, the secondary structure of proteins can be divided into three basic types: α-helix, β-sheet, and random coil, and thus can be encoded as a digital vector containing 39 elements.

[0072] For ncRNA, it consists of four ribonucleotides: Adenine (A), Uracil (U), Guanine (G), Cytosine (C). This means that each ncRNA sequence can be represented by four different letters. By calculating the frequency distributions of 4-mer, 3-mer, 2-mer, and 1-mer, the ncRNA sequence can be converted into a digital vector containing 340 elements. At the same time, the secondary structure of ncRNA can be subdivided into seven types (X, E, S, I, H, B, M), so its secondary structure information can be described by a digital vector with 2800 elements.

[0073] Figure 1 Figure

[15] shows an overview of the RPI-HNBM model, which includes modules such as CNN Layer, Bi-LSTM, MLP-Mixer, and Attention Moudule.

[0074] First, the ncRNA, protein sequence features, and their derived features, as well as the high-order features extracted by the CNN layer, are fed into the Bi-LSTM network. This network can capture the long-distance dependencies of the input features from both the forward and backward perspectives.

[0075] Then, the output of the network is unfolded and reshaped to an appropriate dimension to fit the input of the Mixer Layer, and the MLP-Mixer network is used to further extract deeper feature representations of protein sequences and ncRNA sequences.

[0076] Subsequently, in the Attention Moudule, the complex interactions between nucleotides and amino acids are simulated to discover the subtle associations between ncRNA and protein sequences, the corresponding attention scores are calculated, these attention scores are used to weight the original feature representations, and the feature matrices after weighted summation are concatenated.

[0077] Finally, the present invention inputs the concatenated feature representations into a classifier, which consists of two fully connected layers and a Softmax function. These layers jointly process the combined high-level features and finally generate a binary value as the basis for judging whether the objects interact with each other.

[0078] The structures of each module of the model are described as follows:

[0079] 1 CNN layer

[0080] In the model design of the present invention, there are two independent CNN Layer modules for ncRNA and protein data respectively. Each CNN Layer module consists of a one-dimensional convolutional layer (Conv1D), a max-pooling layer (Maxpooling), and a batch normalization layer (Batch Normalization). These layers are used to extract semantic information from sequence data. Among them, the one-dimensional convolutional layer can identify and capture local features in the entire sequence. As the convolutional filter moves on the protein sequence or ncRNA sequence, it can capture different combinations of amino acids or nucleotides, thereby generating a potential vector containing biological information. The max-pooling layer is used to focus on the most important parts, and the batch normalization layer provides a regularization effect, effectively improving the training efficiency and generalization ability of the model.

[0081] 2 MLP-Mixer network

[0082] To deeply explore the characteristics of ncRNA and protein sequences and make up for the deficiency of CNN in capturing long-range dependencies, the present invention introduces the MLP-Mixer network structure. The feature input of this network is provided by a multi-layer network composed of a CNN Layer and a Bi-LSTM network. The MLP-Mixer network mainly consists of a Per-patch fully connected layer and Mixer Blocks. The Per-patch fully connected layer is responsible for generating a dense representation of the input features. During the processing, first, the input is converted into a feature matrix of a specific size, and then the Per-patch fully connected layer expands the channel dimension of each patch to 128. Subsequently, features are further extracted through Mixer Blocks. The structure of the Mixer Block includes a fully connected layer, a global average pooling (GAP) layer, two MLP-Mixer blocks, and two layer normalization (LN) layers. The specific structure of the Mixer Layer is as shown in Figure 2 shown, and the Mixer Block can be defined as:

[0083] U S = X S + W 3 σ(W 2 LayerNorm(W 1 X S )) (1)

[0084]

[0085] In the formula, represents the input of the Mixer Block, and W 1 , W 2 , W 3 , W 4 , W 5 are the weight matrices of the fully connected layers; σ(·) represents the GELU activation function; LayerNorm is the layer normalization layer; and are the outputs of the channel mixing perceptron and the spatial mixing perceptron respectively.

[0086] 3 Dual-sequence comparison attention module

[0087] To effectively discover the subtle associations between ncRNA and protein sequences and simulate the complex interactions between nucleotides and amino acids, the present invention introduces a special attention mechanism, namely, a dual-sequence comparison attention module. This mechanism is achieved by creating an attention matrix that represents the spatial and channel-wise interactions between the two sequences. Then, the model calculates the average value of this matrix and applies the sigmoid function to obtain the final attention scores, which are subsequently used to weight the original sequences to highlight the parts that the model deems important.

[0088] More specifically, for the outputs R S and P L of the Mixer Block, the present invention converts them into attention vectors Ra S and Pa L through a multi-layer perceptron.

[0089] Ra S = σ(W R ·R S + b) (3)

[0090] Pa L = σ(W P ·P L + b) (4)

[0091] In the formula, σ(·) is the non-linear activation function ReLU; and are weight matrices; b is the bias vector.

[0092] Then, an attention matrix A S,L representing the spatial and channel-wise interactions between the two sequences is created:

[0093] A S,L = σ(W A ·(Ra S + Pa L ) + b) (5)

[0094] In the formula, is the weight matrix.

[0095] After completing the above steps, the attention matrix is successfully constructed. Subsequently, the present invention performs an average operation on each dimension to obtain the ncRNA sequence attention matrix and the protein sequence attention matrix

[0096] A R = α(Mean(A, 2)) (6)

[0097] AP = α(Mean(A, 1)) (7)

[0098] Where α is the Sigmoid activation function used to map the attention score to the range (0, 1); Mean(A, axis) is used to obtain the average value of each row of A in dimension axis.

[0099] Subsequently, based on the ncRNA sequence attention matrix and the protein sequence attention matrix the feature matrices R A and P A are obtained as follows:[[]]

[0100] R A = R S · 0.6 + R S e A R (8)

[0101] P A = P L · 0.6 + P L e A P (9)

[0102] Where e represents element-wise multiplication; R S and P L represent the outputs of the Mixer Block.

[0103] Finally, the feature matrices R A and P A are concatenated and fed into the classifier.

[0104] Example 2

[0105] Model evaluation metrics and experiments:

[0106] In the present invention, sample pairs with interactions are classified as positive samples, while sample pairs without interactions are regarded as negative samples. To more comprehensively compare the performance of the model RPI-HNBM proposed in the present invention with other advanced methods in ncRPI prediction, the present invention adopts seven evaluation metrics: ACC, TPR, TNR, PPV, MCC, F1, and AUC. These metrics together construct a comprehensive evaluation system aimed at comprehensively evaluating and reflecting the prediction performance of the model.

[0107]

[0108] Wherein, TP (true positive) is the number of interacting pairs correctly identified by the model; FP (false positive) is the number of pairs wrongly predicted as interacting pairs by the model; TN (true negative) is the number of non-interacting pairs correctly identified by the model; FN (false negative) is the number of pairs wrongly predicted as non-interacting pairs by the model.

[0109] In addition, ACC (accuracy) represents the proportion of all samples correctly classified by the classifier and is a comprehensive performance metric. TPR (true positive rate) indicates the proportion of samples actually positive identified by the classifier. TNR (true negative rate) reflects the proportion of samples actually negative identified by the classifier. PPV (precision) shows the proportion of actually positive samples among those predicted as positive. MCC (Matthews correlation coefficient) is a comprehensive performance metric considering class imbalance. F1 (F1 score) is the harmonic mean of PPV and TPR and is an indicator reflecting its balance. AUC (area under the curve) is used to measure the ability of the classifier to rank positive classes ahead of negative classes. These metrics jointly measure the prediction accuracy and reliability of the model.

[0110] In the present invention, GeForce RTX 3070 is used as the experimental device, and a deep learning model based on the TensorFlow 2.6 framework is constructed, combined with the CUDA 11.3 acceleration library and Python 3.8 to complete the experiment. During the model training process, the present invention selects Adaptive Moment Estimation (Adam) and Stochastic Gradient Descent (SGD) as the main optimization algorithms.

[0111] In the configuration of the deep learning architecture, the present invention sets the batch size to 128. Such a setting not only ensures sufficient sample size for effective learning but also maintains reasonable utilization of computing resources. To prevent overfitting, when implementing the MLP-Mixer structure, the linear mapping dimension of the per-patch fully connected layer is set to 128, and the Dropout rate is set to 0.2 to enhance the generalization ability of the model.

[0112] Experimental results:

[0113] 1 Comparison with other advanced methods

[0114] To comprehensively evaluate the performance of the RPI-HNBM model, the present invention selects four advanced methods, namely EDLMFC, RPITER, IPMiner, and CFRP, as comparison objects. Five-fold cross-validation is performed on three public datasets, namely RPI1807, NPInter v2.0, and RPI488, to obtain more robust evaluation results. To comprehensively evaluate the performance of these methods from different perspectives, seven performance indicators are adopted: ACC (accuracy), TPR (true positive rate), TNR (true negative rate), PPV (precision), MCC (Matthews correlation coefficient), F1 (F1 score), and AUC (area under the curve). These indicators can comprehensively reflect the performance of the model in classification problems.

[0115] Figures 3 - 5 The experimental results on the RPI1807, NPInter v2.0, and RPI488 datasets are shown, and the specific numerical results are detailed in Table 3-5. Table 4 shows the comparison results between the method RPI-HNBM of the present invention and other advanced research methods on the NPInter v2.0 dataset, highlighting the contribution of the present invention in the field of ncRNA-binding protein prediction.

[0116] As Figure 4As shown, on the NPInter v2.0 dataset, RPI-HNBM outperforms all methods in seven performance metrics. At the same time, as can be seen from Table 4, the ACC of RPI-HNBM is 93.2%, which is 3.5%, 4.2%, and 10.4% higher than that of EDLMFC, RPITER, IPMiner, and CFRP, respectively. The accuracy (ACC) of RPI-HNBM is 93.2%, which is 11.1% higher than that of CFRP, 10.4% higher than that of IPMinter, 4.2% higher than that of RPITER, and 3.5% higher than that of EDLMFC; the true positive rate (TPR) of RPI-HNBM is 95.3%, which is 18.1% higher than that of CFRP, 11.0% higher than that of IPMinter, 3.7% higher than that of RPITER, and 3.6% higher than that of EDLMFC; the true negative rate (TNR) of RPI-HNBM is 91.1%, which is 4.2% higher than that of CFRP, 9.8% higher than that of IPMinter, 4.9% higher than that of RPITER, and 3.4% higher than that of EDLMFC; the precision (PPV) of RPI-HNBM is 91.5%, which is 6.0% higher than that of CFRP, 10.2% higher than that of IPMinter, 4.5% higher than that of RPITER, and 3.3% higher than that of EDLMFC; the F1 score (F1) of RPI-HNBM is 93.3%, which is 12.2% higher than that of CFRP, 10.1% higher than that of IPMinter, 4.0% higher than that of RPITER, and 3.4% higher than that of EDLMFC; the Matthews correlation coefficient (MCC) of RPI-HNBM is 86.5%, which is 22.1% higher than that of CFRP, 20.9% higher than that of IPMinter, 8.4% higher than that of RPITER, and 7.0% higher than that of EDLMFC; the area under the curve (AUC) of RPI-HNBM is 98.1%, which is 9.7% higher than that of CFRP, 15.4% higher than that of IPMinter, 2.4% higher than that of RPITER, and 2.2% higher than that of EDLMFC.

[0117] Compared with the above four methods, RPI-HNBM proposed by the present invention shows more superior performance in predicting ncRPI on the NPInter v2.0 dataset. As Figure 3 shown, RPI-HNBM achieved the highest ACC, F1 score, and MCC on the RPI1807 dataset. And as Figure 5 shown, except for the two metrics of TNR and PPV, RPI-HNBM is higher than the compared advanced methods in other evaluation metrics on the RPI488 dataset. This indicates that RPI-HNBM based on the MLP-Mixer network and introducing the dual-sequence comparison attention mechanism is an efficient ncRPI prediction method.

[0118] Table 3 Performance comparison results of RPI-HNBM and other advanced methods on the RPI1807 dataset

[0119]

[0120] Table 4 Performance comparison results of RPI-HNBM and other advanced methods on the NPInterv2.0 dataset

[0121]

[0122] Table 5 Performance comparison results of RPI-HNBM and other advanced methods on the RPI488 dataset

[0123]

[0124] 2 Ablation experiments

[0125] 2.1 Influence of the MLP-Mixer network

[0126] In the present invention, the contribution of the MLP-Mixer network to the model performance is verified by removing the MLP-Mixer network. Table 6 shows the comparison experiment results on the NPInter v2.0 dataset. Specifically, in the absence of the MLP-Mixer network, the overall performance of the model decreases. The indicators ACC, TPR, TNR, PPV, MCC, F1, and AUC decrease from 93.2%, 95.3%, 91.1%, 91.5%, 93.3%, 86.5%, and 98.1% to 92.3%, 94.9%, 89.8%, 90.3%, 92.5%, 84.8%, and 96.8% respectively. In addition to the overall performance decrease, the decrease in TNR is more significant, indicating a certain degree of weakening in the model's effectiveness in identifying negative samples.

[0127] Meanwhile, the present invention explores the influence of the per-patch linear mapping layer with different dimensions [128, 256, 512] in the MLP-Mixer network on the embedded features. It can be observed from Table 7 that when the representation dimension of the linear mapping layer is set to 128, all evaluation indicators reach the best level.

[0128] Table 6 Performance comparison results of the model in RPI-HNBM before and after applying the MLP-Mixer network on the NPInterv2.0 dataset

[0129]

[0130] Table 7 Performance comparison results of different dimension linear mapping layers in the MLP-Mixer network applied in RPI-HNBM on the NPInterv2.0 dataset

[0131]

[0132] 2.2 Influence of the Dual-Sequence Comparison Attention Module

[0133] In this study, to verify the importance of the dual-sequence comparison attention module, this module was removed to evaluate its specific impact on the overall model performance. According to the data in Table 8, when the dual-sequence comparison attention module was used, the performance metrics ACC, TPR, TNR, PPV, MCC, F1, and AUC were 93.2%, 95.3%, 91.1%, 91.5%, 93.3%, 86.5%, and 98.1% respectively. However, once this attention module was removed, these metrics immediately dropped to 92.1%, 94.4%, 89.8%, 90.3%, 92.3%, 84.3%, and 96.7%. From these data, it can be seen that the F1 score dropped the most, reaching 2.2%. This indicates that the model's ability to balance TPR and PPV decreased the most significantly, followed by a 1.4% decrease in AUC, indicating that the model's overall ability to distinguish positive and negative samples weakened. This shows that the dual-sequence comparison attention module plays a certain role in improving classification performance.

[0134] In addition, the present invention analyzed the influence of different values [0.4, 0.5, 0.6] of the weighted parts of calculating the feature matrices R A and P A in the dual-sequence comparison attention module on the classification results. As shown in Table 9, it can be concluded that when the weight is set to 0.6, the seven evaluation metrics ACC, TPR, TNR, PPV, MCC, F1, and AUC all reach the optimal level. From the results, it can be seen that setting the weight to 0.6 may provide a better balance in the calculation of the feature matrix, enabling the model to more effectively capture the correlations and differences between sequences.

[0135] Table 8 Comparison results of the model performance of RPI-HNBM on the NPInterv2.0 dataset before and after applying the dual-sequence comparison attention module

[0136]

[0137] Table 9 Comparison results of the performance with different weight values in the dual-sequence comparison attention module of RPI-HNBM on the NPInterv2.0 dataset

[0138]

[0139] In summary, the present invention constructs a hybrid network model based on MLP-Mixer, namely RPI-HNBM. This model integrates the MLP-Mixer network, aiming to explore the long-distance dependence relationship between ncRNA sequences and protein sequences, and introduces a dual-sequence comparison attention mechanism to capture the complex interactions between nucleotides and amino acids, revealing more subtle connections between ncRNA and protein sequences. On three publicly available ncRPI datasets, the model of the present invention shows superior performance compared to other methods.

Claims

1. A method for predicting the association between ncRNA and protein interaction based on MLP-Mixer, characterized in that: First, the protein / ncRNA sequence is encoded by the joint k-mer method; then, it is input into the hybrid network combined with the MLP-Mixer network to extract sequence features; finally, the interaction between nucleotides and amino acids is simulated by the double sequence comparison attention module and output to the classification part for classification.

2. The prediction method according to claim 1, characterized in that: The hybrid network has two independent CNNLayer modules, one for ncRNA and the other for protein data; each CNN Layer module consists of a one-dimensional convolutional layer (Conv1D), a maximum pooling layer (Maxpooling) and a batch normalization layer (Batch Normalization). As the convolutional filters move on the protein sequence or ncRNA sequence, they can capture different amino acid combinations or nucleotide combinations, thereby generating a potential vector containing biological information.

3. The prediction method according to claim 2, characterized in that: The MLP-Mixer network is mainly composed of a Per-patch fully connected layer and a Mixer Block; the Per-patch fully connected layer is responsible for generating a dense representation of the input features; during the processing, the input is first converted into a feature matrix with a specific size, and then the channel dimension of each patch is expanded to 128 using the Per-patch fully connected layer; then, the features are further extracted through the Mixer Block; the structure of the Mixer Block includes a fully connected layer, a global average pooling (GAP) layer, two MLP-Mixer blocks, and two layer normalization (LN) layers; the Mixer Block is defined as: U S =X S +W3σ(W2LayerNorm(W1X S )) (1) In the formula, represents the input of Mixer Block, W1, W2, W3, W4, W5 are the weight matrices of the fully connected layer; σ(·) represents the GELU activation function; LayerNorm is the layer normalization layer; and They are the outputs of the channel mixture perceptron and the spatial mixture perceptron respectively.

4. The prediction method according to claim 3, characterized in that: The two-sequence comparison attention module is implemented by creating an attention matrix that represents the spatial and channel interactions between the two sequences; the model then calculates the average of this matrix and applies the sigmoid function to obtain the final attention scores, which are then used to weight the original sequence to highlight the parts that the model considers important; the specific steps are: For the output R of Mixer Block S and P L , and convert them into attention vectors Ra through a multi-layer perceptron S and Pa L ; Ra S =σ(W R ·R S +b) (3) Bye L =σ(W P ·P L +b) (4) Where σ(·) is the nonlinear activation function ReLU; and is the weight matrix; b is the bias vector; Then, we create an attention matrix A that represents the spatial and channel interactions between the two sequences. S,L : A S,L =σ(W A ·(Ra S +Bye L )+b) (5) In the formula, is the weight matrix; After completing the above steps, the attention matrix was successfully constructed; then, an average operation was performed on each dimension to obtain the ncRNA sequence attention matrix and the protein sequence attention matrix A R =α(Mean(A,2)) (6) A P =α(Mean(A,1)) (7) Where α is the Sigmoid activation function, which is used to map the attention score to the range (0, 1); Mean(A, axis) is used to obtain the average value of each row A in dimension axis; Then, according to the ncRNA sequence attention matrix and the protein sequence attention matrix Get the feature matrix R A and P A : R A =R S ·0.6+R S e A R (8) P A =P L ·0.6+P L e A P (9) In the formula, e represents the multiplication of corresponding elements; R S and P L Represents the output of Mixer Block; Finally, the feature matrix R A and P A are concatenated and fed into the classifier.

5. The prediction method according to claim 4, characterized in that: Here are the steps: First, the high-order features of ncRNA and protein sequence features and their derived features are extracted through CNN layers and then transmitted to the Bi-LSTM network; Then, the output of the network is expanded and reshaped to an appropriate dimension to adapt to the input of the Mixer Layer, and the MLP-Mixer network is used to further extract the deeper feature representation of protein sequences and ncRNA sequences; Then, the complex interactions between nucleotides and amino acids are simulated in the Attention Moudule to explore the subtle associations between ncRNA and protein sequences, and the corresponding attention scores are calculated. These attention scores are used to weight the original feature representations, and the weighted summed feature matrices are concatenated. Finally, the concatenated feature representation is input into the classifier, which consists of two fully connected layers and a Softmax function. These layers jointly process the merged high-level features and finally generate a binary value as the basis for judging whether the objects interact with each other.

Images