A prediction method and prediction system for gene regulatory network

Abstract

The present invention provides a prediction method and prediction system for gene regulatory networks, relating to the technical field of quantitative prediction. The method comprises: obtaining transcription factor-target pair data as first data, performing data augmentation processing on the first data to obtain second data; generating pseudo-labels for the first and second data based on a predetermined task, and inputting the pseudo-labeled first and second data into a neural network model for training iterations to obtain a feature matrix including transcription factor-target pairs; increasing the weights of corresponding transcription factor-target pairs in the feature matrix based on transcription factor-target pairs with regulatory relationships in the gene regulatory network data; and determining whether the transcription factor-target pairs have a regulatory relationship based on the fine-tuned feature matrix to obtain a prediction result for the gene regulatory network. The present invention achieves accurate prediction of gene regulatory networks based on unlabeled data and small sample learning.

Description

Technical Field

[0001] The present invention relates to the field of quantitative prediction technology, and in particular to a prediction method and prediction system for a gene regulatory network. Background Art

[0002] With the development of science and technology, existing researchers have found that the study of gene regulatory networks plays a very important role in improving stress resistance and disease resistance, obtaining high-yield plants, etc. in crop breeding. Among them, the gene regulatory network refers to the complex network structure of the interactions between genes and the gene regulatory relationship, involving complex regulatory mechanisms within cells and between tissues. In the existing technology, gene regulatory networks are usually predicted through traditional supervised learning. In the prediction process, a large amount of labeled gene regulatory network data is often required. However, due to the high knowledge threshold and less data of gene regulatory networks, the cost of labeling gene regulatory network data is very high. There is less labeled gene regulatory network data, which cannot meet the requirements of traditional supervised learning for a large amount of labeled gene regulatory network data, resulting in low accuracy of traditional supervised learning in predicting gene regulatory networks. Summary of the Invention

[0003] In view of this, the purpose of the present invention is to provide a prediction method and prediction system for gene regulatory networks, which can solve the technical problem in the prior art that traditional supervised learning has low accuracy when predicting gene regulatory networks without a large amount of labeled data.

[0004] In order to achieve the above objectives, the technical solutions adopted in the embodiments of the present invention are as follows:

[0005] In a first aspect, an embodiment of the present invention provides a method for predicting a gene regulatory network, comprising: obtaining transcription factor-target pair data as first data, performing data enhancement processing on the transcription factor-target pair data, and obtaining processed transcription factor-target pair data as second data; generating pseudo labels for the transcription factors and the target targets based on the assumption of correlation between the first data and the second data based on a predetermined task, and inputting the first data and the second data after generating the pseudo labels into a neural network model for training iteration to obtain a feature matrix including transcription factor-target target pairs; wherein the feature matrix reflects whether each transcription factor-target target pair has a regulatory relationship; obtaining gene regulatory network data, and identifying transcription factor-target target pairs with a regulatory relationship therein, and increasing the weight of the corresponding transcription factor-target target pair in the feature matrix based on the transcription factor-target target pairs with a regulatory relationship; judging whether the transcription factor-target target pair has a regulatory relationship based on the feature matrix after adjusting the weight, and obtaining a prediction result of the gene regulatory network.

[0006] Optionally, the step of fine-tuning the feature matrix based on the transcription factor-target pairs with a regulatory relationship includes: constructing a simple linear encoder based on the transcription factor-target pairs with a regulatory relationship and binary cross entropy for training, labeling the transcription factor-target pairs with a regulatory relationship in the feature matrix, and increasing the weight of the transcription factor-target pairs in the feature matrix.

[0007] Optionally, the predetermined task is: assuming that each transcription factor in the first data and the second data is only correlated with a target target in each transcription factor-target pair, but has no correlation with other target targets.

[0008] Optionally, the step of inputting the first data and the second data after generating pseudo labels into a neural network model for training iteration to obtain a feature matrix including transcription factor-target target pairs includes: setting a neural network model, dividing the first data and the second data after generating pseudo labels into d non-overlapping grids of length s, respectively, and inputting them into the embedding layer of the neural network model for dimensionality increase processing; inputting the first data and the second data after dimensionality increase processing into the encoder of the neural network model respectively to obtain a first embedding sequence and a second embedding sequence; inputting the first embedding sequence and the second embedding sequence into the projection layer of the neural network model for dimensionality increase processing to obtain a first dimensionality increase sequence and a second dimensionality increase sequence, and performing gradient update on the first dimensionality increase sequence and the second dimensionality increase sequence to obtain a first gradient sequence and a second gradient sequence; inputting the first gradient sequence and the second gradient sequence into the prediction layer of the neural network model to obtain a first output sequence and a second output sequence, and obtaining a loss function based on the first dimensionality increase sequence, the second dimensionality increase sequence, the first output sequence and the second output sequence; performing multiple iterations to minimize the loss function to obtain the feature matrix of the transcription factor-target target pairs.

[0009] Optionally, the step of inputting the first data and the second data into the embedding layer of the neural network model for dimensionality increase processing includes: inputting the first data and the second data, which are respectively divided into d non-overlapping grids of length s, into the embedding layer, performing linear projection, and performing dimensionality increase processing through an embedding weight matrix; wherein the expression of the embedding weight matrix is:

[0010]

[0011] In the expression of the embedding weight matrix: E represents the embedding weight matrix, is a set of real numbers, s represents the length of the grid, g represents the number of genes, and h represents the number of hidden layers in the embedding layer.

[0012] Optionally, the step of inputting the first data and the second data after dimensionality increase processing into the encoder of the neural network model to obtain the first embedding sequence and the second embedding sequence respectively includes: inputting the grid segments of the first data and the second data after dimensionality increase processing into the respective encoder blocks in the encoder of the neural network model; based on the multi-head attention mechanism in each encoder block, the grid segments input into each encoder block are subjected to attention weight learning; the grid segments in each encoder block after attention weight learning are input into the feedforward neural network for gradient update; the grid segments in each encoder block after gradient update are added with residual connections and normalized to obtain the first embedding sequence and the second embedding sequence.

[0013] Optionally, the step of obtaining a loss function based on the first dimension-raising sequence, the second dimension-raising sequence, the first output sequence, and the second output sequence includes: calculating the cosine similarity between the first dimension-raising sequence and the second output sequence to obtain a first cosine similarity, and calculating the cosine similarity between the second dimension-raising sequence and the first output sequence to obtain a second cosine similarity; and obtaining the loss function based on the sum of half the first cosine similarity value and half the second cosine similarity value.

[0014] Optionally, the step of obtaining transcription factor-target pair data as the first data includes: obtaining gene expression data, determining the top N gene data with the largest variance in the gene expression data, and normalizing the gene data; obtaining the transcription factor-target pair corresponding to each gene in the normalized gene data as the first data.

[0015] Optionally, the data enhancement processing includes: interpolation processing, noise introduction processing, scaling processing, contrast adjustment and discarding processing.

[0016] An embodiment of the present invention provides a method for predicting gene regulatory networks. Under normal circumstances, the prediction of gene regulatory networks is the process of predicting whether a transcription factor-target pair has a regulatory relationship. Therefore, the transcription factor-target pair data is obtained as the first data. In order to ensure the pattern invariance of the gene sequence during the simulation process, the first data is enhanced to obtain the second data. The correlation between the transcription factor and the target target is judged based on the predetermined task, and a pseudo label between the transcription factor and the target target is generated. The first data and the second data are input into a neural network model for training iteration to obtain a feature matrix reflecting whether each transcription factor-target pair has a regulatory relationship. Based on the obtained transcription factors with regulatory relationships, the pseudo label between the transcription factor and the target target is obtained. The sub-target pairs adjust the weights of the corresponding transcription factor-target pairs in the feature matrix, introduce the contrastive learning framework in self-supervised learning, and use a large number of transcription factor-target pairs for which it is not determined whether they have a regulatory relationship and a small number of transcription factor-target pairs for which it is determined that they have a regulatory relationship for self-supervised learning. Based on the learning results, the weights of the transcription factor-target pairs in the feature matrix are adjusted, and the prediction results of the gene regulatory network are obtained based on the feature matrix after adjusting the weights. The introduction of self-supervised learning can ensure the accuracy of the gene regulatory network prediction without the need for a large amount of labeled gene regulatory network data, and at the same time, avoid the high cost of labeling gene regulatory network data.

[0017] An embodiment of the present invention further provides a gene regulatory network prediction system for executing the above-mentioned gene regulatory network prediction method.

[0018] Other features and advantages of the embodiments of the present invention will be described in the following description, or some features and advantages can be inferred or determined without doubt from the description, or can be learned by implementing the above-mentioned technologies of the embodiments of the present invention.

[0019] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0021] Figure 1 A flow chart of a gene regulatory network prediction method provided by an embodiment of the present invention is shown;

[0022] Figure 2a A waveform diagram of gene data in a gene regulatory network prediction method provided by an embodiment of the present invention is shown;

[0023] Figure 2b A waveform diagram of gene data subjected to interpolation processing in a gene regulatory network prediction method provided by an embodiment of the present invention is shown;

[0024] Figure 2c A waveform diagram of gene data subjected to noise processing in a gene regulatory network prediction method provided by an embodiment of the present invention is shown;

[0025] Figure 2d A waveform diagram of gene data after scaling processing in a gene regulatory network prediction method provided by an embodiment of the present invention is shown;

[0026] Figure 2e A schematic diagram of a waveform of gene data after contrast adjustment in a gene regulatory network prediction method provided by an embodiment of the present invention is shown;

[0027] Figure 2f A waveform diagram showing gene data discarded in a gene regulatory network prediction method provided by an embodiment of the present invention is shown;

[0028] Figure 3 An example diagram showing a gene regulatory network prediction method provided by an embodiment of the present invention for predicting a gene regulatory network is shown;

[0029] Figure 4a A schematic diagram showing the result of predicting the gene regulatory network of a first gene of a corn crop using a gene regulatory network prediction method provided by an embodiment of the present invention;

[0030] Figure 4b A schematic diagram showing the result of predicting the gene regulatory network of a second gene of a corn crop using a gene regulatory network prediction method provided by an embodiment of the present invention;

[0031] Figure 4c A schematic diagram showing the result of predicting the gene regulatory network of a third gene of a corn crop using a gene regulatory network prediction method provided by an embodiment of the present invention;

[0032] Figure 4d A schematic diagram showing the detailed results of gene regulatory network prediction for a first gene of a corn crop using a gene regulatory network prediction method provided by an embodiment of the present invention;

[0033] Figure 5aA schematic diagram showing the results of comparing the AUROC index of a gene regulatory network prediction method provided by an embodiment of the present invention with the AUROC index of other prediction methods after fine-tuning using the RegMap network on data of different sampling scales;

[0034] Figure 5b A schematic diagram showing the results of comparing the AUPRC index of a gene regulatory network prediction method provided by an embodiment of the present invention with the AUPRC index of other prediction methods after fine-tuning using the RegMap network on data of different sampling scales;

[0035] Figure 5c A schematic diagram showing the results of comparing the AUROC index of a gene regulatory network prediction method provided by an embodiment of the present invention with the AUROC index of other prediction methods after fine-tuning using the STRING network on data of different sampling scales;

[0036] Figure 5d A schematic diagram shows the results of comparing the AUPRC index of a gene regulatory network prediction method provided by an embodiment of the present invention after fine-tuning using the RegMap network on data of different sampling scales with the AUPRC index of other prediction methods. DETAILED DESCRIPTION

[0037] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be described below with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0038] At present, gene regulatory networks play an important role in inferring and understanding complex regulatory issues between genes. They help to successfully predict protein expression at the gene level, thereby providing support for further analysis of plant phenotypes such as high yield and disease resistance. Considering that the use of traditional supervised learning in the existing technology to predict gene regulatory networks requires a large amount of labeled (i.e., regulatory relationships have been determined) gene regulatory network data to ensure accurate prediction of gene regulatory networks, and the labeling cost of gene regulatory network data is too high to guarantee the use of traditional supervised learning, resulting in low accuracy of traditional supervised learning for gene regulatory networks. To improve this problem, an embodiment of the present invention provides a gene regulatory network prediction method and prediction system. This technology introduces self-supervised learning, so that a large number of transcription factor-target pairs with undetermined regulatory relationships are compared with a small number of transcription factor-target pairs with determined regulatory relationships to complete the prediction of gene regulatory networks and ensure the accuracy of the prediction during the prediction process. The embodiments of the present invention are introduced in detail below.

[0039] This embodiment provides a method for predicting gene regulatory networks, which can be applied to electronic devices such as computers. Figure 1 A flow chart of a gene regulatory network prediction method is shown, and the method mainly includes the following steps:

[0040] S102, obtaining transcription factor-target pair data as first data, performing data enhancement processing on the transcription factor-target pair data, and obtaining processed transcription factor-target pair data as second data;

[0041] The above steps mainly involve obtaining the data of the transcription factor (TF)-target pair (and TF-Target pair) in each gene in the gene expression as the first data (i.e., unlabeled data), and inputting the data of the transcription factor-target pair into the data enhancement layer for data enhancement processing to obtain the second data (i.e., enhanced data).

[0042] S104, generating pseudo labels for the transcription factors and the target targets based on the assumption of correlation between the first data and the second data and inputting the pseudo-labeled first data and the second data into a neural network model for training iterations to obtain a feature matrix including transcription factor-target pairs; wherein the feature matrix reflects whether each transcription factor-target pair has a regulatory relationship;

[0043] In the above steps, a predetermined task is constructed according to the paradigm of self-supervised learning, and a pseudo-label that can reflect the correlation between the transcription factor and the target is generated based on the predetermined task. The first data and the second data after the pseudo-label is generated are input into the neural network model. The powerful learning ability of the model is used to finally obtain a feature matrix containing all transcription factor-target pairs, and this feature matrix reflects whether each transcription factor-target pair has a regulatory relationship.

[0044] S106, obtaining gene regulatory network data, identifying transcription factor-target pairs with regulatory relationships therein, and adjusting the weights of corresponding transcription factor-target pairs in the feature matrix based on the transcription factor-target pairs with regulatory relationships;

[0045] In the above steps, gene regulatory network data (i.e., small sample data) are obtained, transcription factor-target pairs with regulatory relationships are determined, and the weights of transcription factor-target pairs corresponding to the transcription factor-target pairs with regulatory relationships in the feature matrix are adjusted to ensure the accuracy of the feature matrix in identifying the regulatory relationship of the transcription factor-target pairs.

[0046] S108, judging whether the transcription factor-target pair has a regulatory relationship based on the feature matrix after weight adjustment, and obtaining a prediction result of the gene regulatory network;

[0047] The feature matrix after weight adjustment in the above steps can identify whether the transcription factor-target pair has a regulatory relationship, and finally obtain the prediction result of the gene regulatory network based on the identified transcription factor-target pair.

[0048] The above-mentioned gene regulatory network prediction method provided by the embodiment of the present invention is a process of predicting whether a transcription factor-target pair has a regulatory relationship under normal circumstances. Therefore, the transcription factor-target pair data is obtained as the first data. In order to ensure the pattern invariance of the gene sequence during the simulation process, the first data is subjected to data enhancement processing to obtain the second data. The correlation between the transcription factor and the target is judged based on the predetermined task, and the pseudo label between the transcription factor and the target is generated. The first data and the second data are input as a large amount of unlabeled data (i.e., data for which it is not determined whether there is a regulatory relationship) into the neural network model for training iteration to obtain a feature matrix reflecting whether each transcription factor-target pair has a regulatory relationship. Based on the obtained transcription factor-target pairs with regulatory relationships, a pseudo label is generated between the transcription factor and the target. The target target pairs (i.e., small sample data) adjust the weights of the corresponding transcription factor-target target pairs in the feature matrix, introduce the comparative learning framework in self-supervised learning, and use a large number of transcription factor-target target pairs whose regulatory relationships have not been determined to be self-supervised learning on a small number of transcription factor-target target pairs that have been determined to have regulatory relationships. Based on the learning results, the weights of the transcription factor-target target pairs in the feature matrix are adjusted, and the prediction results of the gene regulatory network are obtained based on the feature matrix after adjusting the weights. The introduction of self-supervised learning can ensure the accuracy of the gene regulatory network prediction without the need for a large amount of labeled gene regulatory network data, and at the same time, avoid the high cost of labeling gene regulatory network data.

[0049] In one embodiment, this embodiment provides a specific implementation method for increasing the weight of the corresponding transcription factor-target pair in the feature matrix based on the transcription factor-target pair with a regulatory relationship:

[0050] A simple linear encoder was constructed based on the transcription factor-target pairs with regulatory relationships and binary cross entropy for training. The transcription factor-target pairs with regulatory relationships in the feature matrix were annotated and the weight of the transcription factor-target pairs in the feature matrix was increased.

[0051] In the above steps, since the gene data has multiple features, a large-scale feature matrix will be formed as the pre-training model. In the fine-tuning stage, a small number of transcription factor-target pairs with regulatory relationships and binary cross-entropy (BCE) are used as the loss function to adjust whether the relationship between the transcription factor-target pairs in the pre-training model is regulatory. Specifically, the expression of the binary cross-entropy function is:

[0052]

[0053] Where N represents the number of samples, y i represents the label of a small-scale labeled sample i, p(y i ) represents the probability that sample i is predicted to be a positive label;

[0054] A simple linear encoder based on ResMLP was constructed based on the binary cross entropy function, and the simple linear encoder was trained based on the pre-trained model and a small-scale transcription factor-target pair with a regulatory relationship. The linear encoder has a linear layer and the output dimension of the linear layer is 1, called the feature coefficient, denoted as α. Based on the binary classification task, all the output results of the linear encoder were thoroughly analyzed, and an appropriate threshold θ was established. When α≥θ, the output is classified as positive, proving that the transcription factor-target pair has a regulatory relationship. When α<θ, the output is classified as negative, proving that the transcription factor-target pair does not have a regulatory relationship. The value of θ is adjusted according to the experimental results to achieve the ideal expected effect.

[0055] In one embodiment, the specific task of the predetermined task provided in this embodiment is: assuming that each transcription factor in the first data and the second data is only correlated with the target target in each transcription factor-target pair, and has no correlation with other target targets;

[0056] In the above steps, the transcription factor-target pairs in the first data and the second data are unlabeled data because it has not been determined whether they have a regulatory relationship. It is assumed that each transcription factor is only correlated with the target target in each transcription factor-target pair and is established as the only positive sample, while each transcription factor has no correlation with the target target other than the transcription factor-target pair as a negative sample, thereby establishing a pseudo-label between the transcription factor and the target target. Specifically, based on the contrastive learning methodology, i.e., minimizing intra-class differences and maximizing inter-class differences, a predetermined task is established to minimize the intra-class differences between the transcription factor and the target target in each positive sample and maximize the inter-class differences between each positive sample. This predetermined task is different from the task established in traditional supervised learning, which is "assuming that all transcription factor-target pairs are unrelated and determine whether the current transcription factor-target pair has a regulatory relationship." This predetermined task establishes a pseudo-label before determining the regulatory relationship between the transcription factor-target pair, so that only fine-tuning is required later, simplifying the prediction process.

[0057] In one embodiment, this embodiment provides a specific implementation method of inputting the first data and the second data after generating pseudo labels into a neural network model for training iteration to obtain a feature matrix including transcription factor-target pairs:

[0058] Setting up a neural network model, dividing the first data and the second data after generating pseudo labels into d non-overlapping grids of length s, and inputting the grids into the embedding layer of the neural network model for dimensionality increase processing;

[0059] The neural network formula set in the above steps is:

[0060] y=f(x,b,W),

[0061] Where x is the input of the neural network, y is the output of the neural network, b is the learnable bias parameter, and W is the learnable network weight;

[0062] Sequence G of each transcription factor-target pair in the first data and the second data i Can be expressed as:

[0063]

[0064] Among them, g represents the number of genes, and n represents the sequence length of the gene expression corresponding to the transcription factor-target pair;

[0065] The first data and the second data are divided into d non-overlapping grids of length s, so that the subsequent order corresponds to the encoder blocks in the encoder.

[0066] Inputting the first data and the second data after the dimension increase processing into the encoder of the neural network model respectively to obtain a first embedding sequence and a second embedding sequence;

[0067] In the above steps, the first data and the second data after the dimension-raising process are input into the encoder (i.e., the encoding layer) in the order of the grid division to obtain the first embedding sequence X1 (l) and the second embedding sequence X2 (l) , and the encoder is based on the Transformer design, in which the multi-head attention mechanism is introduced. The encoder is specifically divided into three parts, including: linear projection layer, attention layer and linear output layer.

[0068] Inputting the first embedded sequence and the second embedded sequence into the projection layer of the neural network model for dimensionality increase processing to obtain a first dimensionality increase sequence and a second dimensionality increase sequence, and performing gradient update on the first dimensionality increase sequence and the second dimensionality increase sequence to obtain a first gradient sequence and a second gradient sequence;

[0069] The projection layer of the neural network model in the above steps is a three-layer fully connected layer, specifically including: input layer (input is 512 dimensions, output is 1024 dimensions), hidden layer (input is 1024 dimensions, output is 1024 dimensions), output layer (input is 1024 dimensions, output is 2048 dimensions), among which, the data of the input layer and hidden layer after the projection layer will be normalized and ReLU activated to improve the classification performance of the neural network model. (l) Input the projection layer to perform dimensionality increase processing to obtain the first dimensionality increase sequence sg(z1) of 2048 dimensions (l) , embed the second sequence X2 (l) Input the projection layer to perform dimensionality increase processing to obtain the second dimensionality increase sequence sg(z2) of 2048 dimensions (l) , input the first dimension-raising sequence into the feedforward neural network for gradient update (stop-gard, sg) to obtain the first gradient sequence z1 of 2048 dimensions (l) , input the second dimension-increasing sequence into the feedforward neural network for gradient update to obtain the 2048-dimensional second gradient sequence z2 (l) ;

[0070] Among them, the sequence X1 (l) 、X2 (l) 、sg(z1) (l) 、sg(z2) (l) 、z1 (l) and z2 (l) They are all vector sequences. Specifically, the first dimension-raising sequence is the result of the first embedding sequence being input to the projection layer without gradient updating, and the first gradient sequence is the result of the first embedding sequence being input to the projection layer with gradient updating; the second dimension-raising sequence is the result of the second embedding sequence being input to the projection layer without gradient updating, and the second gradient sequence is the result of the second embedding sequence being input to the projection layer with gradient updating.

[0071] Inputting the first gradient sequence and the second gradient sequence into the prediction layer of the neural network model to obtain a first output sequence and a second output sequence, and obtaining a loss function based on the first dimension-raising sequence, the second dimension-raising sequence, the first output sequence, and the second output sequence;

[0072] The prediction layer of the neural network model in the above steps is a two-layer fully connected layer, specifically including: input layer (input is 2048 dimensions, output is 512 dimensions), output layer (input is 512 dimensions, output is 2048 dimensions), the first gradient sequence z1 (l) Input to the prediction layer to obtain the first output sequence p1 of 2048 dimensions (l) , the second gradient sequence z2 (l) Input to the prediction layer and get the second output sequence p2 of 2048 dimensions (l) , based on sg(z1) (l) 、sg(z2) (l) 、p1 (l) and p2 (l) The sequence obtains the loss function Loss(p i ,z j ), where sequence p1 (l) and p2 (l) Also a vector sequence.

[0073] Perform multiple iterations to minimize the loss function and obtain the feature matrix of the transcription factor-target pair;

[0074] In the above steps, multiple iterations are performed based on the neural network model to minimize the loss function. During the iteration process, the AdamW optimizer is combined with the automatic mixed precision AMP library to speed up the fitting process and reduce GPU memory consumption.

[0075] In one embodiment, this embodiment provides a specific implementation method for inputting the embedding layer of the neural network model to perform dimensionality increase processing:

[0076] The first data and the second data, which are divided into d non-overlapping grids of length s, are input into the embedding layer and linearly projected, and then dimensionality is increased by the embedding weight matrix. The expression of the embedding weight matrix is:

[0077]

[0078] In the expression of the embedding weight matrix: E represents the embedding weight matrix, is a set of real numbers, s represents the length of the grid, g represents the number of genes, and h represents the number of hidden layers in the embedding layer;

[0079] In the above steps, the embedding layer is a three-layer fully connected layer, specifically including: input layer, hidden layer and output layer. Among them, the data of the input layer and hidden layer passing through the embedding layer will be normalized and ReLU activated. The above embedding weight matrix E can be optimized through training.

[0080] In one embodiment, this embodiment provides a specific implementation method of inputting the first data and the second data after dimensionality increase processing into the encoder of the neural network model to obtain the first embedding sequence and the second embedding sequence:

[0081] Inputting the grid segments of the first data and the second data after the dimension increase processing into respective encoder blocks in the encoder of the neural network model;

[0082] In the above steps, the d grid segments of the first data and the d grid segments of the second data after dimension upgrade are sequentially input into each encoder block, and each grid segment of the first data and the second data can be represented as a matrix:

[0083]

[0084] Among them, W O is the representation matrix of the grid segments, d represents the number of grids or grid segments, and h represents the number of hidden layers of the embedding layer, which is 512 in this embodiment.

[0085] Based on the multi-head attention mechanism in each encoder block, the grid segments input to each encoder block are subjected to attention weight learning;

[0086] In the above steps, the multi-head attention mechanism in each encoder block is used to process the grid segments input to each encoder, so that it learns different attention weights. Specifically, the multi-head attention mechanism processing formula can be expressed as:

[0087] MH(X (l) )=Concat(x1,x2,...,x d )W O ,

[0088] Among them, MH is the abbreviation of Multi-Head, which represents the multi-head attention mechanism, X (l) is the input sequence of the first data or the second data, Concat(x1,x2,...,x d ) represents the output of the attention head corresponding to each grid segment.

[0089] The grid segments in each encoder block after attention weight learning are input into the feedforward neural network for gradient update;

[0090] In the above steps, the grid segments after learning the attention weights in each encoder block are input into the feedforward neural network for gradient update. The feedforward neural network consists of two linear layers, a GELU activation function, and a ReLU activation function. The formula for gradient update based on the feedforward neural network can be expressed as:

[0091] ReLU(x a )=max(0,x a ),

[0092]

[0093] FFN(MH(X (l) ))=ReLU(W2GELU(W1MH(X (l) ))+b2)+b1,

[0094] Among them, ReLU represents the ReLU activation function, x a is the input of the ReLU activation function, GELU represents the GELU activation function, x b is the input of the GELU activation function, h is the number of hidden layers in the embedding layer, FFN is the positional feedforward function in the feedforward neural network, W1 is the first learnable weight matrix, W2 is the second learnable weight matrix, b1 is the first learnable bias term, and b2 is the second learnable bias term.

[0095] Add residual connections to the grid segments in each encoder block after gradient update and perform normalization to obtain a first embedding sequence and a second embedding sequence;

[0096] In the above steps, adding residual connections to the grid segments and performing normalization can obtain the output of the encoder at layer l, which can be expressed as:

[0097] X (l+1) =X (l) +LayerNorm(FFN(MH(X (l) ))),

[0098]

[0099] Among them, X (l+1) is the output of the encoder at layer l+1, LayerNorm is the normalization function z-score, x i,t represents the expression value of gene i at time t, represents the average expression value of gene i at all times, σ i represents the standard deviation of the expression values ​​of gene i at all times.

[0100] In one embodiment, this embodiment provides a specific implementation of obtaining a loss function based on the first dimension-raising sequence, the second dimension-raising sequence, the first output sequence, and the second output sequence:

[0101] Calculating the cosine similarity between the first dimension-raising sequence and the second output sequence to obtain a first cosine similarity, and calculating the cosine similarity between the second dimension-raising sequence and the first output sequence to obtain a second cosine similarity;

[0102] The formula for solving the cosine similarity in the above steps is as follows:

[0103]

[0104] Among them, D(p i ,z j ) represents the cosine similarity between the dimension-raising sequence and the output sequence, sg(z j ) is a dimensional sequence, p i is the output sequence, and ||·||2 represents L2 normalization (L2-Norm), which is equivalent to the mean square error of the L2-norm vector.

[0105] A loss function is obtained based on the sum of half of the first cosine similarity value and half of the second cosine similarity value;

[0106] The loss function in the above steps is defined as follows:

[0107]

[0108] The loss function is obtained based on the sum of half of the first cosine similarity value and half of the second cosine similarity value.

[0109] In one embodiment, this embodiment provides a specific implementation method for obtaining transcription factor-target pair data as the first data:

[0110] Obtain gene expression data, determine the top N gene data with the largest variance in the gene expression data, and normalize the gene data;

[0111] The gene expression data in the above steps is the expression value matrix of genes during biological development, which implies the regulatory relationship between genes. Therefore, obtaining the data of the top N genes with the largest variance can indirectly analyze the regulatory relationship between relatively discrete genes.

[0112] Obtaining the transcription factor-target pair corresponding to each gene in the normalized gene data as first data;

[0113] In the above steps, the normalization process is based on unifying the genetic data to the same scale, thereby improving the accuracy and efficiency of subsequent analysis.

[0114] In one embodiment, this embodiment provides a specific implementation of data enhancement processing, including: interpolation processing, noise introduction processing, scaling processing, contrast adjustment and discarding processing;

[0115] The specific implementation of the data enhancement processing in the above steps includes:

[0116] See for example Figure 2a Gene data: The normalized gene data is the gene expression matrix of the transcription factor-target pair;

[0117] See for example Figure 2b Interpolation processing: Considering that time series data can maintain causal relationships by slightly adjusting the sequence length, n gene data are randomly selected from the original data for replication. At the same time, the replicated n gene data are inserted into the previous position between consecutive gene data to form interpolated gene data;

[0118] See for example Figure 2c Noise treatment: simulates the situation where the genetic data is biased by estimation or annotation, so the sequence G of the transcription factor-target pair i (G TF ,G target ) Gaussian noise N(0, 0.2) with a dropout rate of 0.2 was applied to reduce the noise sensitivity of the genetic data;

[0119] See for example Figure 2d Scaling: represents the change in the sequence of the transcription factor-target pair. A scaling factor between (0.6, 1.4) is randomly selected and applied to the sequence of the transcription factor-target pair to change the steepness of the transcription factor-target pair sequence.

[0120] See for example Figure 2e Contrast adjustment: according to the sequence G of the transcription factor-target pair i Mean(G i ) is selected for the sequence by a scaling factor between (0.6, 1.4) to adjust the contrast of the sequence, where the mean value Mean(G i ), the higher the scaling factor.

[0121] See for example Figure 2f Dropout: Simulating the situation of naturally missing values ​​in the sequences of transcription factor-target pairs, some gene data are randomly removed, and the deleted values ​​range from 0% to 80% of the entire sequence. This allows the gene data to still identify characteristic relationships in the presence of missing values.

[0122] This embodiment also provides a gene regulatory network prediction system for executing the above-mentioned gene regulatory network prediction method.

[0123] In the embodiment of the present invention, specific data enhancement or its combination ensures the pattern invariance of the gene sequences in the first data and the second data, so that the same sequence G i The similar patterns are maintained in the data, and the pattern invariance between the first data and the second data is guaranteed by data enhancement, so that the inherent similarity makes the representation learning more efficient and effective in the comparison and encoding interaction mode.

[0124] In the embodiment of the present invention, the characteristics of self-supervised learning and small sample learning are combined, so that the regulatory labels of transcription factor-target pairs can be generated automatically, thereby eliminating the need for manual labeling of regulatory labels and reducing costs. At the same time, the multi-head attention mechanism is introduced in the prediction process to increase the prediction method's attention to gene expression levels at different time points, thereby improving the ability to predict gene regulatory networks. It can study the expression and regulatory mechanisms of genes in organisms, which is of great significance for revealing the complex molecular interactions in life systems. It has broad application prospects in crop breeding, disease target analysis and other fields.

[0125] Based on the above embodiments, Figure 3 As shown, this embodiment provides an example of applying the aforementioned gene regulatory network prediction method to predict a gene regulatory network, which can be performed by referring to the following steps:

[0126] S201, obtaining gene expression data, determining the top N gene data with the largest variance in the gene expression data, normalizing the gene data, and obtaining the transcription factor-target pair corresponding to each gene as first data. Specifically, the gene expression data comes from three public gene expression datasets for maize seeds, ovules, and nucellus;

[0127] S203, inputting the first data into a data enhancement layer for interpolation processing, noise introduction processing, scaling processing, contrast adjustment, and discarding processing to obtain second data;

[0128] S205, according to the self-supervised learning paradigm, constructing a predetermined task: "Assume that all transcription factor-target pairs have correlations, and determine whether the correlations belong to regulation." The transcription factor-target pairs in the first data and the second data are subjected to correlation assumptions according to the predetermined task. It is assumed that each transcription factor in the first data and the second data has a relationship only with the target target in the corresponding transcription factor-target pair as a positive sample, and has no relationship with other target targets as a negative sample, thereby completing the generation of pseudo labels.

[0129] S207, dividing the first data and the second data after generating pseudo labels into d non-overlapping grids of length s, and inputting the grids into the embedding layer of the neural network model for dimensionality increase processing;

[0130] S209, inputting the first data and the second data after the dimension increase processing into the encoder of the neural network model respectively to obtain a first embedding sequence and a second embedding sequence;

[0131] S211, inputting the first embedded sequence and the second embedded sequence into the projection layer of the neural network model for dimensionality increase processing to obtain a first dimensionality increase sequence and a second dimensionality increase sequence, and performing gradient update on the first dimensionality increase sequence and the second dimensionality increase sequence to obtain a first gradient sequence and a second gradient sequence;

[0132] S213, inputting the first gradient sequence and the second gradient sequence into the prediction layer of the neural network model to obtain a first output sequence and a second output sequence, and obtaining a loss function based on the first dimension-raising sequence, the second dimension-raising sequence, the first output sequence, and the second output sequence;

[0133] S215, performing multiple iterations to minimize the loss function, obtaining a feature matrix of the transcription factor-target pair as a pre-training model, and dividing the pre-training model into a training set, a validation set, and a test set, with the ratio of training set: validation set: test set = 3:1:1;

[0134] S217, obtaining gene regulatory network data and identifying transcription factor-target pairs with regulatory relationships therein, wherein the gene regulatory network data comes from public regulatory networks in different databases, namely STRING, RegMap, FunTBS, Motfi_CE, Motfi-Zma, and Merged-Zma;

[0135] S219, a simple linear encoder based on ResMLP was constructed based on the binary cross entropy function, and the simple linear encoder was trained based on a small number of transcription factor-target pairs with regulatory relationships and pre-trained models in the training set;

[0136] S221, verify and adjust the parameters of the simple linear encoder using the pre-trained model in the validation set, and retain the parameters corresponding to the pre-trained model with the most accurate prediction in the iterative process. Test the pre-trained model in the test set and use two error evaluation criteria to analyze the feasibility and effectiveness, namely the ROC curve integral AUROC and the PR curve integral AUPRC. The formulas are:

[0137]

[0138] Among them, Recall is the True Positive Rate, also known as sensitivity or recall rate, which is calculated as TP is the number of true positive examples, FN is the number of false negative examples; FPR is the false positive rate (False Positive Rate), which is calculated as follows: Where FP is the number of false positives, TN is the number of true negatives; Precision is the accuracy, which is calculated as

[0139] In the embodiment of the present invention Figure 4a 、 Figure 4b and Figure 4c The gene ontology (GO) IDs and loci represented by the serial numbers are shown in Table 1 below as an example.

[0140] Table 1 Figure 4a 、 Figure 4b and Figure 4c The ID of the gene ontology represented by the serial number and the locus where it is located

[0141]

[0142]

[0143]

[0144]

[0145]

[0146]

[0147] The superscript 1 of the enrichment times in Table 1 above indicates the number of proteins annotated with specific terms in the protein-protein interaction (PPI) network.

[0148] In the embodiment of the present invention, see Figure 4a 、 Figure 4b and Figure 4c These are protein sets expressed by three key genes in corn crops, called loci. These genes are believed to be related to seed development and inflorescence development of corn crops. The color depth of each node in the figure represents the in-degree of each node. The darker the node color, the higher the node in-degree. Figure 4a 、 Figure 4b and Figure 4cThe dotted arrows in the figure represent predicted associations, the darker solid arrows represent existing associations, and the lighter solid arrows represent verification of the associations. Based on the results of the protein set expressed by three key genes in corn crops, the protein interaction network was used to verify the results. These genes were paired with all other genes as basic input pairs, and predictions were made based on the gene regulatory network prediction method provided by the embodiment of the present invention to obtain prediction results. Specifically, Figure 4a From the GO perspective of biological processes, there is evidence that this regulatory network is related to proteins expressed in the development of flowers and seeds, such as floral whorl structural organization (GO:0048459), seed growth (GO:0080112), and floral organ number (GO:0048833).

[0149] In the embodiment of the present invention, Figure 4d for Figure 4a Two details of Figure 4d The details in the upper middle section show that the gene ontology represented by node 6 is predicted to have a regulatory relationship with the gene ontology represented by node 19, and the regulatory relationship between the gene ontology represented by node 6 and the gene ontology represented by node 19 is verified. Figure 4d The details in the middle and lower part show that the gene ontology represented by node 6 is predicted to have a regulatory relationship with the gene ontology represented by node 18, and the existing regulatory relationship between the gene ontology represented by node 6 and the gene ontology represented by node 18 is verified; through the prediction of the protein interaction network, it is found that Figure 4a 、 Figure 4b and Figure 4c Among the prediction results, there are associations that are verified by existing regulatory networks and exemplified by gene ontology evidence.

[0150] In the embodiment of the present invention, see Figure 5a 、 Figure 5b 、 Figure 5c and Figure 5d As shown in FIG, INVEST is the index result of the prediction method provided by the embodiment of the present invention after fine-tuning the network under different scale data. The color depth of the table in the figure represents the index size of each prediction method. The larger the index, the darker the color. Based on the color depth of the table and the data in the table, it can be seen that the prediction method provided by the embodiment of the present invention generally has high accuracy in the prediction process. Only in some scale data, the index is slightly lower than that of other methods. For example, see Figure 5aThe AUROC index of the prediction method provided by the embodiment of the present invention on the mSacOvule2000 dataset is only slightly lower than that of the CNNC method. Figure 5a On the mSeed1000 dataset after fine-tuning the RegMap network, INVEST achieves an AUROC score improvement of 5.56% compared to the second best method GENELink. Figure 5b In the mSeed1000 dataset after fine-tuning the RegMap network, INVEST improves the AUPRC score by 31.25% compared with the second best CNNC method. Figure 5c In the mSacOvule2000 dataset after fine-tuning the STRING network, the AUROC score of INVEST increased by 3.67% compared with the second best CNNC method. Figure 5d In

[15] , on the mSacOvule2000 dataset after fine-tuning the STRING network, INVEST's AUPRC score increased by 97.81% compared with the second best STRINGS method.

[0151] An embodiment of the present invention further provides a gene regulatory network prediction system, comprising a data acquisition module, a data enhancement module, a data prediction module, and a performance evaluation module, wherein:

[0152] Data acquisition module, which obtains gene expression data and gene regulatory network data;

[0153] A data enhancement module performs data enhancement processing on the first data;

[0154] A data prediction module feeds the first data and the second data into a neural network model for training iteration to obtain a feature matrix including transcription factor-target pairs, and fine-tunes the feature matrix based on the gene regulatory network data to obtain a prediction result of the gene regulatory network;

[0155] The performance evaluation module evaluates the prediction results based on the ROC curve integral AUROC and the PR curve integral AUPRC.

[0156] In addition, in the description of the embodiments of the present invention, unless otherwise expressly specified or limited, the terms "mounted," "connected," and "connected" should be understood in a broad sense. For example, they may refer to fixed connections, detachable connections, or integral connections; they may refer to mechanical connections or electrical connections; they may refer to direct connections or indirect connections through an intermediate medium; and they may refer to internal communication between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on the specific circumstances.

[0157] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0158] In the description of the present invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings and are intended solely to facilitate and simplify the description of the present invention. They are not intended to indicate or imply that the devices or components referred to must have, be constructed, or operate in a specific orientation, and therefore should not be construed as limitations on the present invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0159] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the above-described embodiments, those skilled in the art should understand that any person skilled in the art can modify or easily conceive of changes to the technical solutions described in the above-described embodiments within the technical scope disclosed by the present invention, or replace some of the technical features therein with equivalents. Such modifications, changes, or replacements do not deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A method for predicting a gene regulatory network, characterized in that: include: Acquiring transcription factor-target pair data as first data, performing data enhancement processing on the transcription factor-target pair data, and obtaining processed transcription factor-target pair data as second data; The first data and the second data are subjected to an assumption of correlation between transcription factors and target targets based on a predetermined task, generating pseudo labels for the transcription factors and the target targets, and the first data and the second data after generating the pseudo labels are input into a neural network model for training iteration to obtain a feature matrix including transcription factor-target target pairs; wherein the feature matrix reflects whether each transcription factor-target target pair has a regulatory relationship; wherein the predetermined task is: assuming that each transcription factor in the first data and the second data is only correlated with the target target in each transcription factor-target target pair, and has no correlation with other target targets; Acquiring gene regulatory network data, identifying transcription factor-target pairs having regulatory relationships therein, and adjusting the weights of corresponding transcription factor-target pairs in the feature matrix based on the transcription factor-target pairs having regulatory relationships; Based on the weighted feature matrix, it is determined whether the transcription factor-target pair has a regulatory relationship, and a prediction result of the gene regulatory network is obtained.

2. The method for predicting gene regulatory networks according to claim 1, wherein: The step of adjusting the weights of the corresponding transcription factor-target pairs in the feature matrix based on the transcription factor-target pairs having a regulatory relationship comprises: Based on the transcription factor-target pairs with regulatory relationships and binary cross entropy, a simple linear encoder is constructed for training, and the transcription factor-target pairs with regulatory relationships in the feature matrix are labeled, and the weights of the transcription factor-target pairs in the feature matrix are adjusted.

3. The method for predicting gene regulatory networks according to claim 1, wherein: The step of inputting the first data and the second data after generating pseudo labels into a neural network model for training iteration to obtain a feature matrix including transcription factor-target pairs includes: Setting a neural network model, dividing the first data and the second data after generating pseudo labels into d non-overlapping grids of length s, and inputting the grids into the embedding layer of the neural network model for dimensionality increase processing; Inputting the first data and the second data after the dimension increase processing into the encoder of the neural network model respectively to obtain a first embedding sequence and a second embedding sequence; Inputting the first embedded sequence and the second embedded sequence into the projection layer of the neural network model for dimensionality increase processing to obtain a first dimensionality increase sequence and a second dimensionality increase sequence, and performing gradient update on the first dimensionality increase sequence and the second dimensionality increase sequence to obtain a first gradient sequence and a second gradient sequence; Inputting the first gradient sequence and the second gradient sequence into the prediction layer of the neural network model to obtain a first output sequence and a second output sequence, and obtaining a loss function based on the first dimension-raising sequence, the second dimension-raising sequence, the first output sequence, and the second output sequence; Multiple iterations are performed to minimize the loss function and obtain the feature matrix of the transcription factor-target pair.

4. The method for predicting gene regulatory networks according to claim 3, wherein: The step of inputting the embedding layer of the neural network model for dimensionality increase processing includes: The first data and the second data, which are divided into d non-overlapping grids of length s, are input into the embedding layer and linearly projected, and then dimensionality is increased by an embedding weight matrix. The expression of the embedding weight matrix is: In the expression of the embedding weight matrix: E represents the embedding weight matrix, is a set of real numbers, s represents the length of the grid, g represents the number of genes, and h represents the number of hidden layers in the embedding layer.

5. The method for predicting gene regulatory networks according to claim 3, wherein: The step of inputting the first data and the second data after the dimension increase processing into the encoder of the neural network model to obtain a first embedding sequence and a second embedding sequence includes: Inputting the grid segments of the first data and the second data after the dimension increase processing into respective encoder blocks in the encoder of the neural network model; Based on the multi-head attention mechanism in each encoder block, the grid segments input into each encoder block are subjected to attention weight learning; Inputting the grid segments in each encoder block after attention weight learning into a feedforward neural network for gradient update; Residual connections are added to the grid segments in each encoder block after gradient update and normalization is performed to obtain the first embedding sequence and the second embedding sequence.

6. The method for predicting gene regulatory networks according to claim 3, wherein: The step of obtaining a loss function based on the first dimension-raising sequence, the second dimension-raising sequence, the first output sequence, and the second output sequence includes: Calculating the cosine similarity between the first dimension-raising sequence and the second output sequence to obtain a first cosine similarity, and calculating the cosine similarity between the second dimension-raising sequence and the first output sequence to obtain a second cosine similarity; The loss function is obtained based on a sum of half the first cosine similarity value and half the second cosine similarity value.

7. The method for predicting a gene regulatory network according to any one of claims 1 to 6, characterized in that: The step of obtaining transcription factor-target pair data as first data includes: Obtaining gene expression data, determining top N gene data with the largest corresponding variance in the gene expression data, and normalizing the gene data; The transcription factor-target pair corresponding to each gene in the normalized gene data is obtained as the first data.

8. The method for predicting gene regulatory networks according to claim 7, characterized in that: The data enhancement processing includes: interpolation processing, noise introduction processing, scaling processing, contrast adjustment and discarding processing.

9. A gene regulatory network prediction system, characterized in that: A method for predicting a gene regulatory network according to any one of claims 1 to 8.

Images