Prediction method and prediction system for gene regulatory network

Abstract

The invention provides a prediction method and a prediction system for a gene regulation and control network, and relates to the technical field of quantitative prediction.The method comprises the following steps: acquiring transcription factor-target pair data as first data, and performing data enhancement processing on the first data to obtain second data; generating a pseudo tag from the first data and the second data based on a predetermined task, and inputting the first data and the second data after the pseudo tag is generated into a neural network model for training iteration to obtain a feature matrix comprising a transcription factor-target spot pair; improving the weight of a corresponding transcription factor-target pair in the feature matrix on the basis of the transcription factor-target pair with a regulation relationship in the gene regulation network data; and judging whether the transcription factor-target spot pair has a regulation and control relationship or not based on the fine-tuned feature matrix to obtain a prediction result of the gene regulation and control network. Accurate prediction of the gene regulation and control network is realized based on label-free data and small sample learning.

Description

Technical Field

[0001] The present invention relates to the field of quantitative prediction technology, and in particular to a prediction method and a prediction system for a gene regulatory network. Background Art

[0002] With the development of science and technology, existing researchers have found that the study of gene regulatory networks plays a very important role in improving stress resistance and disease resistance, obtaining high-yield plants, etc. during crop breeding. Among them, gene regulatory networks refer to the complex network structure of interactions between genes and gene regulatory relationships, involving complex regulatory mechanisms within cells and between tissues. In the existing technology, gene regulatory networks are usually predicted through traditional supervised learning. In the prediction process, a large amount of labeled gene regulatory network data is often required. However, due to the high knowledge threshold and less data of gene regulatory networks, the cost of labeling gene regulatory network data is very high. There is less labeled gene regulatory network data, which cannot meet the requirements of traditional supervised learning for a large amount of labeled gene regulatory network data, resulting in low accuracy of traditional supervised learning in predicting gene regulatory networks. Summary of the invention

[0003] In view of this, the purpose of the present invention is to provide a prediction method and prediction system for a gene regulatory network, which can solve the technical problem in the prior art that traditional supervised learning has low accuracy when predicting gene regulatory networks without a large amount of labeled data.

[0004] In order to achieve the above purpose, the technical solution adopted by the embodiment of the present invention is as follows:

[0005] In a first aspect, an embodiment of the present invention provides a method for predicting a gene regulatory network, comprising: obtaining transcription factor-target target pair data as first data, performing data enhancement processing on the transcription factor-target target pair data, and obtaining processed transcription factor-target target pair data as second data; assuming the correlation between the transcription factor and the target target based on a predetermined task, generating pseudo labels for the transcription factor and the target target based on the first data and the second data, and inputting the first data and the second data after generating the pseudo labels into a neural network model for training iteration to obtain a feature matrix including transcription factor-target target pairs; wherein the feature matrix reflects whether each transcription factor-target target pair has a regulatory relationship; obtaining gene regulatory network data, and identifying the transcription factor-target target pairs having a regulatory relationship therein, and increasing the weight of the corresponding transcription factor-target target pairs in the feature matrix based on the transcription factor-target target pairs having a regulatory relationship; judging whether the transcription factor-target target pairs have a regulatory relationship based on the feature matrix after adjusting the weight, and obtaining a prediction result of the gene regulatory network.

[0006] Optionally, the step of fine-tuning the feature matrix based on the transcription factor-target target pairs with regulatory relationships includes: constructing a simple linear encoder based on the transcription factor-target target pairs with regulatory relationships and binary cross entropy for training, labeling the transcription factor-target target pairs with regulatory relationships in the feature matrix, and increasing the weight of the transcription factor-target target pairs in the feature matrix.

[0007] Optionally, the predetermined task is: assuming that each transcription factor in the first data and the second data is only correlated with a target target in each transcription factor-target target pair, but has no correlation with other target targets.

[0008] Optionally, the step of inputting the first data and the second data after generating pseudo labels into a neural network model for training iteration to obtain a feature matrix including transcription factor-target target pairs includes: setting a neural network model, dividing the first data and the second data after generating pseudo labels into d non-overlapping grids of length s, respectively, and inputting them into the embedding layer of the neural network model for dimensionality increase processing; inputting the first data and the second data after dimensionality increase processing into the encoder of the neural network model to obtain a first embedded sequence and a second embedded sequence; inputting the first embedded sequence and the second embedded sequence into the projection layer of the neural network model for dimensionality increase processing to obtain a first dimension increase sequence and a second dimension increase sequence, and performing gradient update on the first dimension increase sequence and the second dimension increase sequence to obtain a first gradient sequence and a second gradient sequence; inputting the first gradient sequence and the second gradient sequence into the prediction layer of the neural network model to obtain a first output sequence and a second output sequence, and obtaining a loss function based on the first dimension increase sequence, the second dimension increase sequence, the first output sequence and the second output sequence; performing multiple iterations to minimize the loss function to obtain a feature matrix of transcription factor-target target pairs.

[0009] Optionally, the step of inputting the first data and the second data into the embedding layer of the neural network model for dimensionality increase processing includes: inputting the first data and the second data, which are respectively divided into d non-overlapping grids of length s, into the embedding layer for linear projection, and performing dimensionality increase processing through an embedding weight matrix; wherein the expression of the embedding weight matrix is:

[0010]

[0011] In the expression of the embedding weight matrix: E represents the embedding weight matrix, is a set of real numbers, s represents the length of the partitioned grid, g represents the number of genes, and h represents the number of hidden layers in the embedding layer.

[0012] Optionally, the step of inputting the first data and the second data after dimensionality increase processing into the encoder of the neural network model to obtain the first embedding sequence and the second embedding sequence respectively includes: inputting the grid segments of the first data and the second data after dimensionality increase processing into the respective encoder blocks in the encoder of the neural network model; based on the multi-head attention mechanism in each encoder block, the grid segments input into each encoder block are subjected to attention weight learning; after the attention weight learning, the grid segments in each encoder block are input into the feedforward neural network for gradient updating; and residual connections are added to the grid segments in each encoder block after the gradient update and normalized to obtain the first embedding sequence and the second embedding sequence.

[0013] Optionally, the step of obtaining a loss function based on the first dimension-upgraded sequence, the second dimension-upgraded sequence, the first output sequence and the second output sequence includes: calculating the cosine similarity between the first dimension-upgraded sequence and the second output sequence to obtain a first cosine similarity, calculating the cosine similarity between the second dimension-upgraded sequence and the first output sequence to obtain a second cosine similarity; and obtaining the loss function based on the sum of half of the first cosine similarity value and half of the second cosine similarity value.

[0014] Optionally, the step of obtaining transcription factor-target point pair data as the first data includes: obtaining gene expression data, determining the top N gene data with the largest variance in the gene expression data, and normalizing the gene data; obtaining the transcription factor-target point pair corresponding to each gene in the normalized gene data as the first data.

[0015] Optionally, the data enhancement processing includes: interpolation processing, noise introduction processing, scaling processing, contrast adjustment and discarding processing.

[0016] The embodiment of the present invention provides a prediction method for a gene regulatory network. Under normal circumstances, the prediction of a gene regulatory network is the process of predicting whether a transcription factor-target target pair has a regulatory relationship. Therefore, the transcription factor-target target pair data is obtained as the first data. In order to ensure the pattern invariance of the gene sequence during the simulation process, the first data is subjected to data enhancement processing to obtain the second data. The correlation between the transcription factor and the target target is judged based on a predetermined task, and a pseudo label between the transcription factor and the target target is generated. The first data and the second data are input into a neural network model for training iteration to obtain a feature matrix reflecting whether each transcription factor-target target pair has a regulatory relationship. Based on the obtained transcription factor with a regulatory relationship, a pseudo label between the transcription factor and the target target is generated. The sub-target pairs adjust the weights of the corresponding transcription factor-target pairs in the feature matrix, introduce the contrastive learning framework in self-supervised learning, use a large number of transcription factor-target pairs for which it is not determined whether they have a regulatory relationship and a small number of transcription factor-target pairs for which it is determined that they have a regulatory relationship to perform self-supervised learning, adjust the weights of the transcription factor-target pairs in the feature matrix based on the learning results, and obtain the prediction results of the gene regulatory network based on the feature matrix after adjusting the weights. The introduction of self-supervised learning can ensure the accuracy of the prediction of the gene regulatory network without the need for a large amount of labeled gene regulatory network data, and at the same time, avoid the high cost of annotating the gene regulatory network data.

[0017] The embodiment of the present invention further provides a gene regulatory network prediction system for executing the above-mentioned gene regulatory network prediction method.

[0018] Other features and advantages of the embodiments of the present invention will be described in the following description, or some features and advantages can be inferred or determined without doubt from the description, or can be learned by implementing the above-mentioned techniques of the embodiments of the present invention.

[0019] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0021] Figure 1 A flow chart of a gene regulatory network prediction method provided by an embodiment of the present invention is shown;

[0022] Figure 2a A waveform diagram of gene data in a gene regulatory network prediction method provided by an embodiment of the present invention is shown;

[0023] Figure 2b A waveform diagram of gene data after interpolation processing in a gene regulatory network prediction method provided by an embodiment of the present invention is shown;

[0024] Figure 2c A waveform schematic diagram of gene data subjected to noise processing in a gene regulatory network prediction method provided by an embodiment of the present invention is shown;

[0025] Figure 2d A waveform diagram of gene data after scaling processing in a gene regulatory network prediction method provided by an embodiment of the present invention is shown;

[0026] Figure 2e A schematic diagram of a waveform of gene data after contrast adjustment in a gene regulatory network prediction method provided by an embodiment of the present invention is shown;

[0027] Figure 2f A waveform diagram showing gene data after discarding processing in a gene regulatory network prediction method provided by an embodiment of the present invention is shown;

[0028] Figure 3 An example diagram showing a gene regulatory network predicted by a gene regulatory network prediction method provided by an embodiment of the present invention;

[0029] Figure 4a A schematic diagram showing the result of predicting a gene regulatory network for a first gene of a corn crop using a gene regulatory network prediction method provided by an embodiment of the present invention;

[0030] Figure 4b A schematic diagram showing the result of predicting a gene regulatory network for a second gene of a corn crop using a gene regulatory network prediction method provided by an embodiment of the present invention;

[0031] Figure 4c A schematic diagram showing the result of predicting the gene regulatory network of a third gene of a corn crop using a gene regulatory network prediction method provided by an embodiment of the present invention;

[0032] Figure 4d A detailed schematic diagram showing the result of predicting a gene regulatory network of a first gene of a corn crop using a gene regulatory network prediction method provided by an embodiment of the present invention;

[0033] Figure 5aA schematic diagram showing the comparison results of the AUROC index of a gene regulatory network prediction method provided by an embodiment of the present invention after fine-tuning using the RegMap network on data of different sampling scales and the AUROC index of other prediction methods;

[0034] Figure 5b A schematic diagram showing the results of comparing the AUPRC index of a gene regulatory network prediction method provided by an embodiment of the present invention with the AUPRC index of other prediction methods after fine-tuning using the RegMap network on data of different sampling scales;

[0035] Figure 5c A schematic diagram showing the comparison results of the AUROC index of a gene regulatory network prediction method provided by an embodiment of the present invention after fine-tuning using a STRING network on data of different sampling scales and the AUROC index of other prediction methods;

[0036] Figure 5d A schematic diagram showing the results of comparing the AUPRC index of a gene regulatory network prediction method provided by an embodiment of the present invention after fine-tuning using the RegMap network on data of different sampling scales with the AUPRC index of other prediction methods. DETAILED DESCRIPTION

[0037] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution of the present invention will be described below in conjunction with the drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments.

[0038] At present, gene regulatory networks play an important role in inferring and understanding complex regulatory issues between genes. They help to successfully predict protein expression at the gene level, thereby providing support for further analysis of phenotypes such as high yield and disease resistance of plants. Considering that the use of traditional supervised learning in the prior art to predict gene regulatory networks requires a large amount of labeled (i.e., determined regulatory relationships) gene regulatory network data to ensure accurate prediction of gene regulatory networks, and the labeling cost of gene regulatory network data is too high to ensure the use of traditional supervised learning, resulting in low prediction accuracy of traditional supervised learning for gene regulatory networks. To improve this problem, an embodiment of the present invention provides a gene regulatory network prediction method and prediction system. The technology introduces self-supervised learning, so that a large number of transcription factor-target pairs with undetermined regulatory relationships are compared with a small number of transcription factor-target pairs with determined regulatory relationships to complete the prediction of gene regulatory networks, and the accuracy of the prediction is guaranteed during the prediction process. The embodiments of the present invention are described in detail below.

[0039] This embodiment provides a method for predicting a gene regulatory network, which can be applied to electronic devices such as computers, see Figure 1 A flow chart of a gene regulatory network prediction method is shown, the method mainly comprises the following steps:

[0040] S102, obtaining transcription factor-target pair data as first data, performing data enhancement processing on the transcription factor-target pair data, and obtaining processed transcription factor-target pair data as second data;

[0041] The above steps mainly involve obtaining the data of the transcription factor (TF)-target pair (and TF-Target pair) in each gene in the gene expression as the first data (i.e., unlabeled data), and inputting the data of the transcription factor-target pair into the data enhancement layer for data enhancement processing to obtain the second data (i.e., enhanced data).

[0042] S104, based on the assumption of correlation between the transcription factor and the target point in the predetermined task, the first data and the second data are used to generate pseudo labels for the transcription factor and the target point, and the first data and the second data after the pseudo labels are generated are input into the neural network model for training iteration to obtain a feature matrix including transcription factor-target point pairs; wherein the feature matrix reflects whether each transcription factor-target target pair has a regulatory relationship;

[0043] In the above steps, a predetermined task is constructed according to the paradigm of self-supervised learning, a pseudo-label that can reflect the correlation between the transcription factor and the target is generated based on the predetermined task, and the first data and the second data after the pseudo-label is generated are input into the neural network model. The powerful learning ability of the model is used to finally obtain a characteristic matrix containing all transcription factor-target pairs, and the characteristic matrix reflects whether each transcription factor-target pair has a regulatory relationship.

[0044] S106, acquiring gene regulatory network data, identifying transcription factor-target pairs having regulatory relationships therein, and adjusting weights of corresponding transcription factor-target pairs in the feature matrix based on the transcription factor-target pairs having regulatory relationships;

[0045] In the above steps, gene regulatory network data (i.e., small sample data) are obtained to determine the transcription factor-target pairs with regulatory relationships therein, and the weights of the transcription factor-target pairs corresponding to the transcription factor-target pairs with regulatory relationships in the feature matrix are adjusted to ensure the accuracy of the feature matrix in identifying the regulatory relationship of the transcription factor-target pairs.

[0046] S108, judging whether the transcription factor-target pair has a regulatory relationship based on the feature matrix after adjusting the weight, and obtaining a prediction result of the gene regulatory network;

[0047] The feature matrix after adjusting the weights in the above steps can identify whether the transcription factor-target pair has a regulatory relationship, and finally obtain the prediction result of the gene regulatory network based on the identified transcription factor-target pair.

[0048] The above-mentioned gene regulatory network prediction method provided by the embodiment of the present invention, since under normal circumstances, the prediction of the gene regulatory network (Gene Regulatory Network, GRN) is the process of predicting whether the transcription factor-target target pair has a regulatory relationship, the transcription factor-target target pair data is obtained as the first data, in order to ensure the characteristic of the pattern invariance of the gene sequence in the simulation process, the first data is subjected to data enhancement processing to obtain the second data, the correlation between the transcription factor and the target target is judged based on the predetermined task, and the pseudo label between the transcription factor and the target target is generated, and the first data and the second data are used as a large amount of unlabeled data (that is, data for which it is not determined whether there is a regulatory relationship) to input into the neural network model for training iteration to obtain a feature matrix reflecting whether each transcription factor-target target pair has a regulatory relationship, based on the obtained transcription factor-target pair with a regulatory relationship The target target pairs (i.e., small sample data) adjust the weights of the corresponding transcription factor-target target pairs in the feature matrix, introduce the contrastive learning framework in self-supervised learning, and use a large number of transcription factor-target target pairs for which it is not determined whether they have a regulatory relationship to perform self-supervised learning on a small number of transcription factor-target target pairs that have been determined to have a regulatory relationship. The weights of the transcription factor-target target pairs in the feature matrix are adjusted based on the learning results, and the prediction results of the gene regulatory network are obtained based on the feature matrix after adjusting the weights. The introduction of self-supervised learning can ensure the accuracy of the prediction of the gene regulatory network without the need for a large amount of labeled gene regulatory network data, and at the same time, avoid the high cost of annotating the gene regulatory network data.

[0049] In one embodiment, this embodiment provides a specific implementation method for improving the weight of the corresponding transcription factor-target pair in the feature matrix based on the transcription factor-target pair with a regulatory relationship:

[0050] Based on the transcription factor-target pairs with regulatory relationships and binary cross entropy, a simple linear encoder was constructed for training, and the transcription factor-target pairs with regulatory relationships in the feature matrix were annotated, and the weight of the transcription factor-target pairs in the feature matrix was increased;

[0051] In the above steps, since the gene data has multiple features, a large-scale feature matrix will be formed as the pre-training model. In the fine-tuning stage, a small-scale transcription factor-target pair with a regulatory relationship and a binary cross entropy (BCE) are used as the loss function to adjust whether the relationship between the transcription factor-target pair in the pre-training model belongs to regulation. Specifically, the expression of the binary cross entropy function is:

[0052]

[0053] Where N represents the number of samples, y i represents the label of a small-scale labeled sample i, p(y i ) represents the probability that sample i is predicted as a positive label;

[0054] A simple linear encoder based on ResMLP was constructed based on the binary cross entropy function, and the simple linear encoder was trained based on the pre-trained model and a small-scale transcription factor-target pair with a regulatory relationship. The linear encoder has a linear layer and the output dimension of the linear layer is 1, called the feature coefficient, denoted as α. Based on the binary classification task, all the output results of the linear encoder were thoroughly analyzed, and an appropriate threshold θ was established. When α≥θ, the output is classified as positive, proving that the transcription factor-target pair has a regulatory relationship. When α<θ, the output is classified as negative, proving that the transcription factor-target pair does not have a regulatory relationship. The value of θ is adjusted according to the experimental results to achieve the ideal expected effect.

[0055] In one embodiment, the specific task of the predetermined task provided in this embodiment is: assuming that each transcription factor in the first data and the second data is only correlated with the target target in each transcription factor-target target pair, but has no correlation with other target targets;

[0056] In the above steps, the transcription factor-target pairs in the first data and the second data are unlabeled data because it has not been determined whether they have a regulatory relationship. It is assumed that each transcription factor is only correlated with the target target in each transcription factor-target target pair and is established as the only positive sample, while each transcription factor has no correlation with the target target other than the transcription factor-target target as a negative sample, thereby establishing a pseudo-label between the transcription factor and the target target. Specifically, based on the methodology of contrastive learning, i.e., minimizing intra-class differences and maximizing inter-class differences, a predetermined task is established to minimize the intra-class differences between the transcription factor and the target target in each positive sample and maximize the inter-class differences between each positive sample. The predetermined task is different from the task set up in traditional supervised learning, which is "assuming that all transcription factor-target target pairs are unrelated and determining whether the current transcription factor-target target pair has a regulatory relationship." The predetermined task establishes a pseudo-label before determining the regulatory relationship between the transcription factor-target target pair, so that only fine-tuning is required later, thereby simplifying the prediction process.

[0057] In one embodiment, this embodiment provides a specific implementation method of inputting the first data and the second data after generating pseudo labels into a neural network model for training iteration to obtain a feature matrix including transcription factor-target pairs:

[0058] Setting a neural network model, dividing the first data and the second data after generating pseudo labels into d non-overlapping grids of length s, and inputting the grids into an embedding layer of the neural network model for dimensionality increase processing;

[0059] The neural network formula set in the above steps is:

[0060] y=f(x,b,W),

[0061] Among them, x is the input of the neural network, y is the output of the neural network, b is the learnable bias parameter, and W is the learnable network weight;

[0062] The sequence G of each transcription factor-target pair in the first data and the second data i Can be expressed as:

[0063]

[0064] Among them, g represents the number of genes, and n represents the sequence length of the gene expression corresponding to the transcription factor-target pair;

[0065] The first data and the second data are divided into d non-overlapping grids of length s, so as to correspond the subsequent order with the encoder blocks in the encoder.

[0066] Inputting the first data and the second data after the dimension increase processing into the encoder of the neural network model respectively to obtain a first embedding sequence and a second embedding sequence;

[0067] In the above steps, the first data and the second data after the dimension-upgrading process are respectively input into the encoder (i.e., the encoding layer) in the order of dividing the grid to obtain the first embedding sequence X1 (l) and the second embedding sequence X2 (l) The encoder is based on Transformer design, in which a multi-head attention mechanism is introduced. The encoder is specifically divided into three parts, including: linear projection layer, attention layer and linear output layer.

[0068] Inputting the first embedding sequence and the second embedding sequence into the projection layer of the neural network model for dimension-raising processing to obtain a first dimension-raising sequence and a second dimension-raising sequence, and performing gradient updating on the first dimension-raising sequence and the second dimension-raising sequence to obtain a first gradient sequence and a second gradient sequence;

[0069] In the above steps, the projection layer of the neural network model is a three-layer fully connected layer, including: input layer (input is 512 dimensions, output is 1024 dimensions), hidden layer (input is 1024 dimensions, output is 1024 dimensions), output layer (input is 1024 dimensions, output is 2048 dimensions), where the data of the input layer and hidden layer after the projection layer will be normalized and ReLU activated to improve the classification performance of the neural network model. (l) Input the projection layer to perform dimensionality increase processing to obtain the first dimensionality increase sequence sg(z1) of 2048 dimensions (l) , embed the second sequence X2 (l) Input the projection layer to perform dimensionality upscaling to obtain the second dimensional upscaling sequence sg(z2) of 2048 dimensions. (l) , input the first dimension-raising sequence into the feedforward neural network for gradient update (stop-gard, sg) to obtain the 2048-dimensional first gradient sequence z1 (l) , input the second dimension-increasing sequence into the feedforward neural network for gradient update to obtain the 2048-dimensional second gradient sequence z2 (l) ;

[0070] Among them, the sequence X1 (l) , X2 (l) 、sg(z1) (l) 、sg(z2) (l) 、z1 (l) and z2 (l) They are all vector sequences. Specifically, the first dimension-upgraded sequence is the result of the first embedding sequence being input to the projection layer without gradient updating, and the first gradient sequence is the result of the first embedding sequence being input to the projection layer and gradient updating; the second dimension-upgraded sequence is the result of the second embedding sequence being input to the projection layer without gradient updating, and the second gradient sequence is the result of the second embedding sequence being input to the projection layer and gradient updating.

[0071] Inputting the first gradient sequence and the second gradient sequence into the prediction layer of the neural network model to obtain a first output sequence and a second output sequence, and obtaining a loss function based on the first dimension-raising sequence, the second dimension-raising sequence, the first output sequence, and the second output sequence;

[0072] The prediction layer of the neural network model in the above steps is a two-layer fully connected layer, specifically including: input layer (input is 2048 dimensions, output is 512 dimensions), output layer (input is 512 dimensions, output is 2048 dimensions), the first gradient sequence z1 (l) Input to the prediction layer to get the first output sequence p1 of 2048 dimensions (l) , the second gradient sequence z2 (l) Input to the prediction layer and get the second output sequence p2 of 2048 dimensions (l) , based on sg(z1) (l) 、sg(z2) (l) 、p1 (l) and p2 (l) The sequence obtains the loss function Loss(p i ,z j ), where sequence p1 (l) and p2 (l) Also a sequence of vectors.

[0073] Perform multiple iterations to minimize the loss function and obtain the feature matrix of the transcription factor-target pair;

[0074] In the above steps, multiple iterations are performed based on the neural network model to minimize the loss function. In the iterative process, the AdamW optimizer is combined with the automatic mixed precision AMP library to speed up the fitting speed and reduce GPU memory consumption.

[0075] In one embodiment, this embodiment provides a specific implementation method of inputting the embedding layer of the neural network model for dimensionality increase processing:

[0076] The first data and the second data, which are divided into d non-overlapping grids of length s, are input into the embedding layer for linear projection and dimensionality increase processing is performed through the embedding weight matrix; wherein the expression of the embedding weight matrix is:

[0077]

[0078] In the expression of embedding weight matrix: E represents the embedding weight matrix, is a set of real numbers, s represents the length of the partition grid, g represents the number of genes, and h represents the number of hidden layers in the embedding layer;

[0079] In the above steps, the embedding layer is a three-layer fully connected layer, specifically including: an input layer, a hidden layer and an output layer, wherein the data of the input layer and the hidden layer passing through the embedding layer will be normalized and ReLU activated, and the above embedding weight matrix E can be optimized through training.

[0080] In one embodiment, this embodiment provides a specific implementation method of inputting the first data and the second data after the dimension increase processing into the encoder of the neural network model to obtain the first embedding sequence and the second embedding sequence:

[0081] Inputting the grid segments of the first data and the second data after the dimension-upgraded processing into the respective encoder blocks in the encoder of the neural network model;

[0082] In the above steps, the d grid segments of the first data and the d grid segments of the second data after dimension upgrading are respectively input into each encoder block in sequence, and each grid segment of the first data and the second data can be represented as a matrix:

[0083]

[0084] Among them, W O is the representation matrix of the grid segments, d represents the number of grids or grid segments, and h represents the number of hidden layers of the embedding layer, which is 512 in this embodiment.

[0085] Based on the multi-head attention mechanism in each encoder block, the grid segments input into each encoder block are subjected to attention weight learning;

[0086] In the above steps, the multi-head attention mechanism in each encoder block is used to process the grid segments input to each encoder so that it learns different attention weights. Specifically, the multi-head attention mechanism processing formula can be expressed as:

[0087] MH(X (l) )=Concat(x1,x2,...,x d )W O ,

[0088] Among them, MH is the abbreviation of Multi-Head, which represents the multi-head attention mechanism, and X (l) is the input sequence of the first data or the second data, Concat(x1,x2,...,x d ) represents the output of the attention head corresponding to each grid segment.

[0089] The grid segments in each encoder block after attention weight learning are input into the feedforward neural network for gradient update;

[0090] In the above steps, the grid segments after learning the attention weights in each encoder block are input into the feedforward neural network for gradient update. The feedforward neural network consists of two linear layers, a GELU activation function and a ReLU activation function. The formula for gradient update based on the feedforward neural network can be expressed as:

[0091] ReLU(x a )=max(0,x a ),

[0092]

[0093] FFN(MH(X (l) ))=ReLU(W2GELU(W1MH(X (l) ))+b2)+b1,

[0094] Among them, ReLU represents the ReLU activation function, x a is the input of the ReLU activation function, GELU represents the GELU activation function, x b is the input of the GELU activation function, h represents the number of hidden layers of the embedding layer, FFN represents the position feedforward function in the feedforward neural network, W1 is the first learnable weight matrix, W2 is the second learnable weight matrix, b1 is the first learnable bias term, and b2 is the second learnable bias term.

[0095] Add residual connections to the grid segments in each encoder block after gradient update and perform normalization to obtain a first embedding sequence and a second embedding sequence;

[0096] In the above steps, adding residual connections to the grid segments and normalizing them can obtain the output of the encoder at layer l, which can be expressed as:

[0097] X (l+1) =X (l) +LayerNorm(FFN(MH(X (l) ))),

[0098]

[0099] Among them, X (l+1) is the output of the encoder at layer l+1, LayerNorm is the normalization function z-score, x i,t represents the expression value of gene i at time t, represents the average expression value of gene i at all times, σ i Represents the standard deviation of the expression values ​​of gene i at all times.

[0100] In one embodiment, this embodiment provides a specific implementation of obtaining a loss function based on the first dimension-raising sequence, the second dimension-raising sequence, the first output sequence, and the second output sequence:

[0101] Calculating the cosine similarity between the first dimension-raising sequence and the second output sequence to obtain a first cosine similarity, and calculating the cosine similarity between the second dimension-raising sequence and the first output sequence to obtain a second cosine similarity;

[0102] The formula for solving the cosine similarity in the above steps is as follows:

[0103]

[0104] Among them, D(p i ,z j ) represents the cosine similarity between the dimension-raising sequence and the output sequence, sg(z j ) is a dimensional sequence, p i is the output sequence, and ||·||2 represents L2 normalization (L2-Norm), which is equivalent to the mean square error of the L2-norm vector.

[0105] A loss function is obtained based on the sum of half of the first cosine similarity value and half of the second cosine similarity value;

[0106] The loss function in the above steps is defined as follows:

[0107]

[0108] The loss function is obtained based on the sum of half of the first cosine similarity value and half of the second cosine similarity value.

[0109] In one embodiment, this embodiment provides a specific implementation method for obtaining transcription factor-target pair data as the first data:

[0110] Obtain gene expression data, determine the top N gene data with the largest variance in the gene expression data, and normalize the gene data;

[0111] The gene expression data in the above steps is the expression value matrix of genes during biological development, which implies the regulatory relationship between genes. Therefore, obtaining the data of the top N genes with the largest variance can indirectly analyze the regulatory relationship between relatively discrete genes.

[0112] Obtaining the transcription factor-target pair corresponding to each gene in the normalized gene data as the first data;

[0113] In the above steps, the genetic data are unified to the same scale based on normalization processing, thereby improving the accuracy and efficiency of subsequent analysis.

[0114] In one embodiment, this embodiment provides a specific implementation of data enhancement processing, including: interpolation processing, noise introduction processing, scaling processing, contrast adjustment and discarding processing;

[0115] The specific implementation of the data enhancement processing in the above steps includes:

[0116] See Figure 2a Gene data: The normalized gene data is the gene expression matrix of the transcription factor-target pair;

[0117] See Figure 2b Interpolation processing: Considering that time series data can maintain causal relationships by slightly adjusting the sequence length, n gene data are randomly selected from the original data for replication, and the replicated n gene data are inserted into the previous position between consecutive gene data to form interpolated gene data;

[0118] See Figure 2c Noise treatment: simulates the situation where the genetic data is biased by estimation or annotation, so the sequence G of the transcription factor-target pair i (G TF ,G target ) Gaussian noise N(0, 0.2) with a dropout rate of 0.2 was applied to reduce the noise sensitivity of the genetic data;

[0119] See Figure 2d Scaling: represents the change of the sequence of the transcription factor-target pair. A scaling factor between (0.6, 1.4) is randomly selected and applied to the sequence of the transcription factor-target pair, changing the steepness of the transcription factor-target pair sequence.

[0120] See Figure 2e Contrast adjustment: according to the sequence G of the transcription factor-target pair i Mean(G i ) is used to select a scaling factor between (0.6, 1.4) to adjust the contrast of the sequence, where the mean value Mean(G i ) the higher the value, the larger the scaling factor.

[0121] See Figure 2f Discarding: To simulate the situation of naturally missing values ​​in the sequence of transcription factor-target pairs, some gene data were randomly removed, and the deleted values ​​ranged from 0% to 80% of the entire sequence, which enabled the gene data to still identify characteristic relationships in the presence of missing values.

[0122] This embodiment also provides a gene regulatory network prediction system for executing the above gene regulatory network prediction method.

[0123] In the embodiment of the present invention, specific data enhancement or a combination thereof ensures the pattern invariance of the gene sequences in the first data and the second data, so that the same sequence G i The similar patterns are maintained in the data, and the pattern invariance of the first data and the second data is guaranteed through data enhancement, so that the inherent similarity makes the representation learning more efficient and effective in the comparison and encoding interaction mode.

[0124] In the embodiment of the present invention, the characteristics of self-supervised learning and small sample learning are combined, so that the regulatory labels of transcription factor-target pairs can be generated automatically, thereby eliminating the need for manual labeling of regulatory labels and reducing costs. At the same time, a multi-head attention mechanism is introduced in the prediction process to improve the prediction method's attention to gene expression levels at different time points, thereby improving the ability of gene regulatory network prediction. The expression and regulatory mechanism of genes in organisms can be studied, which is of great significance for revealing the complex molecular interactions in life systems, and has broad application prospects in crop breeding, disease target analysis and other fields.

[0125] Based on the above embodiments, Figure 3 As shown, this embodiment provides an example of using the above-mentioned gene regulatory network prediction method to predict a gene regulatory network, which can be specifically performed with reference to the following steps:

[0126] S201, obtaining gene expression data, determining the top N gene data with the largest variance in the gene expression data, normalizing the gene data and obtaining the transcription factor-target pair corresponding to each gene as the first data, specifically, the gene expression data comes from three public gene expression data sets of corn seeds, ovules, and nucellus;

[0127] S203, inputting the first data into a data enhancement layer for interpolation processing, noise introduction processing, scaling processing, contrast adjustment and discarding processing to obtain second data;

[0128] S205, according to the self-supervised learning paradigm, construct a predetermined task: "Assume that all transcription factor-target pairs are correlated, and determine whether the correlation belongs to regulation", and make correlation assumptions for the transcription factor-target pairs in the first data and the second data according to the predetermined task, and assume that each transcription factor in the first data and the second data has a relationship only with the target target in the corresponding transcription factor-target pair as a positive sample, and has no relationship with other target targets as a negative sample, thereby completing the generation of pseudo labels;

[0129] S207, dividing the first data and the second data after pseudo-label generation into d non-overlapping grids of length s respectively, and inputting the grids into the embedding layer of the neural network model for dimensionality increase processing;

[0130] S209, inputting the first data and the second data after the dimension increase processing into the encoder of the neural network model respectively to obtain a first embedding sequence and a second embedding sequence;

[0131] S211, inputting the first embedded sequence and the second embedded sequence into the projection layer of the neural network model for dimensionality increase processing to obtain a first dimensionality increase sequence and a second dimensionality increase sequence, and performing gradient update on the first dimensionality increase sequence and the second dimensionality increase sequence to obtain a first gradient sequence and a second gradient sequence;

[0132] S213, inputting the first gradient sequence and the second gradient sequence into the prediction layer of the neural network model to obtain a first output sequence and a second output sequence, and obtaining a loss function based on the first dimension-raising sequence, the second dimension-raising sequence, the first output sequence, and the second output sequence;

[0133] S215, performing multiple iterations to minimize the loss function, obtaining a feature matrix of the transcription factor-target pair as a pre-training model, and dividing the pre-training model into a training set, a validation set, and a test set, and the ratio of training set: validation set: test set = 3:1:1;

[0134] S217, obtaining gene regulatory network data and identifying transcription factor-target pairs with regulatory relationships therein, wherein the gene regulatory network data comes from public regulatory networks of different databases, namely STRING, RegMap, FunTBS, Motfi_CE, Motfi-Zma and Merged-Zma;

[0135] S219, a simple linear encoder based on ResMLP was constructed based on the binary cross entropy function, and the simple linear encoder was trained based on a small-scale transcription factor-target pair with regulatory relationships and the pre-trained model in the training set;

[0136] S221, verify and adjust the parameters of the simple linear encoder through the pre-trained model in the validation set, and retain the corresponding parameters when the pre-trained model with the most accurate prediction is obtained during the iteration process. The pre-trained model in the test set is tested, and two error evaluation criteria are used to analyze the feasibility and effectiveness, namely ROC curve integral AUROC and PR curve integral AUPRC, and the formulas are:

[0137]

[0138] Among them, Recall is the True Positive Rate, also known as sensitivity or recall, and is calculated as TP is the number of true positive examples, FN is the number of false negative examples; FPR is the false positive rate (False Positive Rate), which is calculated as follows: Where FP is the number of false positive examples, TN is the number of true negative examples; Precision is the accuracy, calculated as

[0139] In the embodiment of the present invention Figure 4a , Figure 4b and Figure 4c The gene ontology (GO) IDs and loci represented by the serial numbers are shown in the following Table 1 as an example.

[0140] Table 1 Figure 4a , Figure 4b and Figure 4c The ID and locus of the gene represented by the serial number

[0141]

[0142]

[0143]

[0144]

[0145]

[0146]

[0147] Among them, the superscript 1 of the enrichment number in the above Table 1 represents the number of proteins annotated with specific terms in the protein-protein interaction (PPI) network.

[0148] In the embodiment of the present invention, see Figure 4a , Figure 4b and Figure 4c The protein sets expressed by the three key genes in corn crops are called loci. These genes are considered to be related to the seed development and inflorescence development of corn crops. The color depth of each node in the figure represents the in-degree of each node. The darker the node color, the higher the in-degree of the node. Figure 4a , Figure 4b and Figure 4cThe dotted arrows in the figure represent predicted associations, the darker solid arrows represent existing associations, and the lighter solid arrows represent verification of the associations. The protein sets expressed by the three key genes in corn crops are analyzed by using protein interaction networks for verification. These genes are paired with all other genes as basic input pairs, and predictions are made based on the prediction method for the gene regulatory network provided in the embodiment of the present invention to obtain prediction results. Specifically, Figure 4a From the GO perspective of biological processes, there is evidence that this regulatory network is related to proteins expressed in the development of flowers and seeds, such as floral whorl structural organization (GO:0048459), seed growth (GO:0080112), and floral organ number (GO:0048833).

[0149] In the embodiment of the present invention, Figure 4d for Figure 4a Two details of Figure 4d The details in the upper middle section show that the gene ontology represented by node 6 is predicted to have a regulatory relationship with the gene ontology represented by node 19, and the existing regulatory relationship between the gene ontology represented by node 6 and the gene ontology represented by node 19 is verified; Figure 4d The details in the middle and lower part show that the gene ontology represented by node 6 is predicted to have a regulatory relationship with the gene ontology represented by node 18, and the existing regulatory relationship between the gene ontology represented by node 6 and the gene ontology represented by node 18 is verified; through the prediction of the protein interaction network, it is found that Figure 4a , Figure 4b and Figure 4c Among the prediction results, there are associations that are verified by existing regulatory networks and exemplified by evidence from gene ontology.

[0150] In the embodiment of the present invention, see Figure 5a , Figure 5b , Figure 5c and Figure 5d As shown in FIG. 1 , INVEST is the index result of the prediction method provided by the embodiment of the present invention after fine-tuning the network under different scale data. The color depth of the table in the figure represents the index size of each prediction method. The larger the index, the darker the color. Based on the color depth of the table and the data in the table, it can be seen that the prediction method provided by the embodiment of the present invention generally has high accuracy in the prediction process. Only the index in some scale data is slightly lower than that of other methods. For example, see Figure 5aThe AUROC index of the prediction method provided by the embodiment of the present invention on the mSacOvule2000 dataset is only slightly lower than that of the CNNC method. Figure 5a On the mSeed1000 dataset after fine-tuning the RegMap network, INVEST achieves a 5.56% improvement in AUROC score compared to the second best method GENELink. Figure 5b In the mSeed1000 dataset after fine-tuning the RegMap network, INVEST improves the AUPRC score by 31.25% compared with the second best CNNC method. Figure 5c In the mSacOvule2000 dataset fine-tuned by the STRING network, INVEST's AUROC score increased by 3.67% compared with the second best CNNC method. Figure 5d In the paper, on the mSacOvule2000 dataset after fine-tuning the STRING network, INVEST's AUPRC score increased by 97.81% compared with the second best STRINGS method.

[0151] The embodiment of the present invention further provides a prediction system for a gene regulatory network, comprising a data acquisition module, a data enhancement module, a data prediction module and a performance evaluation module, wherein:

[0152] Data acquisition module, to obtain gene expression data and gene regulatory network data;

[0153] A data enhancement module performs data enhancement processing on the first data;

[0154] A data prediction module, which sends the first data and the second data into a neural network model for training iteration to obtain a feature matrix including transcription factor-target pairs, and fine-tunes the feature matrix based on gene regulatory network data to obtain a prediction result of the gene regulatory network;

[0155] The performance evaluation module evaluates the prediction results based on the ROC curve integral AUROC and the PR curve integral AUPRC.

[0156] In addition, in the description of the embodiments of the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0157] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0158] In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance.

[0159] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A method for predicting a gene regulatory network, characterized in that: include: Acquiring transcription factor-target point pair data as first data, performing data enhancement processing on the transcription factor-target point pair data, and obtaining processed transcription factor-target point pair data as second data; The first data and the second data are assumed to be correlated with the transcription factor and the target point based on the predetermined task, and pseudo labels of the transcription factor and the target point are generated. The first data and the second data after the pseudo labels are generated are input into the neural network model for training iteration to obtain a feature matrix including transcription factor-target target pairs; wherein the feature matrix reflects whether each transcription factor-target target pair has a regulatory relationship; Acquire gene regulatory network data, identify transcription factor-target pairs having regulatory relationships therein, and adjust the weights of corresponding transcription factor-target pairs in the feature matrix based on the transcription factor-target pairs having regulatory relationships; Based on the feature matrix after adjusting the weight, it is judged whether the transcription factor-target point pair has a regulatory relationship, and the prediction result of the gene regulatory network is obtained.

2. The method for predicting a gene regulatory network according to claim 1, characterized in that: The step of increasing the weight of the corresponding transcription factor-target pair in the feature matrix based on the transcription factor-target pair having a regulatory relationship comprises: Based on the transcription factor-target pairs with regulatory relationships and binary cross entropy, a simple linear encoder is constructed for training, and the transcription factor-target pairs with regulatory relationships in the feature matrix are labeled, and the weight of the transcription factor-target pairs in the feature matrix is ​​increased.

3. The method for predicting a gene regulatory network according to claim 1, characterized in that: The predetermined task is: assuming that each transcription factor in the first data and the second data is only correlated with the target target in each transcription factor-target target pair, but has no correlation with other target targets.

4. The method for predicting a gene regulatory network according to claim 1, characterized in that: The step of inputting the first data and the second data after generating pseudo labels into a neural network model for training iteration to obtain a feature matrix including transcription factor-target site pairs comprises: Setting a neural network model, dividing the first data and the second data after generating pseudo labels into d non-overlapping grids of length s, and inputting the grids into an embedding layer of the neural network model for dimensionality increase processing; Inputting the first data and the second data after the dimension increase processing into the encoder of the neural network model respectively to obtain a first embedding sequence and a second embedding sequence; Inputting the first embedded sequence and the second embedded sequence into the projection layer of the neural network model for dimension-raising processing to obtain a first dimension-raising sequence and a second dimension-raising sequence, and performing gradient updating on the first dimension-raising sequence and the second dimension-raising sequence to obtain a first gradient sequence and a second gradient sequence; Inputting the first gradient sequence and the second gradient sequence into the prediction layer of the neural network model to obtain a first output sequence and a second output sequence, and obtaining a loss function based on the first dimension-raising sequence, the second dimension-raising sequence, the first output sequence, and the second output sequence; Multiple iterations are performed to minimize the loss function and obtain the feature matrix of the transcription factor-target pair.

5. The method for predicting a gene regulatory network according to claim 4, characterized in that: The step of inputting the embedding layer of the neural network model for dimensionality upgrading includes: The first data and the second data, which are respectively divided into d non-overlapping grids of length s, are input into the embedding layer for linear projection, and are subjected to dimensionality increase processing by an embedding weight matrix; wherein the expression of the embedding weight matrix is: In the expression of the embedding weight matrix: E represents the embedding weight matrix, is a set of real numbers, s represents the length of the partitioned grid, g represents the number of genes, and h represents the number of hidden layers in the embedding layer.

6. The method for predicting a gene regulatory network according to claim 4, characterized in that: The step of inputting the first data and the second data after the dimension increase processing into the encoder of the neural network model to obtain the first embedding sequence and the second embedding sequence includes: Inputting the grid segments of the first data and the second data after the dimension increase processing into the respective encoder blocks in the encoder of the neural network model respectively; Based on the multi-head attention mechanism in each encoder block, the grid segments input into each encoder block are subjected to attention weight learning; Inputting the grid segments in each encoder block after attention weight learning into a feedforward neural network for gradient updating; A residual connection is added to the grid segments in each of the encoder blocks after gradient updating and normalization is performed to obtain the first embedding sequence and the second embedding sequence.

7. The method for predicting a gene regulatory network according to claim 4, characterized in that: The step of obtaining a loss function based on the first dimension-raising sequence, the second dimension-raising sequence, the first output sequence, and the second output sequence comprises: Calculating the cosine similarity between the first dimension-raising sequence and the second output sequence to obtain a first cosine similarity, and calculating the cosine similarity between the second dimension-raising sequence and the first output sequence to obtain a second cosine similarity; The loss function is obtained based on a sum of half of the first cosine similarity value and half of the second cosine similarity value.

8. The method for predicting a gene regulatory network according to claims 1-7, characterized in that: The step of obtaining transcription factor-target point pair data as first data comprises: Acquire gene expression data, determine top N gene data with the largest corresponding variance in the gene expression data, and normalize the gene data; The transcription factor-target site pair corresponding to each gene in the normalized gene data is obtained as the first data.

9. The method for predicting a gene regulatory network according to claim 8, characterized in that: The data enhancement processing includes: interpolation processing, noise introduction processing, scaling processing, contrast adjustment and discarding processing.

10. A prediction system for gene regulatory networks, characterized in that: A method for predicting a gene regulatory network according to any one of claims 1 to 9.

Images