Training method of multi-task gene regulation network model based on knowledge graph embedding and feature fusion

Abstract

The application relates to the technical field of biological information, in particular to a multi-task gene regulation network model training method based on knowledge graph embedding and feature fusion. Important information hidden in gene sequences is captured from three aspects of sequence context perception representation, physical and chemical properties and pseudo dinucleotide composition, so that the richness of feature embedding is improved; a bidirectional long short-term memory network is used to splice learning of the extracted features to reduce the coding difficulty of high-throughput gene sequences; two learning task modules with different focuses are designed, which are respectively focused on node features and edge features, and more comprehensive feature utilization is realized through multi-task learning; a gradient normalization algorithm is used to dynamically balance the losses of the two tasks, so that the overall learning efficiency and stability of the model are improved. The application aims to solve the problem of how to improve the prediction accuracy of the regulation relationship of the gene regulation network.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of biological information, in particular to a multi-task gene regulation network model training method based on knowledge graph embedding and feature fusion. BACKGROUND

[0002] The rise of knowledge graphs and deep learning algorithms has opened up new ways to integrate different biological data. At present, there are related literatures (Luo J, Ouyang W, Shen C, et al. Multi-Relation Graph Embedding for Predicting miRNA-Target Gene Interactions by Integrating Gene Sequence Information [J]. IEEE Journal of Biomedical and Health Informatics, 2022, 26:4345-4353.) that propose a model based on multi-relation graph embedding to predict the interaction between miRNA and target genes. Through the combination of multi-relation graph convolution and bidirectional long short-term memory (Bi-LSTM), the prediction accuracy of miRNA-target gene interaction is improved.

[0003] However, the inventors found in the process of conceiving and implementing the present application that the current main gene regulation network inference model focuses on the extraction of node features, while the extraction of edge features is often ignored. The edge in the gene regulation network represents the regulation relationship between genes, and the features of the edge can not only reflect the direct interaction between genes, but also reveal important information such as regulation type and mechanism. If the extraction of edge features is ignored, the model may not be able to fully capture the complex regulation relationship between genes when inferring the gene regulation network, thereby affecting the accuracy of network inference.

[0004] Therefore, the present application proposes a new gene regulation network inference method to realize efficient and accurate regulation relationship prediction. SUMMARY

[0005] The main purpose of the present application is to provide a multi-task gene regulation network model training method based on knowledge graph embedding and feature fusion, which aims to solve the problem of how to improve the accuracy of regulation relationship prediction of gene regulation network.

[0006] To achieve the above purpose, the present application provides a multi-task gene regulation network model training method based on knowledge graph embedding and feature fusion, which comprises:

[0007] encoding each base in an initial gene sequence into a fixed-length binary vector and constituting a two-dimensional matrix, obtaining a physical and chemical property value associated with each dinucleotide unit in the initial gene sequence, and determining a pseudo-dinucleotide group feature according to the frequency of occurrence of dinucleotides in the initial gene sequence;

[0008] learning the physical and chemical property values and the pseudo-dinucleotide group feature of the initial gene sequence using a bidirectional long short-term memory network, splicing the learned features and the two-dimensional matrix as the embedding of each node in a knowledge graph to obtain a training knowledge graph dataset;

[0009] inputting the training knowledge graph dataset into a deep learning model for training, and introducing a gradient normalization algorithm to optimize the loss balance of the deep learning model during training, wherein the deep learning model includes a multi-channel graph convolution layer for learning the node features of the knowledge graph embedding, and a multi-channel dynamic graph convolution layer for learning the edge features of the knowledge graph embedding.

[0010] Optionally, the calculation expression of the pseudo-dinucleotide group feature is:

[0011]

[0012] wherein, is a pseudo-dinucleotide group feature, is the frequency of occurrence of different dinucleotides, and w is a weight factor, is the number of pseudo-components, represents a correlation factor between the jth adjacent dinucleotides.

[0013] Optionally, the calculation expression of the correlation factor is:

[0014]

[0015] wherein, is a correlation factor, and L is the length of the sequence, is a physical and chemical property value, and PC is an identifier of each physical and chemical property value of the dinucleotide, represents a dinucleotide starting from the ith nucleotide of the sequence, represents a dinucleotide starting from the i+jth nucleotide of the sequence.

[0016] Optionally, the step of introducing a gradient normalization algorithm to optimize the loss balance of the deep learning model includes:

[0017] updating the loss value of the deep learning model after each training using the following formula:

[0018]

[0019] wherein, is an updated loss value, is an expression representing a gradient L2 norm of the single-task loss used in the weight layer w at training time t, is an average gradient norm of the gradient L2 norms corresponding to all tasks, is a relative inverse learning rate of task i, is a hyperparameter for setting the strength of adjustment;

[0020] wherein:

[0021]

[0022]

[0023] wherein, is a loss function of task i, is a learning function.

[0024] Optionally, the calculation expression of the relative inverse learning rate is:

[0025]

[0026] wherein, is a loss rate of task i at time t, is an average loss rate of all tasks;

[0027] wherein:

[0028]

[0029] wherein, is a single-task loss, is an initial task loss.

[0030] In addition, to achieve the above-mentioned purposes, the application also provides a multi-task learning model obtained by a training method of any one of the multi-task gene regulation network models based on knowledge graph embedding and feature fusion.

[0031] Optionally, the multi-task learning model comprises:

[0032] a multi-channel graph convolution layer for learning node features of the knowledge graph embedding;

[0033] a multi-channel dynamic graph convolution layer for learning edge features of the knowledge graph embedding.

[0034] Optionally, the number of layers of the multi-channel graph convolution layer and the multi-channel dynamic graph convolution layer is 2.

[0035] Optionally, the embedding dimension of the multi-channel graph convolution layer is 1024, and the embedding dimension of the multi-channel dynamic graph convolution layer is 128.

[0036] In addition, to achieve the above object, the present application also provides a computer system, comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the computer program implements the steps of the training method of the multi-task gene regulation network model based on knowledge graph embedding and feature fusion when executed by the processor.

[0037] The present application has at least the following beneficial effects:

[0038] 1. The important information implied in the gene sequence is captured from three angles of sequence context-aware representation, physical and chemical properties, and pseudo-dinucleotide composition, which improves the richness of feature embedding;

[0039] 2. The extracted features are spliced and learned by using a bidirectional long short-term memory network to reduce the coding difficulty of high-throughput gene sequences;

[0040] 3. Two learning task modules with different focuses are designed, which are respectively focused on node features and edge features, and a more comprehensive feature utilization is realized through multi-task learning;

[0041] 4. Gradient normalization algorithm is used to dynamically balance the losses of the two tasks, thereby improving the overall learning efficiency and stability of the model. BRIEF DESCRIPTION OF DRAWINGS

[0042] Figure 1 The flowchart of the training method of the multi-task gene regulation network model based on knowledge graph embedding and feature fusion involved in the embodiments of the present application;

[0043] Figure 2 The influence diagram of different numbers of layers on the multi-channel dynamic graph convolution module involved in the embodiments of the present application;

[0044] Figure 3 The influence diagram of different numbers of layers on the multi-channel graph convolution network layer module involved in the embodiments of the present application;

[0045] Figure 4 The AUCROC graph involved in the embodiments of the present application;

[0046] Figure 5 The result analysis comparison diagram involved in the embodiments of the present application;

[0047] Figure 6 The result analysis comparison diagram of the KGE-MFGRN model and the conventional model involved in the embodiments of the present application;

[0048] Figure 7 The architecture diagram of the hardware running environment of the computer system involved in the embodiments of the present application.

[0049] The implementation, functional features and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0050] In order to better understand the above technical solutions, the exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided so that the present disclosure can be more thoroughly understood and the scope of the present disclosure can be fully conveyed to those skilled in the art.

[0051] First embodiment

[0052] Referring to Figure 1 The present embodiment provides a training method of a multi-task gene regulation network model based on knowledge graph embedding and feature fusion, which comprises the following steps:

[0053] S10, encode each base in the initial gene sequence into a fixed-length binary vector and form a two-dimensional matrix, obtain the physical and chemical property value associated with each dinucleotide unit in the initial gene sequence, and determine the pseudo-dinucleotide group feature according to the frequency of occurrence of dinucleotide in the initial gene sequence;

[0054] In the present embodiment, the bases (including A, T, C, G of DNA, or A, U, G, C of RNA) in the gene sequence are respectively mapped to a fixed-length binary vector with a length of, so as to convert the bases in the gene sequence into a form acceptable by the machine learning model.

[0055] It should be noted that by converting each base in the sequence into the corresponding code one by one, a two-dimensional matrix representation is finally formed as the (1) below perception representation of the gene sequence. Both the integrity of the sequence is preserved, and the position and type of each base are clearly presented. This method is simple and intuitive, which directly numericalizes the base information without information loss, and completely preserves the sequence of all bases of the original sequence.

[0056] In some optional embodiments, the fixed length can be 4, and the gene sequence can be encoded in the form of One-hot encoding. Specifically, A is represented as [1, 0, 0, 0], T / U is represented as [0, 1, 0, 0], C is represented as [0, 0, 1, 0], and G is represented as [0, 0, 0, 1]. In addition, when an ambiguous base (usually represented by N) appears in the sequence, a full 0 vector [0, 0, 0, 0] is used to represent it.

[0057] In this embodiment, the physical-chemical properties (PCPs) value of a dinucleotide unit in a gene sequence refers to the molecular force characteristics of the dinucleotide unit, which is used to reflect the structural and functional characteristics of the gene sequence.

[0058] In some optional embodiments, the following 10 physical-chemical properties in Table 1 are used:

[0059] Table 1. Correspondence between physical-chemical properties and identifiers

[0060]

[0061] In this embodiment, the values of the 10 properties corresponding to each dinucleotide of DNA and RNA are listed in Tables 2 and 3:

[0062] Table 2. Correspondence between physical-chemical property values of DNA nucleotides

[0063]

[0064] Table 3. Correspondence between physical-chemical property values of RNA nucleotides

[0065]

[0066] In this embodiment, the physical-chemical property values of the dinucleotides of DNA and RNA are pre-set, and can be directly obtained by looking up the table when needed.

[0067] In this embodiment, the PseDNC feature refers to a method for describing a gene sequence, which reflects the correlation between adjacent dinucleotides in the sequence by introducing pseudo-components, thereby capturing more abundant sequence information than simple dinucleotide frequency. PseDNC can effectively extract local and global feature patterns in the sequence and improve the representation ability of sequence characteristics.

[0068] In some optional embodiments, the calculation expression of the pseudo-dinucleotide is:

[0069]

[0070] wherein, is the pseudo dinucleotide group feature, is the occurrence frequency of different dinucleotides, w is the weight factor, is the number of pseudo components, represents the correlation factor between the jth adjacent dinucleotides.

[0071] Further, the calculation expression of the correlation factor is:

[0072]

[0073] wherein, is the correlation factor, L is the sequence length, is the physical and chemical property value, PC is the identifier of each physical and chemical property value of the dinucleotide, represents the dinucleotide starting from the ith nucleotide of the sequence, represents the dinucleotide starting from the i+jth nucleotide of the sequence.

[0074] S20, using a bidirectional long short-term memory network to learn the physical and chemical property values of the initial gene sequence and the pseudo dinucleotide group features, splicing the learned features and the two-dimensional matrix as the embedding of each node in the knowledge graph to obtain a training knowledge graph dataset;

[0075] Further, in the present embodiment, when the length of the high-throughput gene sequence is too long, it is easy to cause the algorithmic cost and coding difficulty of gene coding to be too large, therefore, in the present embodiment, the dependency relationship in the coding process is further learned by using a bidirectional long short-term memory network, and the dimension of the features is compressed.

[0076] In some optional embodiments, autocovariance and cross-covariance are used to splice the physical and chemical features and the pseudo dinucleotide features.

[0077] Specifically, first, the physical and chemical property values of the dinucleotide are constructed into a PCP matrix:

[0078]

[0079] wherein, is the ith physical and chemical property value of the dinucleotide , m is the PCP sequence number, and L is the sequence length;

[0080] Next, according to the autocovariance and the cross-covariance The correlation of any two nucleotides is calculated to complete the splicing of two features:

[0081]

[0082]

[0083] wherein, represents the mean value of the i-th row data in the PCP matrix, m is the PCP number, and k represents the interval of two nucleotides, , represents two different PCPs.

[0084] Further, after the training data set is constructed, it is input into a deep learning model with a specific architecture for training, which will be described in subsequent embodiments and will not be repeated here.

[0085] S30, input the training knowledge graph data set into the deep learning model for training, and introduce a gradient normalization algorithm to optimize the loss balance of the deep learning model during the training process, wherein the deep learning model includes a multi-channel graph convolution layer for learning the node features of the knowledge graph embedding, and a multi-channel dynamic graph convolution layer for learning the edge features of the knowledge graph embedding.

[0086] In this embodiment, the deep learning model needs to perform two different learning tasks: learning the global structure information between nodes in the knowledge graph, and learning the features of each edge in the knowledge graph, so as to predict whether the regulatory relationship between nodes exists. But different learning tasks may have different loss functions and gradients, which leads to the fact that the gradient updates of different tasks may have an unbalanced impact on the model parameters during the training process. For example, if the loss function gradient of a task is larger, it may dominate in updating the model parameters, while the gradient updates of other tasks may be ignored.

[0087] In order to overcome this problem, a gradient normalization algorithm is introduced in this embodiment to dynamically adjust the gradients of different tasks to ensure that all tasks can be updated at a similar rate during the training process, thereby improving the overall performance and generalization ability of the model.

[0088] Specifically, for the loss function GradNorm aims to dynamically adjust the gradient norm by learning the function First, the relevant parameters are described as follows:

[0089]

[0090] where W represents a weight layer shared by all tasks, where L denotes the single-task loss used by the W layer at training time t gradient L2 norm.

[0091]

[0092] where, is the average gradient norm across all tasks. For each module, GradNorm computes the various training rates as follows:

[0093]

[0094]

[0095] where, is the loss rate of task i at time t, is the average loss rate across all tasks, is the relative inverse training rate of task i, which is used to balance the gradients. The higher the value of the higher the weight of the loss of task i should be. Finally, GradNorm computes as a loss specifically for updating which is defined as follows:

[0096]

[0097] where, is a hyperparameter that sets the strength of the adjustment.

[0098] In the technical scheme provided in the embodiment, three different feature extraction methods are used to capture important information implied in the gene sequence from three angles of sequence context perception representation, physical and chemical properties, and pseudo dinucleotide composition, thereby improving the richness of feature embedding; a bidirectional long short-term memory network is adopted to splice and learn the extracted features to reduce the coding difficulty of high-throughput gene sequences; secondly, the model designs two different learning task modules focusing on node features and edge features respectively, and realizes more comprehensive feature utilization through multi-task learning; finally, the model adopts a gradient normalization algorithm to dynamically balance the losses of the two tasks, thereby improving the overall learning efficiency and stability of the model.

[0099] Second embodiment

[0100] As an implementation scheme, in the embodiment, a multi-task learning model is trained based on the method proposed in the first embodiment.

[0101] In the embodiment, the knowledge graph is divided into different subgraphs according to relationships, and each subgraph represents a specific relationship type.

[0102] For each relation type, a Multi-Channel GCN layer and a Multi-Channel Edge Convolution layer are used for learning respectively. The obtained feature vectors are denoted as and where d denotes the dimension of the embedding space.

[0103] The Multi-Channel GCN layer is used to learn the global structure information between nodes, which aggregates the feature information of neighboring nodes to make the representation of each node contain not only its own features but also the features of its neighboring nodes. In the Multi-Channel Edge Convolution layer, the feature vector of each edge is obtained by concatenating the feature vectors of the two nodes connected by the edge and applying a linear transformation plus a nonlinear activation function. Then, the feature vectors of these edges are aggregated to update the feature representation of each node.

[0104] After the calculation of each layer is completed, the features of all relations are finally aggregated by weighted summation as follows:

[0105]

[0106]

[0107] where, denotes the embedding representation associated with node i in the t-th Multi-Channel GCN layer, denotes the embedding representation associated with edge j in the p-th Multi-Channel Edge Convolution layer, is the weighted sum of is the weighted sum of By combining the advantages of these two models, the model can better learn the complex relationships in the knowledge graph and thus improve the performance of link prediction.

[0108] In link prediction, the framework takes the entire heterogeneous graph and the features of all nodes as input. In the two tasks, the Multi-Channel GCN layer embedding features and the Multi-Channel Edge Convolution layer embedding features are used respectively, and the connection probability value between the embedding features of each node and the embedding features of the edge is used to determine the prediction result of the connection relationship between nodes.

[0109] Comparative Example One

[0110] Based on the multi-task learning model proposed in the second embodiment, this comparative example studies the influence of different numbers of layers of the Multi-Channel Edge Convolution layer (abbreviated as Edge) and the Multi-Channel GCN layer (abbreviated as GCN) on the performance of the model.

[0111] Based on the multi-task learning model proposed in the second embodiment, this comparative example studies the influence of different numbers of layers of the Multi-Channel Edge Convolution layer (abbreviated as Edge) and the Multi-Channel GCN layer (abbreviated as GCN) on the performance of the model. ​

[0112] When testing the GCN, the number of layers of the Edge module was fixed at 2 layers; similarly, when testing the Edge module, the number of layers of the GCN was fixed at 2 layers.

[0113] Referring to Figure 2 the impact of different numbers of layers on the multi-channel dynamic graph convolution module, and Figure 3 the impact of different numbers of layers on the multi-channel graph convolution network layer module. The results show that the number of layers of the Edge module has a smaller impact on the performance of the model, but when it is set to 2 layers, the performance is best. However, the number of layers of the GCN module has a more significant impact on the performance of the model. As the number of layers of the GCN increases, the performance of the model gradually improves, but after a certain depth, the performance growth tends to be stable, and the best performance is achieved when the number of layers is set to 2.

[0114] Comparative Example Two

[0115] Based on the multi-task learning model proposed in the second embodiment, Tables 5 and 6 show the impact of different embedding dimensions of the multi-channel dynamic graph convolution layer (Edge) and the multi-channel graph convolution network layer (GCN) on the performance of the model in the two tasks:

[0116] Table 4. Impact of different Edge settings on the model

[0117]

[0118] Table 5. Impact of different GCN settings on the model

[0119]

[0120] The results show that when the embedding dimension of the GCN is fixed at 1024, setting different embedding dimensions for the Edge module has a smaller impact on the overall performance of the model, and the best performance is achieved when the embedding dimension of the Edge module is set to 128, and the performance decreases as the dimension increases. On the other hand, when the embedding dimension of the Edge module is fixed at 128, setting different embedding dimensions for the GCN module significantly affects the performance of the model. The experimental results show that as the embedding dimension of the GCN increases, the performance of the model continues to improve. This phenomenon indicates that the GCN module has stronger sensitivity and decisiveness on the overall performance of the model, and adjusting the embedding dimension of the GCN module can more effectively improve the learning ability and inference performance of the model.

[0121] Third Embodiment

[0122] As a verification embodiment, the KGE-MFGRN model containing the knowledge graph embedding and feature fusion based multi-task gene regulatory network inference based on the model structure in the second embodiment is compared with several traditional models: MGDHGS, GraphSAGE, GIN, GAT, GATv2, and the obtained AUCROC diagram is as shown in Figure 4 The obtained results are shown in Table 6, and the highest score of each index in the table is highlighted:

[0123] Table 6. Comparison of evaluation indexes of KGE-MFGRN and other models

[0124]

[0125] The results show that the KGE-MFGRN performs best in all indexes of AUROC, Accuracy, Precision, Recall, F1-score and AUPRC. Compared with MGDHGS, the KGE-MFGRN model shows significant improvement in main performance indexes. Among them, the AUROC index is about 4.5% higher; the Accuracy index is about 10% higher; and the AUPRC index is about 10% higher. These experimental results clearly show that the model has made significant improvement in performance compared with existing models, and its superiority has been fully verified.

[0126] Fourth embodiment

[0127] As a verification embodiment, the KGE-MFGRN model containing the knowledge graph embedding and feature fusion based multi-task gene regulatory network inference based on the model structure in the second embodiment is compared with several traditional models: MGDHGS, GraphSAGE, GIN, GAT, GATv2, and the obtained AUCROC diagram is as shown in

[0128] First, the model performance of only retaining the multi-channel graph convolutional network layer (GCN) and only retaining the multi-channel dynamic graph convolution layer (Edge) is tested separately, and the result analysis comparison diagram is as shown in Figure 5 The results show that these two modules have an effect on the improvement of the model performance. When using GCN alone, the model has good performance in various performance indexes. Although the performance is relatively weak when using Edge alone, the various indexes of the model are significantly improved when GCN and Edge are constructed into a multi-task model, which also proves that the deep learning model can more fully capture the complex relationship features in the gene regulatory network by combining GCN and Edge.

[0129] Next, the feature extraction method is analyzed, and the effect of different feature combinations is evaluated. Specifically, this section tests the influence of using only One-hot encoding, using only PCP, using only PseDNC, and combining PCP and PseDNC, One-hot and PCP and PseDNC on the performance of the model. The results are shown in Figure 6 The KGE-MFGRN model that combines the three feature extraction methods performs well on all indicators and is significantly better than the single feature method. This shows that the model can more effectively capture the information in the gene sequence. Although the performance is good when using One-hot encoding alone, it is still lower than the multi-feature combination model. The combination of PCP and PseDNC performs better than using PseDNC alone, but the overall performance still lags behind the KGE-MFGRN model.

[0130] As an implementation solution, Figure 7 The figure shows the architecture of the hardware running environment of the computer system involved in the embodiment of the present application.

[0131] As shown in Figure 7 The computer system can include a processor 1001 such as a CPU, a memory 1005, a user interface 1003, a network interface 1004, and a communication bus 1002. The communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 can include a display screen (Display), an input unit such as a keyboard (Keyboard), and an optional user interface 1003 can also include a standard wired interface, a wireless interface. The network interface 1004 can optionally include a standard wired interface, a wireless interface (such as a WI-FI interface). The memory 1005 can be a high-speed RAM memory or a stable memory (non-volatile memory) such as a disk memory. The memory 1005 can also be an independent storage device from the aforementioned processor 1001.

[0132] Those skilled in the art can understand that Figure 7 The computer system architecture shown in the figure does not constitute a limitation on the computer system, and can include more or fewer components than the figure, or combine certain components, or different component arrangements.

[0133] As shown in Figure 7 The memory 1005 as a storage medium can include an operating system, a network communication module, a user interface module, and a computer program. The operating system is a program that manages and controls the hardware and software resources of the computer system, and the running of the computer program and other software or programs.

[0134] In Figure 7The computer system shown, the user interface 1003 is mainly used for connecting terminal, with terminal data communication; network interface 1004 is mainly used for background server, with background server data communication; processor 1001 can be used to call the computer program stored in the memory 1005.

[0135] In the embodiment, the computer system comprises: memory 1005, processor 1001 and computer program stored on the memory and executable on the processor, wherein:

[0136] When the processor 1001 calls the computer program stored in the memory 1005, the following operations are performed:

[0137] Each base in the initial gene sequence is encoded into a fixed-length binary vector and constitutes a two-dimensional matrix, the physical and chemical property values associated with each dinucleotide unit in the initial gene sequence are obtained, and the pseudo-dinucleotide group characteristics are determined according to the frequency of occurrence of dinucleotide in the initial gene sequence;

[0138] The physical and chemical property values and the pseudo-dinucleotide group characteristics of the initial gene sequence are learned by using a bidirectional long short-term memory network, the learned features are spliced with the two-dimensional matrix as the embedding of each node in the knowledge graph, and a knowledge graph dataset for training is obtained;

[0139] The training knowledge graph dataset is input into a deep learning model for training, and a gradient normalization algorithm is introduced to optimize the loss balance of the deep learning model during training, wherein the deep learning model comprises a multi-channel graph convolution layer for learning the node features of the knowledge graph embedding, and a multi-channel dynamic graph convolution layer for learning the edge features of the knowledge graph embedding.

[0140] When the processor 1001 calls the computer program stored in the memory 1005, the following operations are performed:

[0141] The loss value of the deep learning model after each training is updated by using the following formula:

[0142]

[0143] In the formula, is the updated loss value, is the gradient L2 norm expression of weight layer w using single task loss at training time t, is the average gradient norm of the gradient L2 norm corresponding to all tasks, is the relative inverse training rate of task i, is a hyperparameter for setting the strength of adjustment;

[0144] wherein:

[0145]

[0146]

[0147] wherein, is a loss function for task i, is a learning function.

[0148] It should be noted that while the embodiments herein are described in the general context of method steps of a method, system, and computer program product, the embodiments can also be implemented in combination with other computer programs or used in computer systems and host devices. Further, those of ordinary skill in the art will appreciate that the embodiments described herein can be implemented as a computer program product, which can include a computer-readable storage medium having stored computer program code thereon.

[0149] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flowcharts and / or blocks Figure 1 means for functionally implementing the steps listed in the flowchart block or blocks.

[0150] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart block or blocks. Figure 1 one or more flowcharts and / or blocks Figure 1 means for functionally implementing the steps listed in the flowchart block or blocks.

[0151] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flowcharts and / or blocks Figure 1 Figure 1 means for functionally implementing the steps listed in the flowchart block or blocks.

[0152] It is to be noticed that the singular form "a", "an", and, "the", include plural references unless the context clearly dictates otherwise. As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items. Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. It will be apparent to those skilled in the art that various modifications and variations can be made to the present application without departing from the spirit or scope of the application. Thus, it is intended that the present application cover modifications and variations of this application provided they come within the scope of the appended claims and their equivalents.

[0153] Although preferred embodiments of the application have been described herein, it will be apparent to those skilled in the art that various modifications and variations can be made in the embodiments without departing from the spirit or scope of the application. Thus, it is intended that the present application cover modifications and variations of this application provided they come within the scope of the appended claims and their equivalents.

[0154] Obviously, numerous modifications and variations of the present application are possible in light of the above teachings. It is therefore to be understood that within the scope of the appended claims and their equivalents, the application can be practiced otherwise than as specifically described.

Claims

1. A method for training a multi-task gene regulatory network model based on knowledge graph embedding and feature fusion, characterized in that, The method comprises the following steps: Each base in the initial gene sequence is encoded into a fixed-length binary vector and constitutes a two-dimensional matrix, the physical and chemical property value associated with each dinucleotide unit in the initial gene sequence is obtained, and the pseudo-dinucleotide group characteristics are determined according to the frequency of occurrence of dinucleotides in the initial gene sequence; The physical and chemical property values and the pseudo-dinucleotide group characteristics of the initial gene sequence are learned by using a bidirectional long short-term memory network, the learned characteristics are spliced with the two-dimensional matrix as the embedding of each node in the knowledge graph, and a training knowledge graph dataset is obtained; The training knowledge graph dataset is input into a deep learning model for training, and a gradient normalization algorithm is introduced to optimize the loss balance of the deep learning model during the training process, wherein the deep learning model comprises a multi-channel graph convolution layer for learning the node features of the knowledge graph embedding, and a multi-channel dynamic graph convolution layer for learning the edge features of the knowledge graph embedding; The calculation expression of the pseudo-dinucleotide group characteristics is: ； wherein is the feature of the pseudo dinucleotide group, is the frequency of occurrence of different dinucleotides, w is a weighting factor, is the number of pseudo components, represents the correlation factor between the jth adjacent dinucleotides; The calculation expression of the correlation factor is: ； wherein L is the length of the sequence, is a physical-chemical property value, PC is an identifier of each physical-chemical property value of the dinucleotide, denotes the dinucleotide starting at the i th nucleotide of the sequence, denotes the dinucleotide starting at the i + j th nucleotide of the sequence.

2. The method of claim 1, wherein, The step of introducing the gradient normalization algorithm to optimize the loss balance of the deep learning model comprises: The loss value of the deep learning model after each training is updated by using the following formula: ； wherein is the updated loss value, is an expression representing the gradient L2-norm in the weight layer w using the single-task loss at training time t, is the average gradient norm of the gradient L2-norms corresponding to all tasks, is the relative inverse training rate for task i is the relative inverse training rate for task is a hyperparameter for setting the strength of the adjustment; Wherein: ； ； wherein is the loss function for task i, is the learning function.

3. The method of claim 2, wherein, The calculation expression of the relative inverse training rate is: ； wherein is the task i At time t the loss rate, is the average loss rate for all tasks; Wherein: ； In the formula, is a single-task loss, is an initial-task loss.

4. A multi-task learning model obtained by using the training method of the multi-task gene regulation network model based on knowledge graph embedding and feature fusion according to any one of claims 1 to 3.

5. The multi-task learning model of claim 4, wherein, The multi-task learning model comprises: A multi-channel graph convolution layer for learning the node features of the knowledge graph embedding; A multi-channel dynamic graph convolution layer for learning the edge features of the knowledge graph embedding.

6. The multi-task learning model of claim 5, wherein, The number of layers of the multi-channel graph convolution layer and the multi-channel dynamic graph convolution layer is 2.

7. The multi-task learning model of claim 5, wherein, The embedding dimension of the multi-channel graph convolution layer is 1024, and the embedding dimension of the multi-channel dynamic graph convolution layer is 128.

8. A computer system, characterized by The computer system comprises a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the training method of the multi-task gene regulation network model based on knowledge graph embedding and feature fusion according to any one of claims 1 to 3.

Images