A gene sequence pre-training method and device based on a knowledge graph

Abstract

The application discloses a kind of gene sequence pre-training method and device of fusion knowledge graph, by considering the regulation relationship between genes to construct gene regulation graph, and increase motif and bin in gene regulation graph to construct knowledge graph based on gene regulation network, and then learn gene representation in knowledge graph, and the gene representation in knowledge graph is introduced as special token in the gene sequence of gene, improve the prediction accuracy of MLM model to mask, and obtain accurate gene representation, the initial vector of gene in the expansion gene regulation graph is learned as gene representation in sequence, gene representation is extracted again by pluggable representation model, such alternating process realizes the interaction of knowledge graph information and gene sequence information, gene representation is extracted using interactive training MLM model, which can improve and then improve the accuracy of gene correlation property prediction.

Description

Technical Field

[0001] This invention belongs to the technical field of combining biology and artificial intelligence, specifically relating to a gene sequence pre-training method and device that integrates knowledge graphs. Background Technology

[0002] Genes are complex structures that determine the hereditary characteristics of organisms. Currently, many studies use deep learning methods to learn the embedded representation of genes in order to quickly understand the characteristics of related genes and gene-related diseases. For example, patent document CN110400597A discloses a genotype prediction method based on deep learning, and patent document CN113574605A discloses a method for gene mutation detection using deep learning.

[0003] In existing deep learning methods, sequence-based approaches learn gene representations in low-dimensional space by improving message passing processes and network frameworks, and then apply these representations to downstream prediction tasks. However, methods based solely on gene sequences have low accuracy, fail to consider the regulatory relationships between genes, and result in poor gene representations in low-dimensional space, limiting their application in real-world scenarios. Summary of the Invention

[0004] In view of the above, the purpose of this invention is to provide a gene sequence pre-training method and apparatus that integrates knowledge graphs, thereby improving the accuracy of gene-related predictions by enhancing the accuracy of gene representation.

[0005] To achieve the above-mentioned objectives, an embodiment provides a gene sequence pre-training method integrating knowledge graphs, comprising the following steps:

[0006] Step 1: Obtain the regulatory relationships between genes based on species categories, construct a gene regulation network based on the regulatory relationships, and construct a knowledge graph based on the gene regulation network by adding motifs and bins. Here, motifs represent the positions where transcription factors produced by genes specifically bind to DNA sequences, bins represent segments of chromosome sequences, and initialize the initial representation of nodes in the knowledge graph.

[0007] Step 2: Divide the gene sequence corresponding to each gene into multiple tokens according to k-mer, and randomly mask the tokens to obtain the masked gene sequence;

[0008] Step 3: Use a pluggable representation model to extract the representation of each gene in the knowledge graph based on the initial node representation;

[0009] Step 4: Add the gene representation extracted by the pluggable representation model as a special token to the masked gene sequence, and use the gene sequence with the special token to train the MLM model for the mask prediction task, thereby obtaining a new gene representation.

[0010] Step 5: Update the initial representation of nodes in the knowledge graph with the new gene representation;

[0011] Step 6: Repeat steps 3-5 at least once to obtain the trained MLM model.

[0012] Preferably, the regulatory relationships between genes obtained based on species categories include promoting relationships and repressive relationships. When constructing a gene regulation map based on the regulatory relationships, genes are used as nodes, and promoting or repressive relationships are used as the edges between nodes.

[0013] Preferably, the expansion and construction of a knowledge graph based on gene regulation networks by adding motifs and bins to the gene regulation map includes:

[0014] Bins are linked to genes they contain through inclusion relationships, and motifs, being part of bins, are used as bridges to link bins to genes through binding relationships.

[0015] Preferably, the random masking of the token includes:

[0016] The tokens are masked according to a random masking ratio. Three masking methods are used in the masking process, including: randomly replacing part of the tokens with the mask according to the first ratio, randomly assigning random values ​​to part of the tokens according to the second ratio, and randomly keeping the original values ​​of part of the tokens according to the third ratio. The sum of the first ratio, the second ratio, and the third ratio is 1.

[0017] Preferably, the pluggable representation model includes GCN, GAT, and RGCN.

[0018] Preferably, during training, the network parameters of the pluggable representation model and the MLM model are updated alternately.

[0019] Preferably, the method further includes: predicting gene-related properties using a trained MLM model, including:

[0020] The trained MLM model is fine-tuned using gene samples. The fine-tuned MLM model is then used to predict gene-related properties, including:

[0021] The gene sequence corresponding to the gene to be predicted is divided into multiple tokens according to k-mer, and the tokens are randomly masked to obtain the masked gene sequence.

[0022] The masked gene sequence is input into the fine-tuned MLM model to obtain the gene representation of the gene to be predicted;

[0023] Predicting gene-related properties based on the gene representation of the gene to be predicted.

[0024] To achieve the above-mentioned objectives, an embodiment provides a gene sequence pre-training device that integrates knowledge graphs, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the above-mentioned gene sequence pre-training method that integrates knowledge graphs.

[0025] Compared with the prior art, the beneficial effects of the present invention include at least the following:

[0026] By constructing a gene regulation map considering the regulatory relationships between genes, and adding motifs and bins to the gene regulation map to build a knowledge graph based on the gene regulation network, gene representations in the knowledge graph are learned. These gene representations are then introduced into the gene sequences as special tokens, improving the prediction accuracy of the MLM model for masks and obtaining accurate gene representations. The gene representations learned from the sequences are used as the initial vectors for expanding the gene regulation map, and then the gene representations are extracted again through a pluggable representation model. This alternating process realizes the interaction between knowledge graph information and gene sequence information. Using the MLM model trained through this interaction to extract gene representations can improve, and thus improve, the accuracy of gene-related property prediction. Attached Figure Description

[0027] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 This is a flowchart of the gene sequence pre-training method for fusing knowledge graphs provided in the embodiment;

[0029] Figure 2 This is a flowchart of the gene sequence pre-training method fused with knowledge graph provided in the embodiment;

[0030] Figure 3 This is a schematic diagram of constructing an expanded gene regulation map based on the gene regulation map provided in the embodiment. Detailed Implementation

[0031] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not limit the scope of protection of this invention.

[0032] Figure 1 This is a flowchart of the gene sequence pre-training method for fusing knowledge graphs provided in the embodiments. Figure 1 As shown, the gene sequence pre-training method provided in this embodiment includes the following steps:

[0033] Step 1: Obtain the regulatory relationships between genes based on species categories, and construct a gene regulatory network based on these relationships.

[0034] In this embodiment, the regulatory relationships between genes obtained based on species category include facilitating relationships (isPositiveOf) and repressive relationships (isNegativeOf). A gene regulation network is established based on these two regulatory relationships. In the gene regulation network, genes are used as nodes, and facilitating or repressive relationships are used as edges between nodes.

[0035] Step 2: Construct a knowledge graph based on the gene regulation network by adding motifs and bins to the gene regulation network, and initialize the initial representation of the nodes in the knowledge graph.

[0036] In this embodiment, the gene regulation network is expanded by adding motif and bin information to construct a knowledge graph based on the gene regulation network. Here, a motif is the location where a transcription factor produced by a gene specifically binds to a DNA sequence, and a bin is a segment of a chromosome sequence. When constructing the knowledge graph based on motifs and bins, bins are linked to genes contained within them via inclusion relations (isPartOf). Motifs, as part of bins, are used as bridges to link bins to genes via binding relations (isContactedTo). That is, if a transcription factor produced by a gene specifically binds to a DNA sequence within a bin, then the bin is linked to that gene via a binding relation. In the constructed knowledge graph, the triplet format is as follows: (gene, isPositiveOf, gene), (gene, isNegativeOf, gene), (gene, isPartOf, bin), (gene, isContactedTo, bin), where the head and tail entities are either genes or bins.

[0037] The following is statistical information based on a knowledge graph of gene regulatory networks, using yeast as an example:

[0038]

[0039] In this embodiment, after constructing a knowledge graph based on a gene regulatory network, the nodes in the knowledge graph are initialized to obtain the initial representation of the nodes.

[0040] Step 3: Divide the gene sequence corresponding to each gene into multiple tokens according to k-mer, and randomly mask the tokens to obtain the masked gene sequence.

[0041] In this embodiment, each gene corresponds to a gene sequence, and the gene sequence corresponding to each gene consists of multiple bases. k-mer can be understood as dividing a long sequence into many short sequences of length k. k is chosen according to application requirements and can take values ​​such as 3, 4, 5, etc. Based on the k-mer division into multiple tokens, each token contains k bases. When k is 3, the gene sequence is divided into tokens of 3 bases each, meaning each token contains 3 bases.

[0042] After token partitioning, the tokens are randomly masked. A random masking ratio is set, for example, 15%, meaning 15% of the tokens are randomly masked. Three masking methods are used: first, randomly replacing some tokens with the mask according to a first ratio; second, randomly assigning values ​​to some tokens according to a second ratio; and third, keeping some tokens their original values ​​according to a third ratio. The sum of the first, second, and third ratios is 1. For example, if the first ratio is 80% and the second and third ratios are 10%, then 15% of the tokens are masked using these three methods at ratios of 80%, 10%, and 10%. The resulting masked gene sequence is obtained after the masking process.

[0043] Step 4: Use a pluggable representation model to extract the representation of each gene in the knowledge graph based on the initial representation of the nodes.

[0044] In this embodiment, the initial representation of nodes in the knowledge graph is input into a pluggable representation model, and the gene representation of gene nodes and the representation of bin nodes are extracted through inference computation. The pluggable representation model includes GCN, GAT, RGCN, etc., meaning various graph neural networks can be used as pluggable representation models.

[0045] Step 5: Add the gene representation extracted from the pluggable representation model as a special token to the masked gene sequence, and use the gene sequence with the special token to train the MLM model for the mask prediction task, thereby obtaining a new gene representation.

[0046] In this embodiment, gene representations based on knowledge graph learning are added as special tokens to the masked gene sequences. MLM models are then used to predict the mask in the gene sequences with added special tokens, resulting in new gene representations at the same time as obtaining each token representation.

[0047] Step 6: Update the initial representation of the nodes in the knowledge graph with the new gene representation.

[0048] In this embodiment, the new gene representation obtained through the MLM model is used as the initial representation of the node in the knowledge graph to update the gene representation in the knowledge graph.

[0049] Step 7: Repeat steps 4-6 at least once to obtain the trained MLM model.

[0050] When repeating step 4, the pluggable representation model is updated in the knowledge graph that has updated the initial representation, resulting in the new gene representation learned by the gene in the knowledge graph.

[0051] When repeating step 5, the new gene representation learned from the knowledge graph is added as a special token to the masked gene sequence. The MLM model is then used again to predict the mask in the gene sequence with the added special token, thus updating the gene representation. Through this cycle of steps 4 and 5, the interaction between knowledge graph information and gene sequence information is achieved.

[0052] In this embodiment, the trained MLM model can be used to predict downstream gene-related properties, thereby improving the accuracy of these predictions. Gene-related properties include gene expression level prediction and splicing site prediction, among others.

[0053] In application, the trained MLM model is fine-tuned using gene samples. The fine-tuned MLM model is then used to predict gene-related properties, including:

[0054] The gene sequence corresponding to the gene to be predicted is divided into multiple tokens according to k-mer, and the tokens are randomly masked to obtain the masked gene sequence. The masked gene sequence is input into the fine-tuned MLM model to obtain the gene representation of the gene to be predicted. Based on the gene representation of the gene to be predicted, the gene-related properties are predicted.

[0055] Based on the same inventive concept, the embodiment also provides a gene sequence pre-training device integrating knowledge graphs, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the above-mentioned gene sequence pre-training method integrating knowledge graphs, including the following steps:

[0056] Step 1: Obtain the regulatory relationships between genes based on species categories, and construct a gene regulatory network based on these relationships;

[0057] Step 2: Construct a knowledge graph based on the gene regulation network by adding motifs and bins to the gene regulation network, and initialize the initial representation of the nodes in the knowledge graph;

[0058] Step 3: Divide the gene sequence corresponding to each gene into multiple tokens according to k-mer, and randomly mask the tokens to obtain the masked gene sequence;

[0059] Step 4: Use a pluggable representation model to extract the representation of each gene in the knowledge graph based on the initial node representation;

[0060] Step 5: Add the gene representation extracted by the pluggable representation model as a special token to the masked gene sequence, and use the gene sequence with the special token to train the MLM model for the mask prediction task, thereby obtaining a new gene representation.

[0061] Step 6: Update the initial representation of nodes in the knowledge graph with the new gene representation;

[0062] Step 7: Repeat steps 4-6 at least once to obtain the trained MLM model.

[0063] Compared to the traditional method of constructing gene maps using HI-C data, the methods and apparatus provided in the above embodiments of the present invention consider the regulatory relationships of mutual inhibition or promotion between genes, and also consider motif and bin information, making the constructed knowledge graph contain richer information, which is conducive to the learning of gene representations. The gene representations in the knowledge graph are introduced into the gene sequences as special tokens, which improves the prediction accuracy of the MLM model for masks and obtains accurate gene representations. The gene representations learned from the sequences are used as the initial vectors of genes in the expanded gene regulation map, and the gene representations are extracted again through a pluggable representation model. This alternating process realizes the interaction between knowledge graph information and gene sequence information. The gene representations extracted by the interactively trained MLM model can improve and thus improve the accuracy of gene-related property prediction.

[0064] The specific embodiments described above illustrate the technical solution and beneficial effects of the present invention in detail. It should be understood that the above description is only the most preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, additions, and equivalent substitutions made within the scope of the principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A gene sequence pre-training method integrating knowledge graphs, characterized in that, Including the following steps: Step 1: Obtain the regulatory relationships between genes based on species categories, construct a gene regulation network based on the regulatory relationships, and construct a knowledge graph based on the gene regulation network by adding motifs and bins. Here, motifs represent the positions where transcription factors produced by genes specifically bind to DNA sequences, bins represent segments of chromosome sequences, and initialize the initial representation of nodes in the knowledge graph. Among them, the knowledge graph based on the gene regulation network is expanded by adding motifs and bins, including: connecting bins to genes contained in bins through inclusion relationships, motifs as part of bins, and connecting bins to genes through binding relationships using motifs as bridges; Step 2: Divide the gene sequence corresponding to each gene into multiple tokens according to k-mer, and randomly mask the tokens to obtain the masked gene sequence; Step 3: Use a pluggable representation model to extract the representation of each gene in the knowledge graph based on the initial node representation; Step 4: Add the gene representation extracted by the pluggable representation model as a special token to the masked gene sequence, and use the gene sequence with the special token to train the MLM model for the mask prediction task, thereby obtaining a new gene representation. Step 5: Update the initial representation of nodes in the knowledge graph with the new gene representation; Step 6: Repeat steps 3-5 at least once to obtain the trained MLM model.

2. The gene sequence pre-training method fused with knowledge graphs according to claim 1, characterized in that, The regulatory relationships between genes based on species categories include promoting and inhibiting relationships. When constructing a gene regulation map based on these relationships, genes are used as nodes, and promoting or inhibiting relationships are used as the edges between nodes.

3. The gene sequence pre-training method fused with knowledge graphs according to claim 1, characterized in that, The random masking of the token includes: The tokens are masked according to a random masking ratio. Three masking methods are used in the masking process, including: randomly replacing part of the tokens with the mask according to the first ratio, randomly assigning random values ​​to part of the tokens according to the second ratio, and randomly keeping the original values ​​of part of the tokens according to the third ratio. The sum of the first ratio, the second ratio, and the third ratio is 1.

4. The gene sequence pre-training method fused with knowledge graphs according to claim 1, characterized in that, The pluggable representation models include GCN, GAT, and RGCN.

5. The gene sequence pre-training method fused with knowledge graphs according to claim 1, characterized in that, During training, the network parameters of the pluggable representation model and the MLM model are updated alternately.

6. The gene sequence pre-training method fused with knowledge graph according to claim 1, characterized in that, It also includes: using trained MLM models to predict gene-related properties, including: The trained MLM model is fine-tuned using gene samples. The fine-tuned MLM model is then used to predict gene-related properties, including: The gene sequence corresponding to the gene to be predicted is divided into multiple tokens according to k-mer, and the tokens are randomly masked to obtain the masked gene sequence. The masked gene sequence is input into the fine-tuned MLM model to obtain the gene representation of the gene to be predicted; Predicting gene-related properties based on the gene representation of the gene to be predicted.

7. A gene sequence pre-training device integrating knowledge graphs, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the gene sequence pre-training method for fused knowledge graphs as described in any one of claims 1-6.

Images