Method for modeling transcriptional regulation network based on transfer learning
Through a transfer learning-based method, using large-scale transcriptional regulatory factors whole-genome-bound map data pre-training language models, the problem of difficulty in modeling different chromatin states or cell types in the existing technology is solved, and systematic modeling is achieved and the complex relationship between analytical transcriptional regulatory networks.
Patent Information
- Application Number
- CN202411003221.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-24
- Publication Date
- 2024-11-29
AI Technical Summary
The prior art is difficult to systematically model transcriptional regulatory networks in different chromatin states or cell types, and it is difficult to capture complex transcriptional regulatory factor interaction relationships.
Using transfer learning-based methods, large-scale transcriptional regulators genome-wide binding map data are used to pre-train language models. Through mask pre-training and fine-tuning techniques, the embedded representation of transcriptional regulators in each genome region is learned and output. Transcriptional regulatory networks in different scenarios are modeled.
Systematic modeling of complex transcriptional regulatory networks is realized, which can effectively analyze transcriptional regulatory networks in different chromatin states and cell types, capture the interaction relationship between transcriptional regulatory factors, and provide theoretical basis for revealing gene function and disease mechanisms.
Smart Images

Figure CN119049539A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the fields of deep learning and natural language processing, and in particular to a method for modeling a transcriptional regulatory network based on transfer learning. Background Art
[0002] In the process of gene expression regulation, the transcriptional regulatory network determines the gene expression pattern through complex interactions between transcriptional regulatory factors, regulatory elements and target genes.
[0003] Traditional methods for modeling transcriptional regulatory networks include machine learning models based on sequence alignment and feature engineering, such as support vector machines and random forests. However, due to the different interactions of transcriptional regulatory factors in different genomic regions and different cell types, traditional machine learning methods find it difficult to capture this complex interaction relationship. Although the self-attention mechanism used in deep language models can learn the contextual relationship of input sequences, it is widely used to parse transcriptional regulatory networks. However, these current models are mainly based on DNA sequences, gene expression data, or a small amount of transcriptional regulatory factor binding map data to model transcriptional regulatory networks. Because it is difficult to cope with the complexity and high dynamics of transcriptional regulatory networks, it is difficult to systematically parse transcriptional regulatory networks in different chromatin states or cell types. Summary of the invention
[0004] In order to solve the above problems existing in the prior art, the purpose of this case is to provide a method for modeling transcriptional regulatory networks based on transfer learning, which can solve the shortcomings of the existing methods that it is difficult to systematically model transcriptional regulatory networks in different chromatin states or cell types. This method uses data from large-scale genome-wide binding maps of transcriptional regulatory factors to pre-train the language model to model the interaction relationship between different transcriptional regulatory factors in each genomic region.
[0005] In order to achieve the above technical objectives, the technical solution of this case is as follows.
[0006] In the first aspect, this case proposes a method for modeling a transcriptional regulatory network based on transfer learning, the method comprising the following steps: using equal-length genomic region data as training samples, using the degree of binding of each cis-trans group on each genomic region as a sequence feature, and performing mask pre-training on the first language model, the first language model comprising an encoding layer and a first decoding layer; based on downstream tasks, designing labels corresponding to each training sample, and designing a second decoding layer, using the second decoding layer to replace the first decoding layer, and obtaining a second language model; using the training samples and the labels, training the second language model, freezing some parameters of the encoding layer during training, and the trained encoding layer can parse the transcriptional regulatory network for the downstream task, and output the embedded representation of the transcriptional regulatory factor in each genomic region. In one embodiment, the length of the genomic region data is determined according to the resolution.
[0007] In one implementation of the above technical solution, the first language model includes an input layer, an embedding layer, a Transformer block, and a fully connected layer in sequence; wherein the embedding layer and the Transformer block constitute an encoding layer, and the fully connected layer is a first decoding layer.
[0008] In one embodiment of the above technical solution, the step of determining the degree of binding of each cis-trans group in each genomic region includes: obtaining the decile of signal values corresponding to each cis-trans group in all genomic regions, using the decile as a threshold, and considering the transcription regulatory factors whose decile of signal values in a certain region is lower than the threshold as extremely weakly bound or weakly bound to the region, and vice versa as bound, strongly bound or extremely strongly bound; using the same method as above, for extremely weak binding or weak binding, further classification is performed based on the median, and for binding, strong binding or extremely strong binding, further classification is performed based on the one-third quantile of the signal value and the two-thirds quantile of the signal value.
[0009] In one embodiment of the above technical solution, the genomic region data sources include ChIP-seq, DNase-seq and ATAC-seq data.
[0010] In one embodiment of the above technical solution, before determining the degree of binding of the cis- and trans-groups on each genomic region, the genomic region data is filtered by data features, and the steps include: S00, obtaining a cis- and trans-group data; S10, if it meets at least four of the quality control standards, executing S20; otherwise, executing step S50; S20, if it is transcription factor data and the number of signal peaks is greater than or equal to 100, or if it is not transcription factor data and the number of signal peaks is greater than or equal to 1000, executing step S30, otherwise executing step S50; S30, if it is not in the If at least 60% of the genomic region of the signal peak has read coverage, execute S40, otherwise execute step S50; S40, remove redundant data, retain only one cis-trans group for each transcription regulatory factor in each cell type, and execute step S60; S50, discard the data, and execute step S60; S60, if there is the next cis-trans group data, return to step S10; the quality control standards are sequence quality median ≥25, unique alignment rate ≥60%, PCR bottleneck coefficient ≥80%, read length ratio within the signal peak ≥1%, and the number of 10-fold confidence peaks ≥500.
[0011] In one implementation of the above technical solution, the mask training randomly selects a portion of word units in the sequence features of each training sample and replaces them with masks, and trains and learns the first speech model by predicting the true values of these mask positions.
[0012] In one embodiment of the above technical solution, a training sample has M cis-trans groups, which are embedded in position and word unit through the embedding layer to form position encoding and word unit encoding respectively, and the two embedded encoding vectors are added as output, and the shape of the output data is M×d, indicating that each cis-trans group obtains an embedded representation with a feature number of d through the embedding layer, and d is a set value. If a transcriptional regulatory factor corresponds to multiple cis-trans groups, the embedded representations corresponding to the multiple cis-trans groups are averaged or max-pooled, so that the transcriptional regulatory factor corresponds to the embedded representation one by one.
[0013] In one embodiment of the above technical solution, before determining the degree of binding of each cis- and stroma group on each genomic region, the genomic region data is filtered through a data sample so that the sample data used for pre-training does not contain the region in the hg38 blacklist and has at least three cis- and stroma group data.
[0014] In the second aspect, according to the above method, this case proposes a corresponding device for modeling a transcriptional regulatory network based on transfer learning, wherein the device includes a first decoder, a second decoder, and a completely identical first encoder and second encoder; the first encoder and the first decoder are configured as a first language model, and mask pre-training is performed using training samples, the training samples are genomic region data of equal length, and the degree of binding of each cis-trans group in each genomic region is used as a sequence feature; the second encoder and the second decoder are configured as a second language model, and the parameters on the first encoder are loaded on the second encoder, and training is performed using each training sample and the label corresponding to each training sample, and some parameters of the second encoder are frozen during training, and the trained second encoder can parse the transcriptional regulatory network for the downstream task, and output the embedded representation of the transcriptional regulatory factor in each genomic region; wherein the label corresponding to each training sample and the second decoder are both designed based on the downstream task.
[0015] Beneficial technical effects of this case: Pre-training with large-scale data of human transcriptional regulatory factor full-gene binding maps can model complex transcriptional regulatory factor interactions. Fine-tuning the model for different chromatin states and cell types can systematically model transcriptional regulatory networks in different scenarios. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.
[0017] Figure 1 This is a data sample filtering method in an exemplary embodiment of the present invention.
[0018] Figure 2 This is a data discretization method in an exemplary embodiment of the present invention.
[0019] Figure 3 Schematic diagram of a model structure for modeling a transcriptional regulatory network in an exemplary embodiment of the present invention.
[0020] Figure 4 The figure is a schematic diagram of a parsing layer design flow of an exemplary embodiment of the present invention.
[0021] Figure 5 The following is a simplified flowchart of a method according to an exemplary embodiment of the present invention. DETAILED DESCRIPTION
[0022] In view of the shortcoming that existing methods find it difficult to systematically model transcriptional regulatory networks in different chromatin states or cell types, this case uses a pre-trained language model to learn the interaction relationship between such transcriptional regulatory factors, and applies it to different chromatin states and cell types through transfer learning, thereby achieving systematic modeling of complex and highly dynamic transcriptional regulatory networks.
[0023] The following will be combined with the accompanying drawings to clearly and completely describe how the technical solution of this case implements the method of modeling transcriptional regulatory networks based on transfer learning. Obviously, the described implementation method is only a part of the implementation method of this case, not all of the implementation methods. Based on the implementation methods in this case, all other implementation methods obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.
[0024] 1. Raw data preprocessing
[0025] Obtain genome-wide binding map data of transcriptional regulatory factors. In this article, the cistrome is referred to as a cistrome. A cistrome is the binding map of a transcriptional regulatory factor in a cell type or tissue, but a transcriptional regulatory factor may have multiple cistromes in a cell type or tissue.
[0026] All available ChIP-seq, DNase-seq and ATAC-seq data are obtained from public databases to form the cistrome dataset. The genomic regions in the dataset are divided into equal lengths, but users can adjust the length of the divided genomic regions according to their requirements for resolution, for example, 1kb or 200bp.
[0027] The signal value of the corresponding transcriptional regulatory factor in each genomic region is obtained from each ci strome. The signal value data is preprocessed, and the preprocessing process includes data feature filtering and data discretization.
[0028] The data feature filtering mentioned above includes the following steps:
[0029] First, if a cistrome meets at least four of the following five quality control criteria, it will be retained, thereby filtering out cistromes with poor quality control results:
[0030] (1.1) The median sequence quality is ≥25;
[0031] (1.2) Unique matching rate ≥ 60%;
[0032] (1.3) PCR bottleneck coefficient ≥ 80%;
[0033] (1.4) The read length ratio within the signal peak is ≥1%;
[0034] (1.5) The number of 10-fold confidence peaks is ≥ 500.
[0035] Then, transcription factor cistromes with less than 100 signal peaks and other transcription regulatory factor cistromes with less than 1000 signal peaks were filtered.
[0036] Next, cistromes with at least 60% read coverage in genomic regions without signal peaks were retained.
[0037] Finally, redundant data were removed so that only one set of cistromes remained for each transcriptional regulator in each cell type.
[0038] According to the above data feature filtering step description, a flow chart is designed, such as Figure 1 In schematic form, the steps involved can be further described as:
[0039] S00, obtain a cis-trans group data;
[0040] S10, if it meets at least four of the quality control standards, execute S20; otherwise, execute step S50;
[0041] S20, if it is transcription factor data and the number of signal peaks is greater than or equal to 100, or if it is not transcription factor data and the number of signal peaks is greater than or equal to 1000, then execute step S30, otherwise execute step S50;
[0042] S30, if at least 60% of the genomic region without a signal peak has read length coverage, execute S40, otherwise execute step S50;
[0043] S40, removing redundant data, retaining only one cis-trans group for each transcription regulatory factor in each cell type, and executing step S60;
[0044] S50, discard the data and execute step S60;
[0045] S60, if there is the next cistrome data, return to step S10. The data discretization mentioned above refers to classifying and determining the binding degree of the cistrome in different genomic regions according to the signal value of the cistrome in each genomic region.
[0046] In one embodiment, the classification step includes:
[0047] (2.1) Calculate the decile of the signal value of each cistrome in all genomic regions, and use this decile as the threshold. If the signal value of the transcriptional regulatory factor in a certain region is lower than this threshold, it is considered that the transcriptional regulatory factor is very weakly bound or weakly bound to the region; otherwise, it is considered that the transcriptional regulatory factor is bound, strongly bound, or very strongly bound to the region;
[0048] (2.2) Among them, very weak binding or weak binding is further classified into Class I and Class II based on the median value, represented by the values 0 and 1 respectively; binding, strong binding or very strong binding is further classified into Class III, Class IV and Class V based on the one-third quantile of the signal value and the two-thirds quantile of the signal value, represented by the values 2, 3 and 4 respectively (see Figure 2 (indicated).
[0049] (II) Construction of pre-training dataset
[0050] According to the binding status of each cistome on each genomic region, the genomic regions were sample filtered to filter out the regions included in the hg38 blacklist genomic region filter and with less cistome binding, and only the regions with at least three cistome signal peaks were retained.
[0051] Each genomic region after sample filtering is used as a training sample, and the degree of binding of each cistome on the genomic region is used as the sequence feature of the sample. Construct an N×M pre-training data set, where N refers to the number of samples of equal-length genomic regions after sample filtering, and M refers to the number of features after cistrome data feature filtering. The value in the pre-training data set is recorded as X i,j , represents the degree of binding of the jth cistrome to the i-th genomic region:
[0052] X i,j ∈{Category(1),Category(2),Category(3),Category(4),Category(5)}
[0053] Among them, Category(1)=0, Category(2)=1, Category(3)=2, Category(4)=3, and Category(5)=4.
[0054] 3. Pre-trained language model
[0055] The BERT architecture is used to build a language model for modeling transcriptional regulatory networks. First, define the pre-trained language model structure including input layer, embedding layer, Transformer block, fully connected layer, Softmax classification layer (see Figure 3). The loss function is then defined as the cross entropy loss, which is the loss of the one-hot encoding vector of the true label at the mask position and the Softmax probability of the model prediction value. The model is pre-trained using mask training until the loss function converges.
[0056] In the above pre-trained language model structure, the embedding layer and the Transformer block together form the encoding layer, and the fully connected layer is the decoding layer. During mask training, a portion of tokens is randomly selected from the input sequence and replaced with the mask <mask>, the model learns by predicting the true values of these mask positions. In the embedding layer, position embedding and word unit embedding are included to form position encoding and word unit encoding respectively. The two embedded encoding vectors are added together as the input of the subsequent Transformer block. Among them, the output size of each training sample after passing through the embedding layer is M×d, M is the number of filtered cistromes, that is, the number of sample features, and d is the number of hidden features of the embedding layer. The user sets the number of hidden features of the embedding layer according to needs. After the encoding block based on the Transformer structure learns the semantic information, the output vector of the Transformer block is decoded by the fully connected layer, and finally classified by the Softmax classification layer, and the predicted probability of each position belonging to each word in each word book is output.
[0057] During pre-training, the model is trained in a self-supervised training mode, so that it has the ability to reconstruct the structure of transcription regulatory factors in each genomic region. Due to the self-attention mechanism in the encoding layer, the pre-trained language model can learn the interaction relationship between different transcription regulatory factors in each genomic region, and reflect this interaction relationship in the embedded representation vectors of different transcription regulatory factors. That is, the closer the spatial distance between the embedded representation vectors of two transcription regulatory factors in a certain genomic region, the more likely it is that the two transcription regulatory factors have an interaction relationship in the genomic region. This interaction relationship varies with the chromatin state and cell type, and the knowledge learned in the pre-training process is applied to different downstream tasks by using the migration ability of the pre-trained model.
[0058] (IV) Fine-tuning
[0059] Design the structure of the model decoding layer for different downstream tasks. The structure of the decoding layer should be designed based on the input and label values of the decoding layer. The input of the decoding layer is based on the embedded representation of the genomic region output by the encoding layer. The embedded representation output by the encoding layer can be directly applied, or it can be recombined and designed according to different downstream tasks by adding downstream task-specific information (such as Figure 5 ). For example, cell type-specific embeddings can be added to genomic region embeddings to perform data generation and prediction across cell types.
[0060] Designing labels according to downstream tasks means designing labels corresponding to each genomic region. For a specific downstream task, the label value corresponding to each genomic region is redefined according to the data of different chromatin states and cell types. The type of label value is designed according to the downstream task, such as binary or continuous. Obtain the label value of the downstream task, and one training sample corresponds to one label value. For example, the purpose of fine-tuning a downstream task is to distinguish whether EZH2 performs a classical function or a non-classical function in a certain genomic region. When obtaining the label, the two types of regions can be marked as 0 and 1 respectively, and the model can be trained using the label value so that the model can acquire the ability to distinguish between the two types of genomic regions. Similarly, the label can be designed as a continuous type. For example, when the downstream task is to predict gene expression values, the gene expression value can be used as a label.
[0061] By loading the parameters of the pre-trained language model and replacing the model's decoding layer, transfer learning is performed so that the model outputs the target value based on the sequence features of the input genomic region (see Figure 4 ).
[0062] The parameters of the pre-trained language model are those in the encoding layer related to feature extraction, while the parameters in the decoding layer are custom parameters. When fine-tuning, you can freeze some parameters in the encoding layer to fine-tune the remaining parameters in the encoding layer related to feature extraction.
[0063] The target value type of the above output should be the same as the label value type. For example, in a classification task, a probability value should be output, while in a regression task, a continuous value should be output. Design a loss function based on the specific downstream task, and use the label corresponding to the task to iterate the model parameters until the loss function converges, so as to fine-tune the model based on the pre-trained model parameters.
[0064] The fine-tuned model encoding layer can input the combined features of cistrome on a genomic region for a specific downstream task, and return the prediction value for the specific task, that is, output the embedded representation of the transcription regulatory factor in each genomic region. This embedded representation reflects the interaction relationship between different transcription regulatory factors in each genomic region. That is, the closer the spatial distance between the embedded representations of two transcription regulatory factors in a certain genomic region, the more likely it is that the two transcription regulatory factors have an interaction relationship in this region.
[0065] Based on the fine-tuned model, the embedded representation of different transcriptional regulatory factors in each genomic region is obtained, which can be used to parse the transcriptional regulatory network. For example, a sample with M features is input, corresponding to M cistromes, and the size of the output data after the encoding layer is M×d, that is, each cistrom in each genomic region can obtain an embedded representation with a feature number of d, where d is the number of hidden features of the encoding layer, and the user can design a specific value as needed. In the case where a transcriptional regulatory factor corresponds to multiple cistromes, the embedded representation of the corresponding cistromes can be averaged or max-pooled, and the user can design a method as needed to make the transcriptional regulatory factor correspond to the embedded representation one-to-one.
[0066] As can be seen from the above, by fine-tuning the encoding layer, it is possible to systematically model the transcriptional regulatory network for different chromatin states or cell types.
[0067] (V) Summary
[0068] In summary, the entire implementation process of the above implementation method can be found in Figure 5 Schematic diagram. In this case, the interaction relationship between different transcription regulatory factors at each genomic site is learned through the pre-training process. Then, by fine-tuning, the transcription regulatory network in different scenarios is modeled, which can systematically analyze the transcription regulatory network and provide a theoretical basis for revealing gene functions and exploring disease mechanisms.
[0069] Through the description of the above embodiments, the technicians in the relevant field can clearly understand that the disclosed method can be implemented as a corresponding device. Exemplarily, a device for modeling a transcriptional regulatory network based on transfer learning, the device includes a first decoder, a generation module, a first encoder and a second encoder that are exactly the same; the first encoder and the first decoder are configured as a first language model, and mask pre-training is performed using training samples, the training samples are equal-length genomic region data, and the binding degree of different transcription regulatory factors on each genomic region is used as a sequence feature; the generation module is configured to design and generate a label corresponding to each training sample based on the downstream task, and design and generate a second decoder at the same time; the second encoder and the second decoder are configured as a second language model, and the parameters on the first encoder are loaded on the second encoder, and each training sample and the label corresponding to each training sample are used for training. Part of the parameters of the second encoder are frozen during training, and the trained second encoder can parse the transcriptional regulatory network for the downstream task and output the embedded representation of the transcriptional regulatory factor in each genomic region. Here, the encoder is implemented according to the encoding layer, and the decoder is implemented according to the decoding layer.
[0070] Through the description of the above implementation modes, those skilled in the art can clearly understand that the method or device of the present disclosure can be implemented by means of software plus necessary general hardware, and of course can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions performed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structures used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits, etc. However, for the present disclosure, software program implementation is a better implementation mode in most cases.
[0071] Although the embodiments of the present invention are described above in conjunction with the accompanying drawings, the present invention is not limited to the above specific embodiments and application fields, and the above specific embodiments are only illustrative and instructive, rather than restrictive. A person of ordinary skill in the art can also make many forms under the guidance of this specification and without departing from the scope of protection of the claims of the present invention, all of which belong to the protection of the present invention.< / mask>
Claims
1. A method for modeling a transcriptional regulatory network based on transfer learning, characterized in that: The method comprises the following steps: Using equal-length genome region data as training samples, using the binding degree of each cis-trans group in each genome region as a sequence feature, and performing mask pre-training on a first language model, wherein the first language model includes an encoding layer and a first decoding layer; Based on the downstream tasks, the labels corresponding to each training sample are designed, and the second decoding layer is designed. The second decoding layer is used to replace the first decoding layer to obtain the second language model. The second language model is trained using the training samples and the labels, and some parameters of the encoding layer are frozen during training. The trained encoding layer can parse the transcriptional regulatory network for the downstream task and output the embedded representation of the transcriptional regulatory factor in each genomic region.
2. The method according to claim 1, characterized in that: The first language model includes an input layer, an embedding layer, a Transformer block, and a fully connected layer in sequence; wherein the embedding layer and the Transformer block constitute an encoding layer, and the fully connected layer is a first decoding layer.
3. The method according to claim 1, characterized in that The steps for determining the extent of association of each cis-strome at each genomic region include: Obtain the decile of signal values corresponding to each cis-trans group in all genomic regions, use the decile as a threshold, and regard the transcriptional regulatory factors whose decile of signal values in a certain region is lower than the threshold as being extremely weakly bound or weakly bound to the region, otherwise they are regarded as being bound, strongly bound, or extremely strongly bound; Using the same method as above, very weak binding or weak binding was further classified based on the median, and binding, strong binding or very strong binding was further classified based on the one-third quantile of the signal value and the two-thirds quantile of the signal value.
4. The method according to claim 1, characterized in that: The genomic region data sources include ChIP-seq, DNase-seq and ATAC-seq data.
5. The method according to claim 1, characterized in that Before determining the degree of binding of the cis- and trans-groups on each genomic region, the genomic region data is filtered by data features, the steps comprising: S00, obtain a cis-trans group data; S10, if it meets at least four of the quality control standards, execute S20; otherwise, execute step S50; S20, if it is transcription factor data and the number of signal peaks is greater than or equal to 100, or if it is not transcription factor data and the number of signal peaks is greater than or equal to 1000, then execute step S30, otherwise execute step S50; S30, if at least 60% of the genomic region without a signal peak has read length coverage, execute S40, otherwise execute step S50; S40, removing redundant data, retaining only one cis-trans group for each transcription regulatory factor in each cell type, and executing step S60; S50, discard the data and execute step S60; S60, if there is the next cis-trans group data, return to step S10; The quality control standards are sequence quality median ≥25, unique alignment rate ≥60%, PCR bottleneck coefficient ≥80%, read length ratio within signal peak ≥1%, and number of 10-fold confidence peaks ≥500.
6. The method according to claim 1, characterized in that The mask training randomly selects a portion of word units in the sequence features of each training sample and replaces them with masks, and trains and learns the first speech model by predicting the true values of these mask positions.
7. The method according to claim 2, characterized in that: A training sample has M cis-trans groups, which are embedded by position and word unit through the embedding layer to form position encoding and word unit encoding respectively. The two embedded encoding vectors are added as output. The shape of the output data is M×d, which means that each cis-trans group obtains an embedded representation with a feature number of d through the embedding layer, where d is a set value. If a transcriptional regulatory factor corresponds to multiple cis-trans groups, the embedding representations corresponding to these multiple cis-trans groups are average-pooled or max-pooled to make the transcriptional regulatory factor correspond to the embedding representation one-to-one.
8. The method according to claim 1, characterized in that The length of the genomic region data is determined by the resolution.
9. The method according to claim 1, characterized in that: Before determining the binding degree of each cis-group on each genomic region, the genomic region data is filtered through data samples so that the sample data used for pre-training does not contain the region in the hg38 blacklist and has at least three cis-group data.
10. A device for modeling transcriptional regulatory networks based on transfer learning, characterized in that: The apparatus comprises a first decoder, a second decoder, and a first encoder and a second encoder that are identical; The first encoder and the first decoder are configured as a first language model, and mask pre-training is performed using training samples, wherein the training samples are genomic region data of equal length, and the degree of binding of each cis-trans group on each genomic region is used as a sequence feature; The second encoder and the second decoder are configured as a second language model, and the parameters of the first encoder are loaded on the second encoder, and each training sample and the label corresponding to each training sample are used for training, and some parameters of the second encoder are frozen during training, so that the trained second encoder can parse the transcriptional regulatory network for the downstream task and output the embedded representation of the transcriptional regulatory factor in each genomic region; The label corresponding to each training sample and the second decoder are both designed based on downstream tasks.
Citation Information
Patent Citations
Endogenous protein marking method used for Chip-seq genome-wide binding spectrum
CN106399311A
Method for eRNA identification, regulation target prediction and function annotation based on high-throughput transcriptome sequencing data
CN117275579A
Generation method of DNA (Deoxyribose Nucleic Acid) sequence transcription factor binding strength prediction model and electronic equipment
CN118016162A
Methods of identifying cis-regulatory elements and uses thereof
US20220284983A1
