Distributed transcriptional regulation network large model construction method based on transfer learning

Abstract

The invention discloses a distributed transcriptional regulation network large model construction method based on transfer learning, which takes a reliable transcriptional regulation prior relationship supported by literatures as a selection basis of an input variable and an output variable of a distributed sub-network, and performs model training by utilizing a transfer learning thought to obtain a regulation relationship in a transcription process. Comprising the following steps: S1, obtaining a transcriptional regulation relationship through transcriptional regulation priori knowledge; s2, constructing a distributed transcriptional regulation and control network large model; s3, performing model pre-training on the generic transcriptome data; s4, performing fine adjustment on the model by adopting the time sequence data set to obtain a specific distributed large model; and S5, formulating an evaluation index of a prediction result according to the data characteristics. By adopting the method, the direction of the transcriptional regulation relationship can be predicted, the regulation of the expression level of the target gene is further guided, and the related research of the transcriptional regulation relationship is accelerated.

Description

Technical Field

[0001] The present invention relates to the intersection of biological science and artificial intelligence, and in particular to a method for constructing a large model of a distributed transcriptional regulatory network based on transfer learning. Background Art

[0002] Transcriptional regulation is one of the core regulatory mechanisms of cellular metabolic activities. The interactions between a large number of transcription factors and target genes in eukaryotic cells form an extremely large and complex transcriptional regulatory network, which is used to maintain cellular metabolic homeostasis and regulate cellular metabolic differentiation. In view of the extreme complexity and variability of transcriptional regulatory networks in eukaryotic cells, it is urgent to build basic models of transcriptional regulatory networks at the system and global levels to accurately analyze gene regulatory mechanisms and analyze the molecular mechanisms behind important life activities such as cell aging and tumorigenesis at the system level. Therefore, the construction of basic models of eukaryotic cell transcriptional regulatory networks has great research significance for the development of precision medicine and health industries in my country.

[0003] At present, the methods for constructing transcriptional regulatory networks can be divided into mechanism modeling and data-driven modeling. The mechanism modeling method refers to the application of mathematical models or signal pathways to simulate the dynamic expression process of genes based on a priori knowledge of a set of core transcriptional regulatory networks. For cells and systems that have been widely studied, the accumulated knowledge is helpful to construct a better transcriptional regulatory network model. However, to date, the transcriptional regulatory relationship in cells has not been fully resolved, and how it responds to external environmental stimuli is still unknown. Therefore, the mechanism-based transcriptional regulatory network modeling strategy has obvious defects. The data-driven construction method refers to the use of machine learning or statistical methods to infer transcriptional regulatory networks on a large scale based on whole genome or transcriptomics data. This data-driven method can infer possible gene interactions and predict gene targets that affect key functions or regulatory relationships without sufficient prior knowledge of regulatory relationships but with a large amount of transcriptomics data. However, both construction methods have some limitations and cannot accurately simulate the transcriptional regulatory process. Therefore, the mechanism and data fusion driven method can make full use of the known prior knowledge of transcriptional regulation and better use artificial intelligence methods such as machine learning to model the transcriptional regulatory process of cells.

[0004] A key issue in the transcriptional regulation process is to find the regulatory direction of the transcription factor corresponding to the target gene, which will determine whether the corresponding transcription factor needs to be enhanced or inhibited when the expression level of a target gene needs to be regulated. However, due to the extreme complexity and high coupling of the microbial cell growth process, and the differences in the regulatory relationship presented by the same microorganism under different experimental conditions and different growth stages, a single regulatory relationship is difficult to capture directly. Most of the transcriptional regulatory relationships in existing databases are also Boolean, that is, they only characterize whether there is a regulatory relationship between the transcription factor and the target gene, and most of the information on the regulatory direction is missing. Therefore, it is of great significance to learn accurate transcriptional regulatory relationships from batch data through machine learning methods. The intrinsic connection between data can be learned from the data set, thereby helping to understand the regulatory direction between transcription factors and target genes, and then exploring the regulatory means of gene expression and accelerating the study of transcriptional regulatory relationships. Therefore, establishing a transcriptional regulatory network model for describing intracellular transcriptional regulatory relationships plays a very important role in cell life science research.

[0005] The construction of large models in the current biological field mainly consists of two steps: pre-training and fine-tuning, which is the idea of ​​transfer learning. The pre-training process refers to pre-training the model without a specific task, so unsupervised pre-training is used to learn the potential relationship between genes. Fine-tuning is to initialize the pre-trained parameters and perform specific training according to the specific data of the downstream task. Therefore, the pan-transcriptome data containing multiple strains is used as the pre-training data set, and the transcriptional regulatory relationship between genes in a certain type of cell or a certain species is learned from the pan-transcriptome data. As the amount of data fed into the machine learning model increases, the model will extract high-order, abstract, and more essential feature information from these data. In the fine-tuning stage of transfer learning, some parameters of the pre-trained model are fixed, and other parameters are retrained on a more specific data set. That is, starting from the abstract features in the pre-trained model, targeted training is carried out on the data of the downstream task, and the specific features of the downstream task are extracted to obtain a fine-tuned model.

[0006] In the process of transcriptional regulation, a gene generally needs to be co-regulated by multiple transcription factors. At the same time, since the number of genes in cells is very large, generally ranging from thousands to tens of thousands, this makes the transcriptional regulatory network a very large system. Therefore, when we use artificial neural network methods to model it, if all transcription factors and target genes form a complete neural network structure, even if the connections between some layers can be discarded based on prior knowledge of transcriptional regulation, the number of parameters it contains is still very large, which will be limited by the computer's running memory, and the model calculation will be too complicated, resulting in a long training time. In addition, for network models with a large number of parameters, the amount of data required for training is also very large. Summary of the invention

[0007] The purpose of the present invention is to provide a method for constructing a large model of a distributed transcriptional regulatory network based on transfer learning to solve the problems raised in the above-mentioned background technology.

[0008] To achieve the above-mentioned purpose of the invention, the present invention provides a method for constructing a large model of a distributed transcriptional regulatory network based on transfer learning. It uses reliable transcriptional regulatory prior relationships supported by literature as the basis for selecting input variables and output variables of distributed sub-networks, and uses the idea of ​​transfer learning to perform two-stage model training to obtain regulatory relationships in the transcription process, thereby guiding the study of gene expression regulation. To establish a large model of a distributed transcriptional regulatory network, high-order abstract feature information of transcriptional regulatory relationships is extracted from data through machine learning methods. The distributed structure avoids the limitations of computer operating memory, and can achieve the prediction of target gene regulation, providing a reliable tool for the study of regulatory relationships. In order to achieve the acquisition of specific transcriptional regulatory relationships, the idea of ​​transfer learning is adopted. The following steps are included:

[0009] Step S1, obtaining the transcriptional regulation relationship through prior knowledge of transcriptional regulation, and obtaining the pairing of transcription factors corresponding to target genes.

[0010] Step S2, constructing a large model of a distributed transcriptional regulatory network;

[0011] Step S3, pre-training the model on the pan-transcriptome data to obtain high-order features of transcriptional regulatory relationships in multiple strains.

[0012] Step S4, using the time series data set, fine-tuning the model to obtain a specific distributed large model;

[0013] Step S5, formulate the evaluation index of the prediction result according to the data characteristics, divide the prediction result into three levels of accuracy, and observe the influence of the data set on the prediction performance of the model and their reliability according to the accuracy of each sub-network.

[0014] Furthermore, the prior knowledge of transcriptional regulation, the transcriptional regulation relationship is formed by a pair of transcription factors and their target genes. One transcription factor can regulate multiple target genes, and the same target gene may be regulated by multiple transcription factors. The prior knowledge of transcriptional regulation obtained from the database is selected from the studies of transcription factors and target genes that have been reported and supported in the literature. The obtained regulatory relationship is a set of pairings, which characterizes the regulatory relationship between the transcription factor and the target gene, but does not contain other information such as the direction of regulation. By pairing all the relevant genes in the database, a transcriptional regulation knowledge graph can be obtained, and a mechanism network of transcriptional regulation can be constructed based on the knowledge graph.

[0015] Furthermore, the large model of the distributed transcriptional regulation network is composed of distributed sub-networks, which are based on the transcriptional regulation mechanism network obtained from the database. The distributed sub-network separates the mechanism network with the target gene as the center, extracts the transcription factor regulation relationship of the target gene, and uses the expression level of the transcription factor as input and the expression level of the target gene as output; the distributed sub-network has multiple inputs and a single output; the integration of all sub-networks represents the transcriptional regulation relationship of the cell as a whole.

[0016] Furthermore, when the target gene is regulated by only one transcription factor or itself, the distributed subnetwork is a single-input single-output structure.

[0017] Furthermore, the pre-training in step S3 is a process of performing data processing and alignment and distributed training on the pan-transcriptome dataset based on the distributed network after the structure is determined. It includes the following steps:

[0018] Step S301, it is necessary to process the pre-training data. The pan-transcriptome data is in TPM (Transcripts Per Million), which is a unit often used in the field of RNA sequencing data analysis, indicating the proportion of a transcript per million reads, and TPM can eliminate the influence of the sequencing depth of the sample on the analysis of gene expression. Among them, since the pan-transcriptome data set collects sequencing data in a variety of strains, although the gene similarity of multiple strains of the same species is very high, the errors and limitations of the detection cause the number of genes in the sequencing results of each strain to still have some differences, which results in some genes not having data in all strains. Therefore, it is necessary to supplement and align the data.

[0019] Step S302: align all TPM data into logarithmic space to make its data distribution more suitable for machine learning training. The processing formula can be expressed as log 2 (TPM+1).

[0020] Step S303, according to the difference in input and output of each sub-network, the pre-training data is divided to adapt to the characteristics of each sub-network.

[0021] Step S304, after determining the basic parameters of the training process, each sub-network model is subjected to distributed training, and the sub-networks do not affect each other during the training process. Each sub-network is trained on the processed and segmented sub-data set, which reduces the scale of the training network, improves the training efficiency, and overcomes the problem of insufficient data volume.

[0022] Furthermore, the fine-tuning in step S4 is the process of continuing to train each pre-trained sub-model on a specific data set of the downstream task. It includes the following steps:

[0023] Step S401, process the fine-tuning data set, align the pre-training data set with the fine-tuning data set in terms of gene number and gene name, and adjust the structure of each sub-network according to the fine-tuning data set. Since the fine-tuning data set is obtained based on time series detection, and different detection batches use different sampling time intervals, the interpolation method is used to align the time intervals between samples to facilitate the fine-tuning training of the model.

[0024] Step S402, since the distribution of the fine-tuning data set is different, it is the ratio of the expression levels before and after the gene perturbation, and it has been processed so that the unchanged data is set to zero. Therefore, in order to align the fine-tuning data to the logarithmic space, the data processing method used is: log 2 (2 x +1).

[0025] Step S403, fixing some parameters of each sub-model, freezing some parameters on the pre-training results based on transfer learning, and other parameters can continue to be trained based on the pre-training results.

[0026] In step S404, the fine-tuning data set is also segmented to determine the input and output of each sub-network, and the data is sent to each sub-model for model fine-tuning according to the distributed training method.

[0027] Further, the prediction result evaluation index is formulated according to the different relationships between the output result of the fine-tuning model and the label value. Due to the complexity and changeability of the transcriptional regulation process, there are errors between the predicted output value of the fine-tuning distributed model and the actual label value, and these errors are acceptable within a certain range. In addition, the acquisition of qualitative transcriptional regulation relationships is one of the main functions of building a transcriptional regulation network model. Therefore, according to the different manifestations of the error between the output of the fine-tuning model and the actual label value, the results are divided into three levels: excellent, qualified and unqualified, which are used to indicate that the error between the predicted result and the actual label is small, the error between the predicted result and the actual label is large but the direction is correct, and the error between the predicted result and the actual label is large. Based on the customized prediction result evaluation index, the performance of each sub-model after fine-tuning can be clearly analyzed, which can be used to implement further analysis on the sub-model.

[0028] Compared with the prior art, the present system and method have the following advantages:

[0029] (1) Make full use of known prior knowledge of transcriptional regulation to achieve mechanism and data fusion driven modeling and mine high-level abstract information features in the data;

[0030] (2) The huge transcriptional regulatory network is segmented based on the target gene to form a distributed sub-network model framework, overcoming the limitations of computer running memory, data size, and computing time;

[0031] (3) Adopting the idea of ​​transfer learning, we extract abstract features of transcriptional regulatory relationships from different datasets and use them through a fine-tuning process to obtain a specific fine-tuned model for downstream tasks;

[0032] (4) Result evaluation indicators were developed to evaluate the results of each model based on different error performances. The distributed model structure is conducive to further qualitative research on transcription factors. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 Flowchart of the method for constructing a large model of distributed transcriptional regulatory networks based on transfer learning.

[0034] Figure 2 This is an example diagram of prior knowledge of transcriptional regulatory relationships.

[0035] Figure 3 This is the overall framework diagram of the distributed transcriptional regulatory network model of the present invention.

[0036] Figure 4 This is a structural diagram of the distributed transcriptional regulatory subnetwork of the present invention.

[0037] Figure 5 Figure 2. Overall results of pre-training the distributed transcriptional regulatory subnetwork.

[0038] Figure 6 An example graph of interpolation results for the fine-tuning dataset.

[0039] Figure 7 Figure 2. Overall results of fine-tuning training for the distributed transcriptional regulatory subnetwork. DETAILED DESCRIPTION

[0040] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0041] like Figure 1 The flowchart of the method of the present invention is shown. The embodiment of the present invention models the transcriptional regulation process of Saccharomyces cerevisiae. First, all known transcriptional regulation relationships of Saccharomyces cerevisiae in the database are obtained, and then a distributed network model framework is constructed based on the known regulation relationships. The pre-training data set is processed, and the data set consists of sequencing data of 969 different yeast strains. After the pre-training is completed, the time series data after interpolation processing at intervals of five minutes is used as a fine-tuning data set to obtain a large model of a distributed transcriptional regulation network for a specific task. Finally, an evaluation criterion for the model prediction results is formulated, and the model is used to realize the prediction of the transcriptional regulation direction relationship.

[0042] Step S1, obtaining transcriptional regulation prior relationships.

[0043] The systematic and standard nomenclatures of all genes in Saccharomyces cerevisiae cells were obtained from the Saccharomyces cerevisiae databases Yeastract and YeastMine, and more than 6,000 genes in Saccharomyces cerevisiae cells were obtained. Among them, if the gene is a transcription factor, the database will provide the transcription factor binding domain information. Based on this information, 221 transcription factors can be screened out from all genes in Saccharomyces cerevisiae. The Yeastract database supports querying all target genes affected by the transcription factor based on the provided transcription factor, so that all prior knowledge of transcription regulation in Saccharomyces cerevisiae cells can be obtained. Examples of prior knowledge of transcriptional regulation relationships can be seen in Figure 2 .

[0044] Step S2: construct a large model framework of the distributed transcriptional regulatory network.

[0045] Based on the transcriptional regulation prior relationship obtained by the above process, it is possible to determine which transcription factors regulate the target gene, so as to construct a distributed sub-network model of the target gene expression level change. The distributed transcriptional regulation network model framework starts from the genotype and is used to determine whether a gene is missing or knocked out, so as to determine whether the gene expression level needs to be fixed to zero. The overall framework of the distributed transcriptional regulation network model is shown in Figure 3 The distributed sub-network constructed with the target gene as the target adopts a double hidden layer neural network, so each sub-network has a single output, and the number of nodes in the two hidden layers is three times the number of input nodes. The sub-network structure is shown in Figure 4 The reason why the number of hidden layers is tripled is that there are three regulatory relationships between transcription factors and target genes: enhancement, inhibition, and no effect. The number of parameters of the constructed sub-networks ranges from hundreds to tens of thousands, which is due to the different network complexities caused by the different number of transcription factors regulating genes. The total number of parameters of all sub-networks can reach millions to tens of millions.

[0046] Step S3. Distributed pre-training of the large transcriptional regulatory network model.

[0047] Since the total number of parameters of all distributed sub-networks can reach millions to tens of millions, training such a large network as a whole during model training will take up a lot of computer running memory, which may cause memory overflow and inability to complete training. And as the model size grows, the training time will also grow explosively, and the time required for overall network training will be longer, and the efficiency will be significantly reduced. Therefore, a distributed training method is adopted to construct a specific training data set for each sub-network based on its structure and input variables, so as to achieve separate training for each sub-network.

[0048] Among them, the training data set is obtained from the pan-transcriptome sequencing data, which contains sequencing data of 969 different yeast strains and more than 6,000 genes. The data set is saved in TPM (transcripts per million). There are differences in TPM values ​​between different data, and the data needs to be preprocessed when conducting model training. First of all, the gene lists of the 969 strain detection results are not completely consistent, and there is a problem of missing data for some genes in some strains. Considering that the possible reason for the missing data is that the expression level of the gene is low, the data is not detected. Therefore, the gene data of all strains are filled in by zero value filling. Subsequently, in order to align the data to the same distribution and ensure that the zero value remains at zero after scaling, a logarithmic scaling method is adopted: log 2 (TPM+1).

[0049] After data processing, the data set is split to obtain the training data set for each sub-network, and then sent to each sub-network for distributed training. Since it is the pre-training stage, a more conservative training parameter setting is adopted. The specific training parameter settings are as follows: learning rate lr = 0.005; maximum epoch = 2500; Adam optimizer is used, and the loss function is the sum of variances. The overall results of distributed network pre-training can be seen in Figure 5 .

[0050] Step S4. Fine-tune the large model of the transcriptional regulatory network.

[0051] After completing the pre-training of the model, it is necessary to continue training based on the pre-trained model parameters to meet the needs of downstream tasks. The fine-tuning dataset is the time series sequencing data of the expression changes of other genes detected at different time points after the expression of different transcription factors of Saccharomyces cerevisiae is disturbed. Similarly, the fine-tuning dataset needs to be pre-processed before re-training.

[0052] First, the fine-tuning dataset is saved in the form of logarithmic transformation of the ratio of gene expression before and after transcription factor perturbation, and each set of data does not contain missing values, so there is no need to fill the data. It is observed that there are some zero values ​​in the fine-tuning dataset, which indicates that the gene expression is too small to be detected at the current moment. However, when zero values ​​appear in the training data, the model training process may cause gradient vanishing or gradient explosion, so the dataset is also processed in the form of logarithmic transformation: log 2 (2 x +1). Keep the same approach as the pre-training dataset, and try to keep the dataset distribution as consistent as possible.

[0053] Secondly, the fine-tuning data is a time series, but the time interval of each set of detection data is uneven, so the interpolation method is used to process the data. The radial basis interpolation method of the Gaussian (Exp) function is used to interpolate the data set at a time interval of five minutes.

[0054] Finally, the same method as in the pre-training stage is used to split the data set and implement distributed fine-tuning training. The distribution of transcription factors and gene expression levels at the previous moment and the next moment are used as the input and output of the network. Among them, the hidden layer parameters of the fine-tuned sub-network are frozen and not trained, and the parameters of other layers continue to be trained based on the pre-training parameters. Here, since it is a targeted fine-tuning training, more detailed model training parameters are used. The specific training parameter settings are as follows: the training set and the test set are divided in a ratio of 9:1; the learning rate lr = 0.001; the weight decay weight_decay = 0.001; the number of iterations epoch = 200 × the size of the training set; the maximum number of iterations Max_epoch = 20000; the Adam optimizer is used; the loss function is set to the mean square error. The overall situation of the training error and test error of the model fine-tuning can be seen in Figure 6 .

[0055] Step S5. Evaluation indicators of distributed transcriptional regulation prediction results.

[0056] According to the characteristics of the fine-tuning dataset, the output of the fine-tuned model is a specific numerical prediction. However, although the more accurate the numerical prediction, the better, the prediction of the direction of transcriptional regulation is also one of the directions that researchers are interested in. Therefore, in order to evaluate the output results of the fine-tuning model more carefully, three prediction result levels are formulated according to the different performances of the prediction results and label values.

[0057] When the absolute error between the prediction result and the label value is less than or equal to 0.2, it is considered that the result has achieved a high accuracy in numerical prediction and can accurately simulate the quantitative transcriptional regulatory relationship. The prediction result is defined as "excellent".

[0058] When the absolute error between the prediction result and the label value is greater than 0.2, although the prediction result is not accurate enough in terms of quantitative indicators, if they are both greater than 1.2 or less than 0.8, the predicted change trend is considered correct. Therefore, although the prediction accuracy in this case is not high enough, the prediction result is still considered correct, and the prediction result is defined as "qualified".

[0059] When the relationship between the prediction result and the label value does not meet the above two conditions, it is considered that the result cannot reach a certain accuracy in both numerical prediction and trend prediction, and the prediction result is considered to be wrong. The prediction result is defined as "unqualified".

[0060] By counting all the prediction results of each sub-network on the fine-tuning dataset, the performance of the network can be evaluated, thereby guiding the regulation of the target gene expression level. The overall evaluation of all the prediction results of each sub-network is shown in Figure 7 .

[0061] In summary, the method of the present invention can start from the prior knowledge of transcriptional regulation, adopt the idea of ​​transfer learning to obtain high-order abstract features of transcriptional regulation relationships, and perform fine-tuning training for downstream tasks, and grade the prediction results. The distributed network structure can overcome the limitations of computer running memory, data size, computing time, etc. It can achieve the prediction of the direction of transcriptional regulation relationships, further guide the regulation of target gene expression levels, and accelerate the relevant research on transcriptional regulation relationships.

[0062] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for constructing a large model of a distributed transcriptional regulatory network based on transfer learning, characterized in that: The reliable transcriptional regulatory prior relationships supported by literature are used as the basis for selecting the input and output variables of the distributed sub-network, and the model training is carried out using the idea of ​​transfer learning to obtain the regulatory relationship in the transcription process, including the following steps: Step S1, obtaining the transcriptional regulation relationship through prior knowledge of transcriptional regulation, and obtaining the pairing of transcription factors corresponding to target genes; Step S2, constructing a large model of a distributed transcriptional regulatory network; Step S3, pre-training the model on the pan-transcriptome data to obtain high-order features of transcriptional regulatory relationships in multiple strains; Step S4, using the time series data set, fine-tuning the model to obtain a specific distributed large model; Step S5, formulate the evaluation index of the prediction result according to the data characteristics, divide the prediction result into three levels of accuracy, and observe the influence of the data set on the prediction performance of the model and their reliability according to the accuracy of each sub-network.

2. The method for constructing a large model of a distributed transcriptional regulatory network based on transfer learning according to claim 1, characterized in that: In step S1, all relevant genes in the database are paired to obtain a transcriptional regulation knowledge graph, and a transcriptional regulation mechanism network is constructed based on the transcriptional regulation knowledge graph.

3. The method for constructing a large model of a distributed transcriptional regulatory network based on transfer learning according to claim 1, characterized in that: The distributed transcriptional regulation network model is composed of distributed sub-networks. The distributed sub-networks are based on the transcriptional regulation mechanism network obtained from the database, separate the mechanism network with the target gene as the center, take the expression level of the transcription factor as the input, and the expression level of the target gene as the output; the distributed sub-network is a multi-input single-output structure.

4. The method for constructing a large model of a distributed transcriptional regulatory network based on transfer learning according to claim 1, characterized in that: When the target gene is regulated by only one transcription factor or by itself, the distributed subnetwork is a single-input single-output structure.

5. The method for constructing a large model of a distributed transcriptional regulatory network based on transfer learning according to claim 1, characterized in that: The pre-training in step S3 is a process of performing data processing and alignment on a pan-transcriptome dataset and performing distributed training according to a distributed network, comprising the following steps: Step S301, supplementing and aligning the data in the pan-transcriptome dataset; Step S302, aligning all data into logarithmic space to make its data distribution suitable for machine learning training, and its processing formula is expressed as log2(TPM+1); Step S303, according to the difference in input and output of each sub-network, the pre-training data is divided to adapt to the characteristics of each sub-network. Step S304: Train each distributed sub-network separately. The distributed structure ensures that the sub-networks do not affect each other.

6. The method for constructing a large model of a distributed transcriptional regulatory network based on transfer learning according to claim 1, characterized in that: The fine-tuning in step S4 is a process of continuing to train each pre-trained sub-model on a specific data set of a downstream task, including the following steps: Step S401, processing the fine-tuning dataset, aligning the pre-training dataset with the fine-tuning dataset in terms of the number of genes and gene names, adjusting the structure of each sub-network according to the fine-tuning dataset, and aligning the time intervals between samples using an interpolation method to facilitate fine-tuning training of the model; Step S402: align the fine-tuned data to the logarithmic space. The data processing formula is: log2(2 x +1); Step S403, fixing some parameters of each sub-model, freezing some parameters on the pre-training results based on transfer learning, and continuing to train other parameters based on the pre-training results; Step S404, segment the fine-tuning data set, determine the input and output of each sub-network, and send them to each sub-model for model fine-tuning according to the distributed training method.

7. The method for constructing a large model of a distributed transcriptional regulatory network based on transfer learning according to claim 1, characterized in that: The prediction result evaluation index is formulated according to the different relationships between the output results of the fine-tuning model and the actual label value, and the results are divided into three levels, including: excellent, qualified and unqualified, which are respectively used to indicate that the error between the prediction result and the actual label is small, the error between the prediction result and the actual label is large but in the correct direction, and the error between the prediction result and the actual label is large.

Images