Prediction model generation method and device of gene regulation network, server and medium

Abstract

The invention discloses a prediction model generation method and device of a gene regulatory network, a server and a medium, and relates to the technical field of computers. Gene activity changes at different time points in a gene expression time sequence are captured through a gene expression matrix encoder to obtain global gene expression characteristics; a text encoder is used for analyzing the corresponding relation between a gene expression mode and text semantics, multi-time-point gene expression feature data and biological text feature data are aligned through a comparative learning method, the learning rate is gradually and smoothly reduced through a cosine recession learning rate scheduling mechanism, it is ensured that the model is closer to a globally optimal solution, and the optimal solution is obtained. Relevance modeling between multi-time-point gene expression data and biological text description is achieved through the prediction model of the gene regulation and control network, and the precision and generalization ability of a statistical model are improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, device, server and medium for generating a prediction model of a gene regulatory network. Background Art

[0002] Gene expression data is an important concept in the field of genetics and bioinformatics. It reflects the abundance of gene transcription products in cells measured directly or indirectly. Gene expression data records the gene expression levels under different conditions, providing resources for studying gene regulation. Gene regulatory networks refer to the network formed by the interaction between genes in cells. Gene regulatory networks are used to describe the regulatory relationship between genes. By predicting gene regulatory networks, key regulatory factors and regulatory pathways can be obtained, and then which genes have important effects on the expression of other genes can be inferred.

[0003] In the current related technologies, the method for predicting gene regulatory networks is mainly machine learning, that is, taking gene expression data as input features and inputting them into the prediction model to output the prediction results of the gene regulatory network. However, in the related technologies, the input features are manually selected and optimized. The complexity of gene expression data makes it impossible to fully capture the characteristic information of gene expression data when manually selecting feature parameters. The performance of the prediction model depends on the selection and optimization of feature parameters. The method of manually selecting feature parameters reduces the analysis accuracy and generalization ability of the prediction model. Summary of the invention

[0004] The present application provides a method, device, server and medium for generating a prediction model of a gene regulatory network, so as to at least solve the problem of decreased analysis accuracy and generalization ability of the prediction model in the related art.

[0005] The present application provides a method for generating a prediction model of a gene regulatory network, comprising:

[0006] Obtain gene expression time series;

[0007] Inputting the gene expression time series into a gene expression matrix encoder to output a global gene expression signature;

[0008] Get biological text data;

[0009] Inputting the biological text data into a text encoder to output global text features;

[0010] Aligning the global gene expression feature with the global text feature to obtain a contrast loss;

[0011] Optimizing the prediction model according to the contrast loss to generate an optimized prediction model;

[0012] The optimized prediction model is trained through a cosine decay learning rate scheduling mechanism to generate a trained prediction model.

[0013] The present application also provides a device for generating a prediction model of a gene regulatory network, comprising:

[0014] The first acquisition module is used to obtain the gene expression time series;

[0015] A first output module, used for inputting the gene expression time series into a gene expression matrix encoder to output a global gene expression feature;

[0016] A second acquisition module is used to acquire biological text data;

[0017] A second output module, configured to input the biological text data into a text encoder to output global text features;

[0018] An alignment module, used for aligning the global gene expression feature with the global text feature to obtain a contrast loss;

[0019] An optimization module, used for optimizing the prediction model according to the contrast loss to generate an optimized prediction model;

[0020] The training module is used to train the optimized prediction model through a cosine decay learning rate scheduling mechanism to generate a trained prediction model.

[0021] The present application also provides a server, comprising: a memory for storing a computer program; and a processor for implementing the steps of any of the above-mentioned methods for generating a prediction model of a gene regulatory network when executing the computer program.

[0022] The present application also provides a computer-readable storage medium, in which a computer program is stored, wherein when the computer program is executed by a processor, the steps of any of the above-mentioned methods for generating a prediction model of a gene regulatory network are implemented.

[0023] The present application also provides a computer program product, including a computer program, which implements the steps of any of the above-mentioned methods for generating a prediction model of a gene regulatory network when the computer program is executed by a processor.

[0024] Through the present application, since the gene expression matrix encoder captures the changes in gene activity at different time points in the gene expression time series, the feature representation of the time series is extracted to obtain the global gene expression features, the global semantic features are extracted through the text encoder, and the correspondence between the gene expression pattern and the text semantics is analyzed. Through the comparative learning method, the gene expression feature data of multiple time points are aligned with the biological text feature data, and the learning rate is gradually and smoothly reduced through the cosine decay learning rate scheduling mechanism to ensure that the model is closer to the global optimal solution. The correlation modeling between the gene expression data of multiple time points and the biological text description is realized through the prediction model of the gene regulatory network, thereby improving the accuracy and generalization ability of the statistical model. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0026] Figure 1 A schematic diagram of the system structure of a server provided in an embodiment of the present application;

[0027] Figure 2 A schematic diagram of a process for generating a prediction model of a gene regulatory network provided in an embodiment of the present application;

[0028] Figure 3 A schematic diagram of comparative learning provided in an embodiment of the present application;

[0029] Figure 4 A schematic diagram of the structure of a prediction model generation device for a gene regulatory network provided in an embodiment of the present application;

[0030] Figure 5 A schematic diagram of the structure of the server provided for this application. DETAILED DESCRIPTION

[0031] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0032] It should be noted that, in the description of this application, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. The terms "first", "second", etc. in this application are used to distinguish similar objects, and are not used to describe a specific order or sequence.

[0033] First, the terms involved in this application are explained:

[0034] Gene expression: refers to the process of synthesizing genetic information from genes into functional gene products. Gene expression products are usually proteins. Gene expression can be regulated by regulating several steps, including transcription, RNA splicing, translation and post-translational modification.

[0035] Attention mechanism: Attention mechanism is a data processing method in machine learning. Its core goal is to select the more critical information for the current task from a large amount of information. It is widely used in various types of machine learning tasks such as natural language processing, image recognition and speech recognition.

[0036] Query vector: The query vector is the representation of the input sequence that the model is currently paying attention to or processing. It is usually generated by the last hidden state obtained by the encoder or the current decoder state. The query vector represents the information that the model is currently paying attention to, which can be regarded as a question or content that needs to be judged.

[0037] Multimodality: Multimodality refers to the technology of using two or more different senses, data types or information sources for interaction, processing and analysis in fields such as artificial intelligence. Multimodal technology integrates and utilizes two or more different senses, data types or information sources, such as text, images, audio and video, for interaction, processing and analysis. By integrating and utilizing the correlation and differences between these different modalities, higher-level feature extraction, classification, retrieval and other tasks can be achieved.

[0038] Contrastive loss: Contrastive loss is a metric learning loss function that learns the similarities and differences between sample pairs so that the distance between similar sample pairs in the feature space is smaller, while the distance between dissimilar sample pairs is larger.

[0039] In order to solve the problem of decreased analysis accuracy and generalization ability of prediction models in the prior art, the embodiments of the present application propose the following technical concepts: considering the time series characteristics of gene expression data, the inventors designed a gene expression matrix encoder that can capture dynamic changes in time, and captured the changes in gene activity at different time points through the gene expression matrix encoder to obtain global gene expression features; the inventors constructed a text encoder based on a language model, extracted global semantic features through the text encoder, and analyzed the correspondence between gene expression patterns and text semantics to obtain global text features; considering the alignment of gene expression feature data at multiple time points with biological text feature data through comparative learning methods; and gradually and smoothly reducing the learning rate through the cosine decay learning rate scheduling mechanism to ensure that the model is closer to the global optimal solution.

[0040] In order to enable those skilled in the art to better understand the present application, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0041] In combination with the specific application environment architecture or specific hardware architecture on which the execution of the prediction model generation method of the gene regulatory network depends, the specific application environment architecture or specific hardware architecture is described here. Figure 1 , Figure 1 A schematic diagram of the system structure of the server provided in the embodiment of the present application. Figure 1 As shown, the server includes: a receiving device 101, a processor 102 and a display device 103.

[0042] It is understandable that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the object identification method. In other feasible implementations of the present application, the above architecture may include more or fewer components than shown in the figure, or combine certain components, or split certain components, or arrange the components differently, which can be determined according to the actual application scenario and is not limited here. Figure 1 The components shown may be implemented in hardware, software, or a combination of software and hardware.

[0043] In a specific implementation process, the receiving device 101 may be an input / output interface or a communication interface, and may acquire a gene expression time series.

[0044] The processor 102 may generate a trained prediction model.

[0045] The display device 103 can be used to display the trained prediction model and the like.

[0046] The display device may also be a touch display screen, which is used to receive user instructions while displaying the above-mentioned content, so as to realize operational interaction with the user.

[0047] It should be understood that the above-mentioned processor can be implemented by the processor reading instructions in the memory and executing the instructions, or it can be implemented by a chip circuit.

[0048] In addition, the network architecture and business scenarios described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Ordinary technicians in this field can know that with the evolution of network architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0049] Figure 2 A schematic diagram of a process for generating a prediction model of a gene regulatory network provided in an embodiment of the present application, such as Figure 2 As shown, the method includes:

[0050] S201: Obtain gene expression time series.

[0051] In this embodiment, the gene expression time series data captures the changes in gene activity at different time points, showing the dynamic regulatory mechanism of the organism in the time dimension, covering the entire time frame from the initial stimulation to the end of the biological process.

[0052] For example, in a psoriasis study, by applying imiquimod to mice as a zero-time-point intervention, gene expression data were sampled in the acute phase, remission phase, and relapse phase to explore the dynamic changes of key genes in the skin pathology process.

[0053] In this example, the gene expression time series dataset is represented as follows:

[0054]

[0055] In the formula, Datasets representing gene expression time series; Represents the gene expression time series matrix of the i-th sample; Represents a text comment.

[0056] The content of text annotations includes but is not limited to experimental design background, gene function annotations, biological significance of time points, and treatment conditions.

[0057] In this embodiment, each sample Include time points, among which It is expressed as:

[0058]

[0059] In the formula, M represents the number of genes analyzed; F represents the feature dimension at each time point; express Dimensional real space matrix; Indicates samples at each time point.

[0060] The characteristic dimensions of each time point include but are not limited to gene expression level FPKM (Fragments Per Kilobase of exon model per Million mapped fragments), gene expression level TPM (Transcripts Per Million) and additional statistical values ​​under experimental conditions.

[0061] Specifically, in order to standardize the training process and adapt to the fixed input structure of the model, the sample time points are sampled, and the maximum number of time points for each sample is limited to k, expressed as:

[0062]

[0063] In the formula, Represents the sample collection after sampling; Indicates A sample of samples; Indicates from The index set selected from satisfies .

[0064] In this embodiment, biologically critical time points are preferentially preserved during sampling.

[0065] For example, in psoriasis research, acute phase samples reflect a strong inflammatory response of the mouse immune system, remission phase samples reveal the activation of immunosuppressive signals, and relapse phase samples can reveal the underlying molecular mechanisms of pathological recurrence.

[0066] Specifically, when the number of time points of a sample is less than the maximum number of time points per sample, that is, When, fill zero-filled time points, expressed as:

[0067]

[0068] In the formula, represents samples after the supplementation time point; Indicates samples at each time point; To fill the matrix, initialize it to zero; express Dimensional real space matrix; Indicates the number of filling time points, a total of indivual.

[0069] In this embodiment, the filling strategy serves as a placeholder and does not introduce additional biological information.

[0070] S202: Input the gene expression time series into the gene expression matrix encoder to output the global gene expression features.

[0071] Specifically, the gene expression matrix in the gene expression time series is input into the feature extraction model to obtain an embedding vector, the weight of each time point in the gene expression time series is calculated through the attention mechanism, and the global gene expression features are calculated based on the weights.

[0072] S203: Obtain biological text data.

[0073] In this embodiment, biomedical corpus is integrated in the pre-training stage to obtain biological text data.

[0074] Among them, the sources of biomedical corpus include but are not limited to corpora, websites and articles.

[0075] S204: Input the biological text data into a text encoder to output global text features.

[0076] Specifically, the time points in biological text data are dynamically annotated through the attention mechanism, key time points are screened to retain the core biological text information, and different weights are assigned to different time points through weighted feature aggregation and normalization of the attention mechanism; the biological text is segmented by updating the vocabulary, the segmented text is input into the language model, and the output hidden state sequence is extracted to obtain the global text feature.

[0077] S205: Align the global gene expression features with the global text features to obtain the contrast loss.

[0078] Specifically, the global text features are normalized, the similarity of the feature vectors is calculated based on the global gene expression features and the global text features, and a two-way loss function is calculated based on the similarity of the feature vectors: the loss function from gene expression to text and the loss function from text to gene expression, and the contrast loss is obtained based on the two-way loss function.

[0079] S206: Optimizing the prediction model according to the contrast loss to generate an optimized prediction model.

[0080] Specifically, the semantic consistency between the gene expression time series and the corresponding text annotations is judged according to the contrast loss, and the optimized prediction model is obtained by dynamically adjusting the weight parameters.

[0081] Figure 3 A schematic diagram of comparative learning provided in an embodiment of the present application.

[0082] like Figure 3 As shown, the gene expression matrix encoder integrates the gene expression features at different time points according to the attention mechanism to obtain the global gene expression features; the text encoder parses the text semantics, combines the context and domain knowledge, obtains the biological semantics and time relationship, and obtains the global text features; the text semantics and gene expression data are aligned through contrastive learning to optimize the prediction model.

[0083] S207: Train the optimized prediction model through a cosine decay learning rate scheduling mechanism to generate a trained prediction model.

[0084] Specifically, by introducing cosine decay, based on the cosine function, the learning rate of each iteration is calculated according to the initial learning rate, the prediction model is optimized by batch gradient descent, and the matching gene expression and text are optimized according to the contrast loss.

[0085] In this embodiment, during the training process, the prediction model learns the semantic consistency between gene expression features and text descriptions, and dynamically adjusts parameters to improve robustness under different data distributions.

[0086] In addition, it should be noted that in the inference phase, the prediction model generates aggregate features based on the unprocessed new gene expression time series data through the feature representation ability of the training phase. , the aggregated features are represented by the pre-stored text features Perform similarity calculation to determine the degree of match between the two.

[0087] The formula for calculating the similarity between the aggregated features and the pre-stored text features is:

[0088]

[0089] In the formula, Indicates the similarity between the aggregated features and the pre-stored text features; Represents aggregate features; Represents pre-stored text features.

[0090] Specifically, in the inference stage, the prediction model identifies gene expression patterns associated with specific biological processes or experimental conditions by calculating the similarity between gene expression features and text descriptions, displays key features in the dynamic regulation process, generates semantic annotations for unlabeled gene expression data, and predicts the potential functions of unknown genes or genes with unclear functions.

[0091] It can be seen from the above embodiments that the gene expression matrix encoder is used to capture the changes in gene activity at different time points in the gene expression time series, and the feature representation of the time series is extracted to obtain the global gene expression features. The text encoder is used to extract global semantic features, and the correspondence between gene expression patterns and text semantics is analyzed. Through the comparative learning method, the gene expression feature data at multiple time points are aligned with the biological text feature data. The learning rate is gradually and smoothly reduced through the cosine decay learning rate scheduling mechanism to ensure that the model is closer to the global optimal solution. The correlation modeling between the gene expression data at multiple time points and the biological text description is realized through the prediction model of the gene regulatory network, which improves the accuracy and generalization ability of the statistical model.

[0092] In one embodiment of the present application, step S202 includes:

[0093] S2021: Obtain gene expression matrix from gene expression time series.

[0094] In this embodiment, the gene expression matrix at each time point in the gene expression time series is recorded as .

[0095] S2022: Input the gene expression matrix into the feature extraction model to output an embedding vector.

[0096] In this embodiment, the feature extraction model includes but is not limited to a convolutional neural network model, a multi-head attention mechanism model, and a wavelet transform model.

[0097] In this embodiment, the embedding vector is expressed as:

[0098]

[0099] In the formula, represents the embedding vector; represents the number of features; represents the embedding dimension; express Dimensional real space matrix; Represents the gene expression matrix; Encoder() represents encoding.

[0100] Among them, the global features at each time point are embedded by a specific tag Extracted as the representative feature corresponding to this time point.

[0101] Exemplarily, specific tags include, but are not limited to, a [CLS] tag, a [SEP] tag, and a [PAD] tag.

[0102] S2023: Calculate the weight of each time in the gene expression time series based on the embedding vector through the attention mechanism.

[0103] In this embodiment, the formula for calculating the weight of each time in the gene expression time series according to the embedding vector through the attention mechanism is:

[0104]

[0105] In the formula, represents weight; represents the attention score function; Indicates Characteristics of each time point; represents the query vector; Indicates Characteristics of each time point; Indicates the number of time points.

[0106] In this embodiment, the attention score function is used to measure the correlation between the time point feature and the query vector.

[0107] In this embodiment, the query vector It can be initialized to the global query vector learned during the training process, or generated by prior information of the experimental design.

[0108] For example, in psoriasis research, gene expression in the acute phase often has a research role in the immune mechanism of the disease, and its attention weight can be dynamically adjusted through training.

[0109] S2024: Calculate and generate global gene expression features based on the embedding vector and the weight of each time in the gene expression time series.

[0110] In this embodiment, the formula for calculating and generating the global gene expression feature is as follows based on the embedding vector and the weight of each time in the gene expression time series:

[0111]

[0112] In the formula, represents the global gene expression signature; represents weight; Represents time point characteristics; Indicates the number of time points.

[0113] In this embodiment, the introduction of the attention mechanism enables the model to dynamically adjust the degree of attention paid to different time points, highlighting the feature extraction of key time periods of biological processes.

[0114] For example, in a mouse model of psoriasis, the acute phase is usually accompanied by a strong immune response, while the remission phase involves the upregulation of immunosuppressive signals. The attention mechanism can better capture important immune signals by assigning higher weights to the acute phase time points. At the same time, for time point data including the relapse period, the attention mechanism can focus on key changes before and after the relapse, revealing potential relapse triggers.

[0115] It can be seen from the above embodiments that according to the temporal characteristics of gene expression data, through the dynamic feature aggregation method based on the attention mechanism, the model can extract the correlation between time points and dynamically adjust the weight of each time point according to the importance of the time point, avoiding over-smoothing and ignoring detailed information, thereby improving the model's ability to understand time series.

[0116] In one embodiment of the present application, step S204 includes:

[0117] S2041: Convert biological text data into sequences of text tokens.

[0118] In this embodiment, the text tag sequence is represented as:

[0119]

[0120] In the formula, Represents a sequence of text tokens; Indicates the maximum sequence length; Indicates word or subword unit.

[0121] S2042: Perform text processing on the text tag sequence through a word segmentation algorithm to obtain a text after word segmentation.

[0122] Specifically, by recording the co-occurrence frequency of characters or subwords in the corpus, the initialization vocabulary is updated according to the set number of iterations, and the text token sequence is processed according to the updated vocabulary to obtain the segmented text.

[0123] S2043: Input the segmented text into the language model to output a hidden state sequence.

[0124] In this embodiment, the text after word segmentation is input into the language model to output the formula of the hidden state sequence, which is:

[0125]

[0126] In the formula, represents a hidden state sequence; Represents the text sequence after word segmentation; express Dimensional real matrix space; Indicates the length of the sequence; represents the embedding dimension; BERT() represents processing through a language model.

[0127] In this embodiment, the language model is a pre-trained language model.

[0128] S2044: Perform hidden state extraction on the hidden state sequence to output global text features.

[0129] In this embodiment, the global text feature is expressed as:

[0130]

[0131] In the formula, Represents global text features; represents a sequence of hidden states.

[0132] It can be seen from the above embodiments that the global semantic features of the text description are extracted through the text encoder, and the relationship between the semantics and time of the biological text is captured by introducing biological text data with time context and combining context and domain knowledge, thereby improving the text encoding capability.

[0133] In one embodiment of the present application, step S2042 includes:

[0134] S421: Create an initialization vocabulary.

[0135] In this embodiment, the contents recorded in the initialization vocabulary are basic character sets, and the words in the text are decomposed into single characters.

[0136] Exemplary, basic character sets include, but are not limited to, letters, numbers, and punctuation marks.

[0137] For example, the word "apple" is decomposed into the character sequence "a", "p", "p", "l", "e".

[0138] S422: Record the co-occurrence frequencies of characters or sub-words in the corpus, and merge the sub-words with the highest co-occurrence frequencies according to the set number of iterations to generate an updated vocabulary.

[0139] Specifically, the word segmentation algorithm counts the co-occurrence frequency of adjacent characters or sub-word pairs in the corpus, records the number of co-occurrences, identifies common character or sub-word combinations, and in each iteration, merges the sub-words with the highest co-occurrence frequency to generate new sub-words.

[0140] The number of iterations is capped at the size of the vocabulary or the frequency threshold.

[0141] S423: Perform text processing on the text tag sequence according to the updated vocabulary to obtain a segmented text.

[0142] Specifically, the text token sequence is tokenized using the updated vocabulary, and for each input word, the longest subword unit is matched from the updated vocabulary using a matching algorithm.

[0143] In this embodiment, the matching algorithm is a maximum matching algorithm or a greedy algorithm.

[0144] For example, if the vocabulary includes "Bus" and "##pirone" but does not record the word "Buspirone", the algorithm decomposes "Buspirone" into "Bus" and "##pirone" for recognition.

[0145] Among them, "##" represents a subword and needs to be connected to the previous subword.

[0146] From the above embodiments, it can be seen that the word is decomposed into smaller sub-word units through the word segmentation algorithm, so that when a complete word is not in the vocabulary of the prediction model, the model can be represented by a combination of sub-words, thereby reducing the probability that the prediction model will not recognize an unregistered word.

[0147] In one embodiment of the present application, before step S2041, the method further includes:

[0148] S301: Parse the static semantics of biological text data to generate parsed biological text data.

[0149] Specifically, the lexical structure and grammar in the text data are parsed, and each text type in the text data is identified according to semantic rules to obtain parsed biological text data.

[0150] S302: Identify biological process text in the parsed biological text data.

[0151] In this embodiment, the content recorded in the biological text data includes but is not limited to time, experimental subjects, and experimental steps.

[0152] S303: Time-calibrate the biological process text to generate calibrated biological text data.

[0153] Specifically, the order of occurrence of biological processes in the biological process text is identified according to the attention mechanism, and the time information of each biological process is calibrated to obtain the calibrated biological text data.

[0154] For example, the text description of a biological process is as follows: "In the acute phase, the immune response is significantly enhanced; then it enters the remission phase, and the immunosuppressive signals gradually increase." The time words "acute phase" and "remission phase" are calibrated, and the time sequence in the text description is identified through the self-attention mechanism, and corresponding weights are assigned to different time points.

[0155] From the above embodiments, it can be seen that the time information of the sequence of biological processes in text data is captured by the attention mechanism, so that the model can recognize the time sequence relationship in the text description, assign corresponding weights to different times, and improve the alignment effect between text and time series gene expression data.

[0156] In one embodiment of the present application, step S205 includes:

[0157] S2051: Calculate the similarity of feature vectors based on global gene expression features and global text features.

[0158] In this embodiment, the similarity of feature vectors is used to measure the degree of alignment between global gene expression features and global text features in the multimodal space.

[0159] Among them, the similarity of feature vectors is expressed as:

[0160]

[0161] In the formula, represents the transposed normalized global gene expression signature; Represents the normalized global text features; Represents the similarity of feature vectors.

[0162] S2052: Calculate the comparison loss between each gene expression feature and the text feature in the global gene expression feature according to the similarity of the feature vector, and obtain the loss function from gene expression to text.

[0163] In this embodiment, the formula for calculating the contrast loss between each gene expression feature and the text feature in the global gene expression feature is:

[0164]

[0165] In the formula, Represents the loss function of gene expression to text; represents the temperature parameter; Similarity of feature vectors representing gene expression features; The similarity of feature vectors representing global gene expression features and global text features; represents the number of gene expression features; Indicates the number of text features.

[0166] S2053: Calculate the comparison loss between each text feature in the global text feature and the gene expression feature according to the similarity of the feature vector, and obtain the loss function from text to gene expression.

[0167] In this embodiment, the formula for calculating the contrast loss between each text feature in the global text feature and the gene expression feature is:

[0168]

[0169] In the formula, Represents the loss function from text to gene expression; represents the temperature parameter; Similarity of feature vectors representing text features; The similarity of feature vectors representing global gene expression features and global text features; represents the number of gene expression features; Indicates the number of text features.

[0170] S2054: Calculate and generate contrast loss based on the loss function from gene expression to text and the loss function from text to gene expression.

[0171] In this embodiment, the formula for generating the contrast loss is calculated based on the loss function from gene expression to text and the loss function from text to gene expression:

[0172]

[0173] In the formula, represents contrast loss; Represents the loss function of gene expression to text; Represents the loss function for text-to-gene expression.

[0174] In addition, it should be noted that contrast loss reflects the corresponding relationship in biology. In psoriasis research, gene expression characteristics It can capture dynamic gene regulation patterns at different time points, such as acute phase, remission phase and relapse phase, while text features The semantic description of the biological process, such as the enhancement or inhibition of the immune response, is recorded, and the matching similarity is improved based on the contrast loss of gene expression features and text features. The contrast loss has the following characteristics:

[0175] 1. Contrastive loss can reflect the dynamic changes of gene expression over time. For example, high expression of inflammation-related genes may be detected in the acute phase, while gene expression may be downregulated in the remission phase. This mapping of dynamic changes helps the model capture the semantic consistency between gene regulation patterns and text descriptions.

[0176] 2. Contrastive loss can align the features of heterogeneous biological processes and reveal the synergistic relationship of multiple mechanisms. For example, gene expression features may reflect the activation state of a signal transduction pathway, while text features describe the molecular functions associated with it, thereby achieving deep integration in a multimodal semantic space.

[0177] 3. Contrastive loss has the ability to detect abnormal patterns. When the model finds that some gene expression features cannot be aligned with the text description during contrastive learning, it records this scenario as a potential abnormal pattern, such as a new type of gene expression change that has not been described in the experimental data.

[0178] It can be seen from the above embodiments that by calculating the loss function of global gene expression features and global text features, each gene expression feature is aligned with the correctly matched text features and away from the unmatched text features, each text feature is aligned with the correctly matched gene expression features and away from the unmatched gene expression features, and the bidirectional loss functions from gene expression to text and from text to gene expression are calculated, so that the model simultaneously focuses on the bidirectional alignment process from gene expression to text and from text to gene expression, thereby achieving comprehensive alignment of multimodal features.

[0179] In one embodiment of the present application, before step S2051, the method further includes:

[0180] The global text features are normalized to obtain processed global text features.

[0181] In this embodiment, the process of text feature normalization is expressed as follows:

[0182]

[0183] In the formula, Represents the normalized global text features; represents the unprocessed global text features; Represents the absolute value of the global text feature.

[0184] It can be seen from the above embodiments that the normalization operation ensures that the characteristic scales of text and gene expression in the multimodal space are consistent, which facilitates comparative learning optimization; by automatically associating the corresponding time points in the gene expression data, combining biological semantics and time relationships, the text encoder can more accurately parse complex medical texts and provide convenience for diagnosis and prediction.

[0185] In one embodiment of the present application, step S207 includes:

[0186] S2071: Obtain the initial learning rate of the optimized prediction model.

[0187] In this embodiment, represents the initial learning rate.

[0188] S2072: Calculate the learning rate after iteration according to the set number of iterations and the initial learning rate.

[0189] In this embodiment, the formula for calculating the learning rate after iteration according to the set number of iterations and the initial learning rate is:

[0190]

[0191] In the formula, Represents the learning rate after iteration; represents the initial learning rate; Indicates the number of iterations; Represents the total number of training steps.

[0192] S2073: Train the optimized prediction model according to the iterative learning rate to generate a trained prediction model.

[0193] Specifically, the weight of the prediction model is adjusted according to the learning rate after multiple iterations to generate a trained prediction model.

[0194] It can be seen from the above embodiments that by introducing the cosine decay learning rate scheduling mechanism in the training process, based on the cosine function, the learning rate is gradually reduced during the training process. The lower learning rate enables the model to focus on subtle weight adjustments and avoid the model from falling into a suboptimal solution too early; by gradually and smoothly reducing the learning rate, the weight oscillation problem that occurs in the later stage of training can be effectively avoided, thereby improving the stability of the model; the cosine decay learning rate scheduling mechanism is used to reduce the imbalance problem of gene expression and text features in the fusion process, thereby improving the training efficiency and the model's adaptability to complex multimodal data.

[0195] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method.

[0196] Figure 4 A schematic diagram of the structure of a prediction model generation device for a gene regulatory network provided in an embodiment of the present application, such as Figure 4 As shown, the prediction model generation device 40 for the gene regulatory network provided in this embodiment includes: a first acquisition module 401, a first output module 402, a second acquisition module 403, a second output module 404, an alignment module 405, an optimization module 406 and a training module 407.

[0197] The first acquisition module 401 is used to acquire the gene expression time series.

[0198] The first output module 402 is used to input the gene expression time series into the gene expression matrix encoder to output the global gene expression feature.

[0199] The second acquisition module 403 is used to acquire biological text data.

[0200] The second output module 404 is used to input the biological text data into a text encoder to output global text features.

[0201] The alignment module 405 is used to align the global gene expression features with the global text features to obtain the contrast loss.

[0202] The optimization module 406 is used to optimize the prediction model according to the contrast loss to generate an optimized prediction model.

[0203] The training module 407 is used to train the optimized prediction model through a cosine decay learning rate scheduling mechanism to generate a trained prediction model.

[0204] In a possible implementation, the first output module 402 includes:

[0205] The first acquisition unit is used to acquire the gene expression matrix in the gene expression time series.

[0206] The first output unit is used to input the gene expression matrix into the feature extraction model to output an embedding vector.

[0207] The first computing unit is used to calculate the weight of each time in the gene expression time series according to the embedding vector through the attention mechanism.

[0208] The second calculation unit is used to calculate and generate the global gene expression feature according to the embedding vector and the weight of each time in the gene expression time series.

[0209] In a possible implementation, the formula for calculating the weight of each time in the gene expression time series according to the embedding vector by the attention mechanism in the first computing unit is:

[0210]

[0211] In the formula, represents weight; represents the attention score function; Indicates Characteristics of each time point; represents the query vector; Indicates Characteristics of each time point; Indicates the number of time points.

[0212] In a possible implementation, the second computing unit calculates and generates a formula for global gene expression features based on the embedding vector and the weight of each time in the gene expression time series:

[0213]

[0214] In the formula, represents the global gene expression signature; represents weight; Represents time point characteristics; Indicates the number of time points.

[0215] In a possible implementation, the second output module 404 includes:

[0216] A conversion unit that converts biological text data into sequences of text tokens.

[0217] The text processing unit is used to perform text processing on the text tag sequence through a word segmentation algorithm to obtain a text after word segmentation.

[0218] The second output unit is used to input the segmented text into the language model to output a hidden state sequence.

[0219] The extraction unit is used to perform hidden state extraction on the hidden state sequence to output a global text feature.

[0220] In a possible implementation, the text processing unit includes:

[0221] Creates a subunit for creating an initial vocabulary.

[0222] The recording subunit is used to record the co-occurrence frequency of characters or subwords in the corpus, and merge the subwords with the highest co-occurrence frequency according to the set number of iterations to generate an updated vocabulary.

[0223] The text processing subunit is used to perform text processing on the text tag sequence according to the updated vocabulary to obtain the segmented text.

[0224] In a possible implementation, the second output unit inputs the segmented text into the language model to output a formula for a hidden state sequence, which is:

[0225]

[0226] In the formula, represents a hidden state sequence; Represents the text sequence after word segmentation; express Dimensional real matrix space; Indicates the length of the sequence; represents the embedding dimension; BERT() represents processing through a language model.

[0227] In a possible implementation, the second output module 404 further includes:

[0228] The parsing unit is used to parse the static semantics of the biological text data and generate parsed biological text data.

[0229] The recognition unit is used to recognize biological process texts in the parsed biological text data.

[0230] The calibration unit is used to perform time calibration on the biological process text to generate calibrated biological text data.

[0231] In a possible implementation, the alignment module 405 includes:

[0232] The third calculation unit is used to calculate the similarity of the feature vector according to the global gene expression feature and the global text feature.

[0233] The fourth calculation unit is used to calculate the contrast loss between each gene expression feature and the text feature in the global gene expression feature according to the similarity of the feature vector, and obtain the loss function from gene expression to text.

[0234] The fifth calculation unit is used to calculate the comparison loss between each text feature in the global text feature and the gene expression feature according to the similarity of the feature vector, and obtain the loss function from text to gene expression.

[0235] The sixth calculation unit is used to calculate and generate the contrast loss according to the loss function from gene expression to text and the loss function from text to gene expression.

[0236] In a possible implementation, the formula for generating the contrast loss is calculated in the sixth calculation unit according to the loss function from gene expression to text and the loss function from text to gene expression, which is:

[0237]

[0238] In the formula, represents contrast loss; Represents the loss function of gene expression to text; Represents the loss function for text-to-gene expression.

[0239] In a possible implementation, the training module 407 includes:

[0240] The second acquisition unit is used to acquire the initial learning rate of the optimized prediction model.

[0241] The seventh calculation unit is used to calculate the learning rate after iteration according to the set number of iterations and the initial learning rate.

[0242] The training unit is used to train the optimized prediction model according to the iterative learning rate to generate a trained prediction model.

[0243] In a possible implementation, the formula for calculating the learning rate after iteration according to the set number of iterations and the initial learning rate in the seventh calculation unit is:

[0244]

[0245] In the formula, Represents the learning rate after iteration; represents the initial learning rate; Indicates the number of iterations; Represents the total number of training steps.

[0246] For the description of the features in the embodiment corresponding to the device for generating a prediction model of a gene regulatory network, reference may be made to the relevant description of the embodiment corresponding to the method for generating a prediction model of a gene regulatory network, which will not be described in detail here.

[0247] Figure 5 This is a schematic diagram of the server structure provided for this application. Figure 5 As shown, the server 50 provided in this embodiment includes: at least one processor 501 and a memory 502. Optionally, the server 50 also includes a communication component 503. The processor 501, the memory 502 and the communication component 503 are connected via a bus 504.

[0248] In the specific implementation process, at least one processor 501 executes the computer execution instructions stored in the memory 502, so that at least one processor 501 executes the above-mentioned embodiment of the method for generating a prediction model of a gene regulatory network.

[0249] The specific implementation process of the processor 501 can be found in the above method embodiment, and its implementation principle and technical effect are similar, so this embodiment will not be repeated here.

[0250] In the above embodiments, it should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the method disclosed in the application may be directly implemented as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor.

[0251] The memory may include a high-speed memory (Random Access Memory, RAM), and may also include a non-volatile memory (NVM), such as at least one disk storage.

[0252] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of this application is not limited to only one bus or one type of bus.

[0253] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above-mentioned embodiments of the method for generating a prediction model of a gene regulatory network when running.

[0254] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0255] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps in any of the above-mentioned embodiments of the method for generating a prediction model of a gene regulatory network are implemented.

[0256] An embodiment of the present application also provides another computer program product, including a non-volatile computer-readable storage medium, the non-volatile computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, implementing the steps in any of the above-mentioned embodiments of the method for generating a predictive model of a gene regulatory network.

[0257] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0258] The above describes in detail a method, device, server and medium for generating a prediction model of a gene regulatory network provided by the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application, and the description of the above embodiments is only used to help understand the method and core idea of ​​the present application. It should be pointed out that for those of ordinary skill in the art, without departing from the principles of the present application, several improvements and modifications may also be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of the present application.

Claims

1. A method for generating a prediction model of a gene regulatory network, characterized in that: include: Obtain gene expression time series; Inputting the gene expression time series into a gene expression matrix encoder to output a global gene expression signature; Get biological text data; Inputting the biological text data into a text encoder to output global text features; Aligning the global gene expression feature with the global text feature to obtain a contrast loss; Optimizing the prediction model according to the contrast loss to generate an optimized prediction model; The optimized prediction model is trained through a cosine decay learning rate scheduling mechanism to generate a trained prediction model.

2. The method for generating a prediction model of a gene regulatory network according to claim 1, characterized in that: The step of inputting the gene expression time series into a gene expression matrix encoder to output global gene expression features comprises: Obtaining a gene expression matrix in the gene expression time series; Inputting the gene expression matrix into a feature extraction model to output an embedding vector; Calculating the weight of each time in the gene expression time series according to the embedding vector through an attention mechanism; A global gene expression feature is generated by calculation according to the embedding vector and the weight of each time in the gene expression time series.

3. The method for generating a prediction model of a gene regulatory network according to claim 2, characterized in that: The formula for calculating the weight of each time in the gene expression time series according to the embedding vector through the attention mechanism is: In the formula, represents weight; represents the attention score function; Indicates Characteristics of each time point; represents the query vector; Indicates Characteristics of each time point; Indicates the number of time points.

4. The method for generating a prediction model of a gene regulatory network according to claim 2, characterized in that: The formula for calculating and generating the global gene expression feature according to the embedding vector and the weight of each time in the gene expression time series is: In the formula, represents the global gene expression signature; represents weight; Represents time point characteristics; Indicates the number of time points.

5. The method for generating a prediction model of a gene regulatory network according to claim 1, characterized in that: The step of inputting the biological text data into a text encoder to output global text features comprises: Converting the biological text data into a sequence of text tokens; Performing text processing on the text tag sequence by using a word segmentation algorithm to obtain a text after word segmentation; Inputting the segmented text into a language model to output a hidden state sequence; Hidden state extraction is performed on the hidden state sequence to output a global text feature.

6. The method for generating a prediction model of a gene regulatory network according to claim 5, characterized in that: The text processing of the text tag sequence by a word segmentation algorithm to obtain a text after word segmentation includes: Create an initial vocabulary; Record the co-occurrence frequency of characters or subwords in the corpus, and merge the subwords with the highest co-occurrence frequency according to the set number of iterations to generate an updated vocabulary; The text tag sequence is subjected to text processing according to the updated vocabulary to obtain a segmented text.

7. The method for generating a prediction model of a gene regulatory network according to claim 5, characterized in that: The formula for inputting the segmented text into the language model to output the hidden state sequence is: In the formula, represents a hidden state sequence; Represents the text sequence after word segmentation; express Dimensional real matrix space; Indicates the length of the sequence; represents the embedding dimension; BERT() represents processing through a language model.

8. The method for generating a prediction model of a gene regulatory network according to claim 5, characterized in that: Before converting the biological text data into a text tag sequence, the method further includes: parsing the static semantics of the biological text data to generate parsed biological text data; identifying biological process text in the parsed biological text data; The biological process text is time-calibrated to generate calibrated biological text data.

9. The method for generating a prediction model of a gene regulatory network according to claim 1, characterized in that: The aligning the global gene expression feature with the global text feature to obtain a contrast loss includes: Calculating the similarity of feature vectors according to the global gene expression feature and the global text feature; Calculating the contrast loss between each gene expression feature and the text feature in the global gene expression feature according to the similarity of the feature vector, and obtaining a loss function from gene expression to text; Calculating the contrast loss between each text feature in the global text feature and the gene expression feature according to the similarity of the feature vector to obtain a loss function from text to gene expression; The contrast loss is calculated based on the loss function from the gene expression to the text and the loss function from the text to the gene expression.

10. The method for generating a prediction model of a gene regulatory network according to claim 9, characterized in that: The formula for calculating and generating the contrast loss based on the loss function from the gene expression to the text and the loss function from the text to the gene expression is: In the formula, represents contrast loss; Represents the loss function of gene expression to text; Represents the loss function for text-to-gene expression.

11. The method for generating a prediction model of a gene regulatory network according to any one of claims 1 to 10, characterized in that: The optimized prediction model is trained by a cosine decay learning rate scheduling mechanism to generate a trained prediction model, including: Obtaining an initial learning rate of the optimized prediction model; Calculate the learning rate after iteration according to the set number of iterations and the initial learning rate; The optimized prediction model is trained according to the iterated learning rate to generate a trained prediction model.

12. The method for generating a prediction model of a gene regulatory network according to claim 11, characterized in that: The formula for calculating the learning rate after iteration according to the set number of iterations and the initial learning rate is: In the formula, Represents the learning rate after iteration; represents the initial learning rate; Indicates the number of iterations; Represents the total number of training steps.

13. A server, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of the method for generating a prediction model of a gene regulatory network as claimed in any one of claims 1 to 12 when executing the computer program.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the steps of the method for generating a prediction model of a gene regulatory network as claimed in any one of claims 1 to 12.

15. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method for generating a prediction model of a gene regulatory network as claimed in any one of claims 1 to 12 are implemented.

Images