A method for targeted design optimization of mRNA 5′ untranslated region based on generative language models and reinforcement learning

Abstract

This invention relates to a method for targeted design and optimization of the 5′ untranslated region (UTR) of mRNA based on generative language models and reinforcement learning. The method includes: constructing pre-training data for the 5′ UTR; obtaining a prediction task dataset; calculating the minimum free energy (MFE) corresponding to the pre-training data; constructing a generative language model and performing phased pre-training on the generative language model using the pre-training data and the MFE to obtain a generative model and a prediction task fine-tuning model; fine-tuning the prediction task fine-tuning model based on the prediction task dataset to obtain a functional prediction model; evaluating the generative model's generation capability in the 5′ UTR sequence generation task; and performing targeted optimization design of the 5′ UTR sequence based on a reinforcement learning framework, combining the generative model and the functional prediction model. This invention can efficiently generate 5′ UTR sequences with specific biological functions, providing strong support for functional mRNA design and showing broad application prospects.

Description

Technical Field

[0001] This invention relates to the fields of bioinformatics and computer application technology, and in particular to a method for targeted design optimization of the 5′ untranslated region of mRNA based on generative language models and reinforcement learning. Background Technology

[0002] Messenger RNA (mRNA) gene therapy and vaccine development have dramatically changed the way diseases are treated, achieving significant progress in recent years in various diseases such as autoimmune diseases, pulmonary fibrosis, COVID-19, and cancer, highlighting the crucial role of mRNA therapy in improving treatment outcomes. However, the efficacy and sustainability of mRNA therapy are highly dependent on whether the delivered mRNA can be fully and accurately translated into functional proteins. This process is influenced by various factors such as mRNA stability, translation efficiency, and immunogenicity. Within the regulatory regions of mRNA molecules, the 5′ untranslated region (5′ UTR), located upstream of the coding region, is a key region affecting mRNA stability, localization, and translation efficiency. In eukaryotes, the cap structure of the 5′ UTR is responsible for recruiting ribosomes, which then scan along the 5′ UTR and recognize the start codon (AUG) to initiate translation. Specific motifs may also hinder or delay ribosome entry and scanning, reducing translation efficiency. Therefore, given the complex role of the 5′ UTR in gene regulation, its function in translation regulation has become a key research focus.

[0003] In recent years, various computational tools have been developed to study the functional properties of 5′ UTRs. For example, some supervised learning-based machine learning models can predict specific functional indicators, including mean ribosome load (MRL), mRNA translation efficiency (TE), and expression level (EL), based on 5′ UTR sequences or their biological characteristics. While these models provide effective tools for 5′ UTR functional prediction, most only target a single functional indicator and lack the ability to predict multiple functions uniformly. Therefore, some researchers have drawn on language modeling methods from Natural Language Processing (NLP) to develop language models for mRNA to decode the multiple functions of 5′ UTRs. These models mainly capture the contextual information of sequences through masked language modeling (MLM) and have shown high performance in functional prediction. However, these methods mainly focus on the functional characterization of existing 5′ UTR sequences and are usually limited to screening known 5′ UTR libraries. It should be noted that natural 5′ UTRs themselves are highly heterogeneous, with significant differences in sequence length, structure, and regulatory function, while the theoretically possible space of 5′ UTR sequences is even larger. Relying solely on screening limited libraries makes it difficult to systematically obtain 5′ UTR sequences with ideal functions. In contrast, generative models can overcome library limitations, exploring and designing novel 5′ UTR sequences with superior functions in a broader theoretical sequence space. Therefore, there is an urgent need to develop a unified method that combines functional prediction and sequence generation to support multidimensional functional decoding of mRNA 5′ UTRs and the targeted optimization design of novel sequences. Summary of the Invention

[0004] The purpose of this invention is to propose a method for targeted design optimization of the 5′ untranslated region of mRNA based on generative language models and reinforcement learning, so as to solve the problems existing in the prior art.

[0005] To achieve the above objectives, the present invention provides the following solution:

[0006] A method for targeted design optimization of mRNA 5′ untranslated region based on generative language models and reinforcement learning, comprising:

[0007] Construct a 5′ UTR sequence dataset and perform data preprocessing to obtain pre-training data;

[0008] Obtain the prediction task dataset; wherein, the prediction task dataset includes: MRL prediction task, TE prediction task and EL prediction task;

[0009] Calculate the minimum free energy value MFE corresponding to the pre-trained data;

[0010] A generative language model is constructed, and the generative language model is pre-trained in stages using the pre-training data and the minimum free energy value to obtain a generative model and a prediction task fine-tuning model. The generative model is used to generate 5′ UTR sequences, and the prediction task fine-tuning model is used for subsequent downstream function prediction task fine-tuning.

[0011] Based on the prediction task dataset, fine-tune the prediction task fine-tuning model to obtain the functional prediction model;

[0012] Based on the reinforcement learning framework, and combining the generative model and the functional prediction model, a targeted optimization design for the 5′ UTR sequence is carried out.

[0013] Optionally, the data preprocessing includes:

[0014] The collected dataset was dereduplicated and sequences were filtered to retain those that conformed to biological characteristics.

[0015] Optionally, the generative language model adopts a generative language model based on the GPT-2 architecture.

[0016] Optionally, the phased pre-training of the generative language model includes:

[0017] NTP task training: The pre-training data is segmented by a word segmenter and then input into the generative language model for NTP task training. The training objective is to predict the next base of the 5′ UTR sequence through context, and the cross-entropy loss function is used to evaluate the prediction accuracy of the model to obtain the generative model.

[0018] MFE regression task training: Fine-tuning the generative model using the minimum free energy value to obtain the fine-tuned model for the prediction task.

[0019] Optionally, the word segmenter employs a single nucleotide segmentation strategy.

[0020] Optionally, fine-tuning the generative model includes:

[0021] Based on the hidden representations of the last layer of the model, a ResNet regression head is added for minimum free energy (MFE) prediction.

[0022] The ResNet regression head consists of two one-dimensional convolutional layers and one fully connected layer.

[0023] Optionally, the training objective of the MFE regression task is to minimize the mean squared error loss function.

[0024] Optionally, fine-tuning the prediction task model includes:

[0025] Based on the prediction task fine-tuning model, the initial weights are inherited from the prediction task fine-tuning model. Based on the prediction task dataset, the mean squared error loss is used as the optimization objective to fine-tune the prediction task fine-tuning model and obtain the functional prediction model.

[0026] Optionally, based on the reinforcement learning framework, and combining the generative model and the function prediction model, the targeted optimization design of the 5′ UTR sequence includes:

[0027] The GRPO framework based on reinforcement learning is adopted, and the generative model is used as the policy model to generate candidate 5′UTR sequences.

[0028] The functional prediction model is used as the basis for the reward function to evaluate the function of the generated 5′ UTR sequence, and the prediction results are fed back to the generative model as a reward signal.

[0029] During GRPO training, the generative model continuously adjusts its parameters and optimizes its generation strategy based on the reward signal, thereby gradually enhancing the performance of the generated sequences on the target function.

[0030] The beneficial effects of this invention are as follows:

[0031] This invention combines generative language models with reinforcement learning to address the problem of targeted design and optimization of mRNA 5′ untranslated region (5′ UTR) sequences. It proposes a novel modeling framework that integrates phased pre-training with functional guidance. Specifically, this invention achieves efficient learning of the contextual dependencies and functional features of 5′ UTR sequences by standardizing a large-scale 5′ UTR dataset and introducing functional labels, employing a generative language model based on the GPT-2 architecture, and combining it with a minimum free energy (MFE) regression task. In the functional prediction stage, multi-task fine-tuning of multiple functional indicators such as MRL, TE, and EL significantly enhances the model's prediction accuracy for mRNA translation regulation-related functions. Furthermore, this invention innovatively introduces the GRPO reinforcement learning framework to co-optimize the sequence generation model and the functional prediction model, achieving targeted optimization design of 5′ UTR sequences. Compared with traditional empirical or high-throughput experimental screening methods, this invention can efficiently generate 5′ UTR sequences with specific biological functions, providing strong support for functional mRNA design and showing broad application prospects. Attached Figure Description

[0032] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0033] Figure 1 This is a flowchart illustrating the mRNA 5′ untranslated region targeted design optimization method based on generative language models and reinforcement learning, according to an embodiment of the present invention.

[0034] Figure 2 This is a schematic diagram illustrating the generation and functional prediction of the 5′ non-translated region based on a generative language model according to an embodiment of the present invention;

[0035] Figure 3 This is a schematic diagram of a reinforcement learning-based 5′ non-translated region orientation design optimization method according to an embodiment of the present invention;

[0036] Figure 4 This invention compares the model performance of TE and EL prediction in three cell lines (Muscle, PC3, and HEK) in an embodiment of the present invention; wherein, (a) shows the performance of each model in the TE prediction task, and (b) shows the performance of each model in the EL prediction task.

[0037] Figure 5 The distribution of minimum edit distances in this embodiment of the invention is as follows: (a) is the distribution of minimum edit distances between each generated 5′ UTR sequence and all pre-trained sequences; (b) is the distribution of minimum edit distances between generated sequences (excluding themselves).

[0038] Figure 6 Comparison of predicted distributions of GC content and MFE, MRL and TE among natural and random 5′ UTR sequences generated by UTRgen for embodiments of the present invention; wherein, (a) is a comparison of GC content distribution, (b) is a comparison of predicted MFE distribution, (c) is a comparison of predicted MRL distribution, and (d) is a comparison of predicted TE distribution.

[0039] Figure 7 This is a schematic diagram comparing the distribution of 5′ UTR sequences generated by UTRGen before and after optimization based on GRPO reinforcement learning in an embodiment of the present invention on the TE and EL functional indicators; wherein, (a) is the distribution change of 5′ UTR sequences on TE before and after optimization, and (b) is the distribution change of 5′ UTR sequences on EL before and after optimization. Detailed Implementation

[0040] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0041] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0042] The following is an explanation of the terms used in this embodiment:

[0043] 5′ UTR: The 5′ untranslated region of mRNA.

[0044] MRL: Mean Ribosome Load, used to measure the number of ribosomes translating a particular mRNA at a specific time point.

[0045] TE: Translation efficiency, which reflects the efficiency with which mRNA is translated into protein.

[0046] EL: Expression level, representing the relative abundance of intracellular mRNA transcripts.

[0047] GRPO: Guided Reinforcement Policy Optimization.

[0048] like Figure 1 As shown, this embodiment proposes a method for targeted design optimization of the 5′ untranslated region of mRNA based on generative language models and reinforcement learning, including:

[0049] Construct a 5′ UTR sequence dataset and perform data preprocessing to obtain pre-training data;

[0050] Obtain the prediction task dataset; wherein, the prediction task dataset includes: MRL prediction task, TE prediction task and EL prediction task;

[0051] Calculate the minimum free energy value MFE corresponding to the pre-trained data;

[0052] A generative language model is constructed, and the generative language model is pre-trained in stages using the pre-training data and the minimum free energy value to obtain a generative model and a prediction task fine-tuning model. The generative model is used to generate 5′ UTR sequences, and the prediction task fine-tuning model is used for subsequent downstream function prediction task fine-tuning.

[0053] Based on the prediction task dataset, fine-tune the prediction task fine-tuning model to obtain the functional prediction model;

[0054] Based on the reinforcement learning framework, and combining the generative model and the functional prediction model, a targeted optimization design for the 5′ UTR sequence is carried out.

[0055] Specifically, in this embodiment, S1, collecting and constructing a large-scale 5′ UTR sequence dataset: collecting 5′ UTR sequences from multiple species from public databases to construct a high-quality, diverse 5′ UTR dataset; then standardizing the collected dataset, including redundancy removal and sequence selection, retaining sequences that conform to biological characteristics, and generating the final pre-training data. This specifically includes:

[0056] First, mRNA 5′ UTR sequences from ten representative eukaryotes (including Human, Chicken, House Mouse, Norway Rat, Callithrix jacchus, Macaca mulatta, Zebrafish, Xenopustropicalis, Arabidopsis thaliana, and Drosophila melanogaster) were collected from the NCBI database. These sequences were extracted from annotated transcripts aligned with a reference genome assembly based on RefSeq gene annotations provided by NCBI. For each transcript, the region upstream of the translation start codon (AUG) in the annotation was extracted as the 5′ UTR. To ensure data quality, the 5′ UTR sequences underwent necessary filtering and redundancy removal. Specifically, empty sequences and sequences containing non-standard nucleotides were excluded, along with low-complexity sequences (such as polynucleotides AAAAAAAAA and repetitive sequences ATATATATATAT). Subsequently, the CD-HIT-EST tool was used to remove redundant sequences with a 95% identity threshold to ensure the uniqueness of each sequence. After the above preprocessing, approximately 820,000 high-quality, non-redundant 5′ UTR sequences were finally obtained, such as... Figure 2 As shown in (a).

[0057] Specifically, in this embodiment, S2, collecting the downstream functional prediction task dataset, including: MRL prediction, TE prediction, and EL prediction tasks. This specifically includes:

[0058] For downstream tasks of MRL prediction, four fixed-length 5′ UTR synthetic libraries were collected from the literature: U1, U2, mC-U1, and mC-U2. Each library contains approximately 280,000 random sequences of 50 nucleotides each. For downstream tasks of TE and EL prediction, endogenous gene datasets from three human cell types were collected from the literature: human muscle tissue, human prostate cancer cell line PC3 (PC3), and human embryonic kidney (HEK) 293T cell line.

[0059] Specifically, in this embodiment, S3, the minimum free energy (MFE) value corresponding to all pre-training data in S1 is calculated using ViennaRNA software, and it is used as a functional supervision signal to provide a reference for learning the functional characteristics of the model.

[0060] Specifically, in this embodiment, S4, constructing and pre-training a generative language model in stages, such as... Figure 2 As shown in (b). Specifically, it includes:

[0061] Define the word segmenter: adopt a single nucleotide word segmentation strategy, that is, each token represents a base (A,U,G,C), to ensure that the model captures sequence information and local features at the nucleotide level;

[0062] Model Architecture and Pre-training: The preprocessed data is segmented by a tokenizer and then input into a generative language model (decoder-only Transformer) based on the GPT-2 architecture. In pre-training, a strategy combining Next Token Prediction (NTP) and Minimum Free Energy (MFE) regression is adopted, enabling the model to simultaneously capture the contextual information and biological functional characteristics of the sequence.

[0063] The pre-training process is conducted in stages, as follows:

[0064] NTP task training: In the first stage, the model is trained on the NTP task, which aims to predict the next base of the 5′ UTR sequence through context, helping the model to understand the local and global dependencies in the sequence.

[0065] MFE Regression Task Training: After the NTP task is completed, a second stage of MFE regression task training is performed based on its weights. By adding a ResNet regression head to the last hidden representation layer of the model, the MFE value is estimated, and the training objective is to minimize the mean squared error (MSE), further enhancing the model's understanding of the functional features of the sequence.

[0066] Through the above steps, the pre-training of the generative language model is completed, and it is finally named UTRGen. The model that only completes the NTP task training is named UTRGen(+NTP), and the model that completes both the NTP and MFE regression tasks is named UTRGen(+NTP+MFE). UTRGen(+NTP) is used to generate 5′ UTR sequences, and UTRGen(+NTP+MFE) is used for fine-tuning in subsequent downstream function prediction tasks.

[0067] The NTP pre-training task is the foundational task of UTRGen, aiming to predict the next base in a 5′ UTR sequence based on context. Specifically, given a partial sequence... The model needs to predict the next base. ,in It belongs to the base set {A,U,G,C}. Where: A represents adenine, U represents uracil, G represents guanine, and C represents cytosine. Through this task, the model can learn the grammatical rules (such as sequence dependencies) and semantic information (such as contextual features of functional fragments) of the 5′ UTR sequence. The NTP task uses the cross-entropy loss function to evaluate the model's accuracy in predicting the next nucleotide. For a sequence... The cross-entropy loss function is defined as:

[0068]

[0069] in, Indicates sequence length (i.e., number of nucleotides); The size of the nucleotide set, here = {A,U,G,C}, therefore . Indicates position The one-hot encoding of a real nucleotide, for a real nucleotide have The rest are 0. Indicates the model at position The predicted value is a nucleotide. The probability of.

[0070] After completing the NTP task training, the base model UTRgen(+NTP) is obtained. Then, the MFE data obtained in step S3 is used as labels to fine-tune the model. Specifically, a ResNet regression head is added to the hidden representations of the last layer of the model for MFE prediction. This regression head consists of two one-dimensional convolutional layers and one fully connected layer. Its training objective is to minimize the mean squared error (MSE) loss function.

[0071]

[0072] Where N represents the total number of training samples, Indicates the first The true minimum free energy (MFE) value of the sequence. This represents the MFE value obtained from predicting the same sequence. After training through this stage, the model UTRGen(+NTP+MFE) is obtained. Among them, the base model UTRGen(+NTP) is used for the subsequent 5′ UTR sequence generation task, and UTRGen(+NTP+MFE) is used for the pre-training weight initialization of the downstream function prediction task.

[0073] Specifically, in this embodiment, S5, the weights of UTRgen(+NTP+MFE) are fine-tuned on a specific function prediction task dataset. First, a 5′ UTR function prediction task is performed, which aims to predict three function metrics: MRL, TE, and EL (see the glossary section for their specific meanings). After fine-tuning, the function prediction models are obtained: UTRgen-MRL, UTRgen-TE, and UTRgen-EL. This specifically includes:

[0074] After obtaining the pre-trained embedding model UTRGen(+NTP+MFE), it was further fine-tuned on three downstream function prediction task datasets. Its performance in different function prediction tasks was systematically evaluated, including MRL prediction, mRNA TE prediction, and EL prediction. Figure 2 As shown in (c). The specific implementation is as follows: Based on the last hidden representation layer of the model, a ResNet regression head is added for regression prediction of MRL, TE, or EL, with initial weights inherited from the pre-trained model UTRGen(+NTP+MFE). The mean squared error (MSE) loss is also used as the optimization objective.

[0075]

[0076] Where N represents the total number of training samples, Indicates the first The true MRL (or TE or EL) value of the sequence. This represents the MRL (or TE or EL) value predicted for the same sequence. Through this step, models for 5′ UTR MRL prediction, TE prediction, and EL prediction can be obtained, named UTRGen-MRL, UTRGen-TE, and UTRGen-EL respectively. The results are as follows:

[0077] For the MRL prediction task: UTRGen was first compared with six baseline methods on four fixed-length synthetic libraries (U1, U2, mC-U1, and mC-U2). Following the experimental setup of UTR-LM, two data splitting strategies were employed: Rank splitting and Random splitting. In Rank splitting, the top 20,000 5′ UTRs with the highest read counts were used as the test set, and the remaining sequences were used for training. In Random splitting, the dataset was randomly split into a training set of 260,000 sequences and a test set of 20,000 sequences. As shown in Table 1, UTRGen consistently maintained the best performance across all evaluations. Specifically, in Spearman R, UTRGen outperformed UTR-LM by 0.03, Optimus by 0.11, FramePool by 0.08, and RNA-FM by 0.47.

[0078] Table 1. Comparison of prediction performance of UTRGen with six baseline methods on the MRL task.

[0079]

[0080] For TE and EL prediction tasks: the pre-trained UTRGenmeng was fine-tuned on three 5′ UTR datasets (Muscle, PC3, HEK) using 10-fold cross-validation, and compared with eight mainstream baseline methods (RNABERT, RNA-FM, MTtrans, FramePool, Cao-RF, Optimus, UTR-LM, and mRNA2vec). Figure 4 As shown. All experimental settings were consistent with UTR-LM. All baseline results, except for mRNA2vec, were derived from UTR-LM. Because the original mRNA2vec did not use 10-fold cross-validation for TE and EL predictions, mRNA2vec was retrained under the same settings for fair comparison. Figure 3 As shown, UTRGen achieved state-of-the-art performance on all datasets for both the TE and EL prediction tasks; Figure 4 (a) shows the performance of each model in the TE prediction task. Figure 4(b) shows the performance of each model in the EL prediction task. Compared with the state-of-the-art baseline UTR-LM, UTRGen improves the Spearman R of TE prediction by 6%, 5% and 7% on the Muscle, PC3 and HEK datasets, respectively, and improves the Spearman R of EL prediction by 11%, 2% and 3%.

[0081] Specifically, in this embodiment, step S6 is also set up to evaluate the generation capability of the base model UTRGen(+NTP) obtained in step S4 in the 5′ UTR sequence generation task. This specifically includes:

[0082] The evaluation of the generation capability of the base model UTRGen(+NTP) obtained in step S4 in the 5′ UTR sequence generation task includes the following steps: First, using the UTRGen base model trained only on the next base prediction (NTP) task, a certain number of 5′ UTR sequences are generated according to different length intervals (e.g., 25-50nt, 50-100nt, 100-150nt, 150-200nt).

[0083] Novelty Assessment: For each generated sequence, the minimum Levenshtein distance between it and all pre-trained natural 5′ UTR sequences is calculated. This approach is used to effectively determine whether the generated sequence has a high similarity to existing natural sequences, thus ensuring the generated sequence is innovative and avoiding the model simply copying the training data. Specifically, the minimum edit distance between each generated sequence and all natural sequences in the pre-training set (approximately 820,000 sequences) is calculated. Figure 5 As shown in (a), the minimum distance between all generated sequences and any pre-trained sequence is greater than 10, and the median distance is 52, which indicates that all generated sequences are novel and have sufficiently significant differences from the training data.

[0084] Diversity assessment: For each generated sequence, calculate its minimum Levenshtein distance to all other generated sequences (excluding itself). The purpose of this design is to examine the diversity among the sequences generated by the model, thereby ensuring that the model has the ability to generate rich and diverse sequences, rather than only generating a limited number of repetitive or patterned sequences. Figure 5 As shown in (b), the minimum distance is above 10 and the median distance is 56, indicating significant diversity among the generated sequences.

[0085] Quality (fidelity) assessment: To assess fidelity, three sets of comparative sequence sets were designed: (i) sequences generated by UTRGen (2000 sequences generated in each of the four length intervals: 25-50 nt, 50-100 nt, 100-150 nt, and 150-200 nt, for a total of 8000 sequences); (ii) the natural 5′ UTR dataset (2000 sequences randomly sampled from the pre-training data for each of the above length intervals, for a total of 8000 sequences); and (iii) a random sequence set (2000 sequences for each length interval, for a total of 8000 sequences). Four evaluation metrics were used, including GC content distribution, predicted MRL distribution, predicted TE distribution, and predicted MFE distribution, to systematically compare the three sets of sequence sets. Figure 6 As shown in (a), the average GC content of generated sequences and natural samples is very close, and the distribution range is wider. The Mann-Whitney U test results show no significant difference between the two (p = 0.636, effect size = 0.015); while there is a significant difference between natural sequences and random sequences (p < 0.001, effect size = 0.842). This indicates that UTRGen can capture and reproduce the sequence structure features of natural 5′ UTRs well. For functional properties, the predicted MRL, TE, and MFE values ​​were compared among the three groups. MRL and TE were predicted using a UTRGen model fine-tuned on the U2 and PC3 datasets, representing the translation rate and efficiency of mRNA, while MFE was calculated using VienaRNA as a measure of structural stability. The results show that the MFE between generated sequences and natural sequences (p = 0.149, effect size = 0.013; Figure 6 (b) and MRL (p = 0.309, effect size = 0.008); Figure 6 (c) or TE (p = 0.058, effect size = 0.061); Figure 6 There was no significant difference between (d) and the natural sequence, while the difference between the random sequence and the natural sequence was significant in all cases (p < 0.05).

[0086] Specifically, in this embodiment, S7, based on the reinforcement learning GRPO (Guided Reinforcement Policy Optimization) framework, and combining the generative model UTRgen(+NTP) and functional prediction models (UTRGen-MRL, UTRgen-TE, and UTRgen-EL) obtained in the above steps, a directional optimization design for the 5′ UTR sequence is performed. This specifically includes:

[0087] Based on the reinforcement learning GRPO framework, and combining the generative model UTRgen(+NTP) and the function prediction model obtained in the above steps, a targeted optimization design for the 5′ UTR sequence is performed (e.g., Figure 2 (As shown). Specifically, the GRPO framework based on reinforcement learning is adopted, using the sequence generation model UTRgen(+NTP) obtained in the previous steps as the policy model to generate candidate 5′ UTR sequences. Simultaneously, a functional prediction model (UTRGen-TE or UTRgen-EL) is used as the basis for the reward function to evaluate the performance of the generated 5′ UTR sequences on biological functional indicators such as TE and EL, and the prediction results are fed back to the generation model as a reward signal. During GRPO training, the generation model continuously adjusts its parameters based on the reward signal, optimizing the generation strategy and gradually enhancing the performance of the generated sequences on the target function. In this way, targeted optimization design of 5′ UTR sequences can be achieved, enabling the generated sequences to meet the expected requirements on specific functional indicators (such as high TE or specific EL).

[0088] In practical applications, this invention uses TE and EL as environmental rewards respectively to train two sets of optimization models, and uses the trained UTRGen-TE and UTRGen-EL to perform functional scoring on the generated sequences. The TE and EL distributions of 2000 sequences generated by UTRGen before and after optimization are compared (e.g., ...). Figure 7 As shown in the figure, the results show that the TE and EL indices of the generated sequence are significantly improved after GRPO reinforcement learning, which fully verifies the effectiveness and advantages of the method of the present invention in the functionally oriented optimization design of 5′ UTR.

[0089] The following embodiment uses the human embryonic kidney (HEK) 293T cell line TE prediction dataset as an example to illustrate a method for targeted design optimization of the mRNA 5′ untranslated region based on generative language models and reinforcement learning, including the following steps:

[0090] S1. Collection and Construction of a Large-Scale 5′ UTR Sequence Dataset: First, mRNA 5′ UTR sequences from ten representative eukaryotes (including Human, Chicken, House Mouse, Norway Rat, Callithrix jacchus, Macacamulatta, Zebrafish, Xenopus tropicalis, Arabidopsis thaliana, and Drosophila melanogaster) were collected from the NCBI database. These sequences were extracted from annotated transcripts aligned with a reference genome assembly based on RefSeq gene annotations provided by NCBI. For each transcript, the region upstream of the translation start codon (AUG) in the annotation was extracted as the 5′ UTR. To ensure data quality, the 5′ UTR sequences underwent necessary filtering and redundancy removal. Specifically, empty sequences and sequences containing non-standard nucleotides were excluded, while low-complexity sequences (such as polynucleotides AAAAAAAAA and repetitive sequences ATATATATATAT) were removed. Subsequently, the CD-HIT-EST tool was used to remove redundant sequences with a 95% identity threshold to ensure the uniqueness of each sequence. After the above preprocessing, approximately 820,000 high-quality, non-redundant 5′ UTR sequences were finally obtained.

[0091] S2. Collect downstream functional prediction task dataset: For the downstream task of TE prediction, collect the endogenous gene dataset from the human embryonic kidney (HEK) 293T cell line—HEK TE—from the literature.

[0092] S3. Use ViennaRNA software to calculate the minimum free energy (MFE) value corresponding to all pre-training data in S1, and use it as a functional supervision signal to provide a reference for learning the functional characteristics of the model.

[0093] S4. Construct and pre-train generative language models in stages:

[0094] Define the word segmenter: adopt a single nucleotide word segmentation strategy, that is, each token represents a base (A,U,G,C), to ensure that the model captures sequence information and local features at the nucleotide level;

[0095] Model Architecture and Pre-training: The preprocessed data is segmented by a tokenizer and then input into a generative language model (decoder-only Transformer) based on the GPT-2 architecture. During pre-training, a strategy combining Next Token Prediction (NTP) and Minimum Free Energy (MFE) regression is employed to enable the model to simultaneously capture the contextual information and biological functional characteristics of the sequence.

[0096] The pre-training process is conducted in stages, as follows:

[0097] The NTP pre-training task is the foundational task of UTRGen, aiming to predict the next base in a 5′ UTR sequence based on context. Specifically, given a partial sequence... The model needs to predict the next base. ,in It belongs to the base set {A,U,G,C}. Where: A represents adenine, U represents uracil, G represents guanine, and C represents cytosine. Through this task, the model can learn the grammatical rules (such as sequence dependencies) and semantic information (such as contextual features of functional fragments) of the 5′ UTR sequence. The NTP task uses the cross-entropy loss function to evaluate the model's accuracy in predicting the next nucleotide. For a sequence... The cross-entropy loss function is defined as:

[0098]

[0099] in, Indicates sequence length (i.e., number of nucleotides); The size of the nucleotide set, here = {A,U,G,C}, therefore . Indicates position The one-hot encoding of a real nucleotide, for a real nucleotide have The rest are 0. Indicates the model at position The predicted value is a nucleotide. The probability of.

[0100] After completing the NTP task training, the base model UTRgen(+NTP) is obtained. Then, the MFE data obtained in step S3 is used as labels to fine-tune the model. Specifically, a ResNet regression head is added to the hidden representations of the last layer of the model for MFE prediction. This regression head consists of two one-dimensional convolutional layers and one fully connected layer. Its training objective is to minimize the mean squared error (MSE) loss function.

[0101]

[0102] Where N represents the total number of training samples, Indicates the first The true minimum free energy (MFE) value of the sequence. This represents the MFE value obtained from predicting the same sequence. After training through this stage, the model UTRGen(+NTP+MFE) is obtained. Among them, the base model UTRGen(+NTP) is used for the subsequent 5′ UTR sequence generation task, and UTRGen(+NTP+MFE) is used for the pre-training weight initialization of the downstream function prediction task.

[0103] S5. After obtaining the pre-trained embedding model UTRGen(+NTP+MFE), further fine-tuning is performed on the HEK TE prediction task dataset. Specifically, ResNet regression heads are added to the last hidden representation layer of the model for TE regression prediction, with initial weights inherited from the pre-trained model UTRGen(+NTP+MFE). The mean squared error (MSE) loss is also used as the optimization objective.

[0104]

[0105] Where N represents the total number of training samples, Indicates the first The true TE value of the sequence, This represents the TE value predicted for the same sequence. Through this step, models for 5′ UTR TE prediction can be obtained, named UTRgen-TE.

[0106] S6. Evaluate the generation capability of the base model UTRGen(+NTP) obtained in step S4 in the 5′ UTR sequence generation task. The specific method includes the following steps: First, using the UTRGen base model trained only on the next base prediction (NTP) task, generate a certain number of 5′ UTR sequences according to different length intervals (such as 25-50nt, 50-100nt, 100-150nt, 150-200nt).

[0107] Novelty Assessment: For each generated sequence, the minimum Levenshtein distance between it and all pre-trained natural 5′ UTR sequences is calculated. This approach is used to effectively determine whether the generated sequence has a high similarity to existing natural sequences, thus ensuring the generated sequence is innovative and avoiding the model simply copying the training data. Specifically, the minimum edit distance between each generated sequence and all natural sequences in the pre-training set (approximately 820,000 sequences) is calculated. Figure 5 As shown in (a), the minimum distance between all generated sequences and any pre-trained sequence is greater than 10, and the median distance is 52, which indicates that all generated sequences are novel and have sufficiently significant differences from the training data.

[0108] Diversity assessment: For each generated sequence, calculate its minimum Levenshtein distance to all other generated sequences (excluding itself). The purpose of this design is to examine the diversity among the sequences generated by the model, thereby ensuring that the model has the ability to generate rich and diverse sequences, rather than only generating a limited number of repetitive or patterned sequences. Figure 5 As shown in (b), the minimum distance is above 10 and the median distance is 56, indicating significant diversity among the generated sequences.

[0109] Quality (fidelity) assessment: To assess fidelity, three sets of comparative sequence sets were designed: (i) sequences generated by UTRGen (2000 sequences generated in each of the four length intervals: 25-50 nt, 50-100 nt, 100-150 nt, and 150-200 nt, for a total of 8000 sequences); (ii) the natural 5′ UTR dataset (2000 sequences randomly sampled from the pre-training data for each of the above length intervals, for a total of 8000 sequences); and (iii) a random sequence set (2000 sequences for each length interval, for a total of 8000 sequences). Four evaluation metrics were used, including GC content distribution, predicted MRL distribution, predicted TE distribution, and predicted MFE distribution, to systematically compare the three sets of sequence sets. Figure 6 As shown in (a), the average GC content of generated sequences and natural samples is very close, and the distribution range is wider. The Mann-Whitney U test results show no significant difference between the two (p = 0.636, effect size = 0.015); while there is a significant difference between natural sequences and random sequences (p < 0.001, effect size = 0.842). This indicates that UTRGen can capture and reproduce the sequence structure features of natural 5′ UTRs well. For functional properties, the predicted MRL, TE, and MFE values ​​were compared among the three groups. MRL and TE were predicted using a UTRGen model fine-tuned on the U2 and PC3 datasets, representing the translation rate and efficiency of mRNA, while MFE was calculated using VienaRNA as a measure of structural stability. The results show that the MFE between generated sequences and natural sequences (p = 0.149, effect size = 0.013; Figure 6 (b) and MRL (p = 0.309, effect size = 0.008); Figure 6 (c) or TE (p = 0.058, effect size = 0.061); Figure 6 There was no significant difference between (d) and the natural sequence, while the difference between the random sequence and the natural sequence was significant in all cases (p < 0.05).

[0110] S7. Based on the reinforcement learning GRPO framework, and combining the generative model UTRgen(+NTP) and the function prediction model obtained in the above steps, perform targeted optimization design of the 5′ UTR sequence (e.g., Figure 3 (As shown). Specifically, the GRPO framework based on reinforcement learning is adopted. The sequence generation model UTRgen(+NTP) obtained in the previous steps is used as the policy model to generate candidate 5′ UTR sequences. Simultaneously, the function prediction model (UTRGen-TE) is used as the basis for the reward function to evaluate the performance of the generated 5′ UTR sequences on the TE function metric, and the prediction result is fed back to the generation model as a reward signal. During GRPO training, the generation model continuously adjusts its parameters based on the reward signal, optimizing the generation strategy and gradually enhancing the performance of the generated sequences on the target function. In this way, targeted optimization design of 5′ UTR sequences can be achieved, enabling the generated sequences to meet the expected requirements on a specific function metric (high TE).

[0111] This example uses TE as the environmental reward to train and optimize the model, and then uses the trained UTRGen-TE to perform functional scoring on the generated sequences. The TE distribution of 2000 sequences generated by UTRGen before and after optimization is compared. Figure 7 As shown in (a), the results show that the TE score of the generated sequence is significantly improved after GRPO reinforcement learning, which fully verifies the effectiveness and advantages of the method of the present invention in the functional orientation optimization design of 5′ UTR.

[0112] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made to the technical solutions of the present invention by those skilled in the art without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. A method for targeted design optimization of mRNA 5′ untranslated region based on generative language models and reinforcement learning, characterized in that, include: Construct a 5′ UTR sequence dataset and perform data preprocessing to obtain pre-training data; Obtain the prediction task dataset; wherein, the prediction task dataset includes: MRL prediction task, TE prediction task and EL prediction task; Calculate the minimum free energy value MFE corresponding to the pre-trained data; A generative language model is constructed, and the pre-training data and minimum free energy value are used to perform phased pre-training on the generative language model to obtain a generative model and a prediction task fine-tuning model. The generative model is used to generate 5′UTR sequences, and the prediction task fine-tuning model is used for subsequent downstream function prediction task fine-tuning. The phased pre-training of the generative language model includes: NTP task training: The pre-training data is segmented by a word segmenter and then input into the generative language model for NTP task training. The training objective is to predict the next base of the 5′ UTR sequence based on context, and the cross-entropy loss function is used to evaluate the prediction accuracy of the model to obtain the generative model. The word segmenter adopts a single nucleotide segmentation strategy. MFE regression task training: Fine-tuning the generative model using the minimum free energy value to obtain the fine-tuned model for the prediction task; Fine-tuning the generative model includes: Based on the hidden representations of the last layer of the model, a ResNet regression head is added for minimum free energy (MFE) prediction. The ResNet regression head includes: two one-dimensional convolutional layers and one fully connected layer; Based on the prediction task dataset, fine-tune the prediction task fine-tuning model to obtain the functional prediction model; Based on the reinforcement learning framework, and combining the generative model and the functional prediction model, a targeted optimization design for the 5′ UTR sequence is carried out. Based on the reinforcement learning framework, and combining the generative model and the function prediction model, the targeted optimization design of the 5′ UTR sequence includes: The GRPO framework based on reinforcement learning is adopted, and the generative model is used as a policy model to generate candidate 5′ UTR sequences. The functional prediction model is used as the basis for the reward function to evaluate the function of the generated 5′ UTR sequence, and the prediction results are fed back to the generative model as a reward signal. During GRPO training, the generative model continuously adjusts its parameters and optimizes its generation strategy based on the reward signal, thereby gradually enhancing the performance of the generated sequences on the target function.

2. The method for targeted design and optimization of mRNA 5′ untranslated region based on generative language models and reinforcement learning according to claim 1, characterized in that, The data preprocessing includes: The collected dataset was dereduplicated and sequences were filtered to retain those that conformed to biological characteristics.

3. The method for targeted design optimization of mRNA 5′ untranslated region based on generative language models and reinforcement learning according to claim 1, characterized in that, The generative language model described adopts a generative language model based on the GPT-2 architecture.

4. The method for targeted design optimization of mRNA 5′ untranslated region based on generative language models and reinforcement learning according to claim 1, characterized in that, The training objective of the MFE regression task is to minimize the mean squared error loss function.

5. The method for targeted design optimization of mRNA 5′ untranslated region based on generative language models and reinforcement learning according to claim 1, characterized in that, Fine-tuning the prediction task includes: Based on the prediction task fine-tuning model, the initial weights are inherited from the prediction task fine-tuning model. Based on the prediction task dataset, the mean squared error loss is used as the optimization objective to fine-tune the prediction task fine-tuning model and obtain the functional prediction model.

Images