A method of scoring mRNA sequences and related apparatus

Abstract

The application discloses an mRNA sequence scoring method and related equipment, and the method comprises the following steps: acquiring an mRNA sequence to be scored; extracting a key base sequence from the mRNA sequence, wherein the key base sequence comprises a start codon context and a stop codon context; inputting the key base sequence into a target scoring model to obtain a score of the mRNA sequence on a target index; wherein the target scoring model comprises a multi-modal RNA sequence encoding module and a large language model, and the target scoring model is obtained based on unsupervised pre-training of transcription sample data and supervised fine-tuning of a score label data set corresponding to the target index. The application integrates non-coding regions and key regions of coding regions (start / stop codon contexts) as inputs, and processes them by using a large language model trained based on massive data, so that the application can comprehensively consider various complex factors such as sequence context information and secondary structure, and can be widely applied to the technical field of gene sequence analysis.

Description

Technical Field

[0001] This invention relates to the field of gene sequence analysis technology, and in particular to an mRNA sequence scoring method and related equipment. Background Technology

[0002] mRNA vaccines have shown great potential in the treatment of respiratory diseases, oncology, and rare diseases. Their effectiveness depends on the translation efficiency and protein expression level of the mRNA sequence. Current methods for screening high-quality mRNA sequences largely rely on traditional experimental techniques, which are not only time-consuming and labor-intensive but also suffer from low throughput and difficulty in comprehensively considering multiple factors, failing to meet the high-efficiency requirements of practical applications. Existing technologies have shortcomings in evaluating the effectiveness of mRNA sequences, making it difficult to comprehensively and accurately screen sequences with higher translation efficiency and protein expression levels. This, to some extent, limits the progress in gene therapy, protein drug development, and other fields.

[0003] Specifically, the limitations of the experimental method are as follows:

[0004] (1) Low experimental throughput: Traditional experimental methods, such as fluorescent reporter gene assay, usually require the translation efficiency of individual mRNA sequences to be detected one by one. It is difficult to detect and screen a large number of mRNA sequences in a high-throughput manner, which limits the rapid identification of sequences with high translation efficiency in a large-scale mRNA sequence library.

[0005] (2) Large differences in experimental conditions: Different experiments have differences in operating conditions, cell types, detection instruments, etc., which leads to poor comparability of experimental results and makes it difficult to accurately assess the translation efficiency of different mRNA sequences under a unified standard. This is not conducive to a comprehensive and objective comparison and analysis of mRNA translation efficiency.

[0006] The limitations of bioinformatics methods are as follows:

[0007] (1) Simple model and lack of sufficient training data: Early bioinformatics methods were based on relatively simple models and limited training data, which could not fully cover the influence of complex structures and regulatory elements in mRNA sequences on translation efficiency, resulting in low accuracy in predicting translation efficiency.

[0008] (2) Insufficient consideration of sequence context information: Some methods only focus on local sequence features when analyzing mRNA sequences, which makes the evaluation results inaccurate and incomplete.

[0009] (3) Difficulty in integrating multiple experimental indicators: Previous bioinformatics methods can only be analyzed based on a single or a few experimental indicators. However, in reality, there are many factors that affect translation efficiency and involve multiple experimental data. These methods are difficult to effectively integrate multiple experimental indicators to comprehensively evaluate mRNA translation efficiency. Summary of the Invention

[0010] The main objective of this invention is to provide an mRNA sequence scoring method, apparatus, electronic device, storage medium, and program product, aiming to solve at least one problem of the prior art.

[0011] To achieve the above objectives, one aspect of this invention proposes an mRNA sequence scoring method, the method comprising:

[0012] Obtain the mRNA sequence to be scored. The mRNA sequence includes a non-coding region and a coding region. The non-coding region includes a 5' non-coding region located upstream of the start end of the coding region and a 3' non-coding region located downstream of the end end of the coding region.

[0013] Key base sequences are extracted from mRNA sequences. These key base sequences include start codon context and stop codon context. The start codon context represents the codon sequence at the start codon with a predetermined window length, and the stop codon context represents the codon sequence at the stop codon with a predetermined window length.

[0014] The key base sequence is input into the target scoring model, and the mRNA sequence is processed to obtain the score of the target index.

[0015] The target scoring model includes a multimodal RNA sequence encoding module and a large language model. The target scoring model is obtained by unsupervised pre-training based on transcription sample data and supervised fine-tuning based on the score label dataset corresponding to the target index.

[0016] In some embodiments, extracting key base sequences from mRNA sequences includes the following steps:

[0017] Using the first nucleotide position of the start codon and the last nucleotide position of the stop codon as dual-center reference points, the mRNA sequence is normalized and aligned to obtain the aligned sequence.

[0018] Based on the alignment sequence, a sequence of a preset length upstream of the center and a sequence of a preset length downstream of the center are extracted for the center corresponding to the start codon to form the start codon context; wherein, the preset length is half of the preset window length;

[0019] Based on the alignment sequence, a sequence of a preset length upstream of the center and a sequence of a preset length downstream of the center are extracted for the center corresponding to the stop codon to form the stop codon context;

[0020] The start codon context and the stop codon context are marked and separated to form the key base sequence.

[0021] In some embodiments, the multimodal RNA sequence coding module includes a one-dimensional coding unit and a two-dimensional coding unit. Key base sequences are input into a target scoring model, and the model processes the data to obtain a score for the mRNA sequence on a target indicator. This process includes the following steps:

[0022] Key base sequences are input into the multimodal RNA sequence coding module, and coding information is obtained by integrating base composition information and secondary structure features;

[0023] Among them, the base composition information is obtained by processing one-dimensional coding units, and the secondary structure features are obtained by processing two-dimensional coding units;

[0024] Based on the encoded information, a large language model is used to process the output mRNA sequence and score the target indicator.

[0025] In some embodiments, encoding information is obtained by integrating base composition information with secondary structure features, including the following steps:

[0026] One-hot encoding of key base sequences is performed using one-dimensional coding units to obtain base composition information;

[0027] Based on key base sequences, a two-dimensional base pairing matrix is ​​generated by two-dimensional coding units based on base complementary pairing rules as secondary structure features.

[0028] Encoding information is obtained by integrating base composition information and secondary structure characteristics.

[0029] In some embodiments, encoding information is obtained by integrating base composition information and secondary structure features, including the following steps:

[0030] The secondary structure features are weighted using an attention matrix;

[0031] By using matrix multiplication, the weighted result is fused with the base composition information to obtain the encoded information.

[0032] In some embodiments, the method further includes the following steps:

[0033] The initial model is configured based on the multimodal RNA sequence coding module and the large language model;

[0034] The multimodal RNA sequence coding module includes a one-dimensional coding unit and a two-dimensional coding unit. The one-dimensional coding unit is used to process and obtain base composition information, and the two-dimensional coding unit is used to process and obtain secondary structure features.

[0035] Based on transcription sample data, the initial model was unsupervised pre-trained using a position-aware mask prediction task, so that the initial model could learn the sequence patterns of mRNA sequences and obtain a pre-trained model.

[0036] The target indicator's corresponding score label dataset includes multiple mRNA sequence samples and the score label of each mRNA sequence sample on the target indicator.

[0037] Based on mRNA sequence samples and score labels, the pre-trained model is fine-tuned in a supervised manner to obtain the target scoring model.

[0038] In some embodiments, an initial model is unsupervised pre-trained based on transcriptional sample data using a position-aware mask prediction task, including the following steps:

[0039] The mask probability is constructed based on two Gaussian distributions centered on the start and stop codons;

[0040] Based on the masking probability, a portion of the transcribed sample sequence in the transcribed sample data is masked to obtain Gaussian weighted masked data.

[0041] Among them, the two-dimensional coding unit integrates the secondary structure features of the sequence through attention mechanism, gating fusion and residual connection;

[0042] Unsupervised pre-training of the initial model is guided by the structural features of Gaussian weighted mask data and two-dimensional coding units.

[0043] In some embodiments, supervised fine-tuning of a pre-trained model based on mRNA sequence samples and score labels includes the following steps:

[0044] The key base sequences extracted from mRNA sequence samples are used as input data to the pre-trained model so that the pre-trained model can process and output the prediction scores of the corresponding mRNA sequence samples.

[0045] A loss function is constructed based on the predicted scores and the score labels corresponding to the mRNA sequence samples. The parameters of the pre-trained model are then fine-tuned using the loss function.

[0046] To achieve the above objectives, another aspect of the present invention provides an mRNA sequence scoring device, the device comprising:

[0047] The sequence acquisition module is used to acquire the mRNA sequence to be scored. The mRNA sequence includes a non-coding region and a coding region. The non-coding region includes a 5' non-coding region located upstream of the start end of the coding region and a 3' non-coding region located downstream of the end end of the coding region.

[0048] The sequence extraction module is used to extract key base sequences from mRNA sequences. The key base sequences include start codon context and stop codon context. The start codon context represents the codon sequence at the start codon with a preset window length, and the stop codon context represents the codon sequence at the stop codon with a preset window length.

[0049] The scoring module is used to input key base sequences into the target scoring model and process them to obtain the score of the mRNA sequence on the target indicator.

[0050] The target scoring model includes a multimodal RNA sequence encoding module and a large language model. The target scoring model is obtained by unsupervised pre-training based on transcription sample data and supervised fine-tuning based on the score label dataset corresponding to the target index.

[0051] In some embodiments, the apparatus further includes a model training module for performing the following operations:

[0052] The initial model is configured based on the multimodal RNA sequence coding module and the large language model;

[0053] The multimodal RNA sequence coding module includes a one-dimensional coding unit and a two-dimensional coding unit. The one-dimensional coding unit is used to process and obtain base composition information, and the two-dimensional coding unit is used to process and obtain secondary structure features.

[0054] Based on transcription sample data, the initial model was unsupervised pre-trained using a position-aware mask prediction task, so that the initial model could learn the sequence patterns of mRNA sequences and obtain a pre-trained model.

[0055] The target indicator's corresponding score label dataset includes multiple mRNA sequence samples and the score label of each mRNA sequence sample on the target indicator.

[0056] Based on mRNA sequence samples and score labels, the pre-trained model is fine-tuned in a supervised manner to obtain the target scoring model.

[0057] To achieve the above objectives, another aspect of the present invention provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the aforementioned method.

[0058] To achieve the above objectives, another aspect of the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the aforementioned method.

[0059] To achieve the above objectives, another aspect of the present invention provides a computer program product, including a computer program that, when executed by a processor, implements the aforementioned method.

[0060] The embodiments of the present invention include at least the following beneficial effects: The present invention provides an mRNA sequence scoring method, apparatus, electronic device, storage medium, and program product. This scheme obtains an mRNA sequence to be scored, the mRNA sequence including a non-coding region and a coding region. The non-coding region includes a 5' non-coding region upstream of the start end of the coding region and a 3' non-coding region downstream of the stop end of the coding region. Key base sequences are extracted from the mRNA sequence, including a start codon context and a stop codon context. The start codon context represents a codon sequence with a preset window length at the start codon, and the stop codon context represents a codon sequence with a preset window length at the stop codon. The key base sequences are input into a target scoring model to obtain the score of the mRNA sequence on a target indicator. The target scoring model includes a multimodal RNA sequence encoding module and a large language model. The target scoring model is obtained based on unsupervised pre-training of transcription sample data and supervised fine-tuning based on a score label dataset corresponding to the target indicator. This invention replaces conventional experimental procedures with a model, enabling instant batch scoring of massive mRNA sequences. This solves the problems of low throughput and time-consuming, labor-intensive individual testing inherent in traditional methods, meeting the high-efficiency requirements of practical applications. Furthermore, the model-based scoring method processes all sequences using the same standardized model, resulting in highly consistent and comparable outputs. This overcomes the challenges of incomparable results and inconsistent evaluation standards caused by significant differences in experimental conditions. Simultaneously, this invention uses "key base sequences" integrating key regions (start / stop codon context) of the uncoding region (UTR) and coding region (CDS) as input, processed by a large language model trained on massive amounts of data. This allows the model to comprehensively consider sequence context information, secondary structure, and other complex factors, addressing the shortcomings of early bioinformatics methods, such as "simple models" and "insufficient consideration of sequence context information." Specifically, the use of labeled data for specific indicators during the "supervised fine-tuning" stage allows the same model framework to be adapted for accurate predictions of different indicators, providing a feasible technical path for "integrating multiple experimental indicators" and overcoming the limitations of previous methods in effectively integrating multiple indicators. Attached Figure Description

[0061] Figure 1 This is a schematic diagram of an implementation environment for the mRNA sequence scoring method provided in this embodiment of the invention;

[0062] Figure 2 This is a schematic flowchart of an mRNA sequence scoring method provided in an embodiment of the present invention;

[0063] Figure 3 This is a schematic diagram illustrating an example of the model architecture provided in an embodiment of the present invention;

[0064] Figure 4 This is a schematic diagram illustrating an example of comparing the average results of six RNA large language models trained and tested on eight benchmark tasks with different lengths, as provided in an embodiment of the present invention.

[0065] Figure 5 This is a schematic diagram illustrating the effect of comparing mRNA-UCS with existing large language models provided in an embodiment of the present invention;

[0066] Figure 6 This is a schematic diagram illustrating the results of the mRNA sequence scoring method provided in this embodiment of the invention, which scores mRNA sequences from multiple dimensions.

[0067] Figure 7 This is a schematic diagram of the structure of the mRNA sequence scoring device provided in an embodiment of the present invention;

[0068] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0069] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of this invention; they are merely examples of apparatuses and methods consistent with some aspects of the embodiments of this invention as detailed in the appended claims.

[0070] It is understood that the terms “first,” “second,” etc., used in this invention may be used to describe various concepts herein, but unless specifically stated otherwise, these concepts are not limited by these terms. These terms are used only to distinguish one concept from another. For example, first information may also be referred to as second information without departing from the scope of embodiments of the invention, and similarly, second information may also be referred to as first information. Depending on the context, the words “if,” “when,” or “in response to determination” as used herein may be interpreted as “when…” or “when…” or “in response to determination.”

[0071] The terms “at least one,” “multiple,” “each,” “any,” etc., used in this invention, “at least one” includes one, two, or more than two; “multiple” includes two or more than two; “each” refers to each of the corresponding multiple; and “any” refers to any one of the multiple.

[0072] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein is for the purpose of describing embodiments of the invention only and is not intended to limit the invention.

[0073] In related technologies, current methods for screening high-quality mRNA sequences largely rely on traditional experimental techniques, which are not only time-consuming and labor-intensive but also suffer from low throughput and difficulty in comprehensively considering multiple factors, making it difficult to meet the high-efficiency requirements of practical applications. Existing technologies have shortcomings in evaluating the effectiveness of mRNA sequences, making it difficult to comprehensively and accurately screen sequences with higher translation efficiency and protein expression levels, which to some extent limits the progress in gene therapy, protein drug development, and other fields.

[0074] In view of this, this invention provides an mRNA sequence scoring method and related equipment. The method involves acquiring an mRNA sequence to be scored, which includes a non-coding region and a coding region. The non-coding region includes a 5' non-coding region upstream of the start end of the coding region and a 3' non-coding region downstream of the stop end of the coding region. Key base sequences are extracted from the mRNA sequence, including a start codon context and a stop codon context. The start codon context represents a codon sequence with a preset window length at the start codon, and the stop codon context represents a codon sequence with a preset window length at the stop codon. The key base sequences are input into a target scoring model to obtain the mRNA sequence score on a target indicator. The target scoring model includes a multimodal RNA sequence encoding module and a large language model. The target scoring model is obtained through unsupervised pre-training based on transcription sample data and supervised fine-tuning based on a score label dataset corresponding to the target indicator. This invention replaces conventional experimental procedures with a model, enabling instant batch scoring of massive mRNA sequences. This solves the problems of low throughput and time-consuming, labor-intensive individual testing inherent in traditional methods, meeting the high-efficiency requirements of practical applications. Furthermore, the model-based scoring method processes all sequences using the same standardized model, resulting in highly consistent and comparable outputs. This overcomes the challenges of incomparable results and inconsistent evaluation standards caused by significant differences in experimental conditions. Simultaneously, this invention uses "key base sequences" integrating key regions (start / stop codon context) of the uncoding region (UTR) and coding region (CDS) as input, processed by a large language model trained on massive amounts of data. This allows the model to comprehensively consider sequence context information, secondary structure, and other complex factors, addressing the shortcomings of early bioinformatics methods, such as "simple models" and "insufficient consideration of sequence context information." Specifically, the use of labeled data for specific indicators during the "supervised fine-tuning" stage allows the same model framework to be adapted for accurate predictions of different indicators, providing a feasible technical path for "integrating multiple experimental indicators" and overcoming the limitations of previous methods in effectively integrating multiple indicators.

[0075] It is understood that the mRNA sequence scoring method provided by this invention can be applied to any computer device with data processing and computing capabilities, and this computer device can be various terminals or servers. When the computer device in the embodiments is a server, the server is an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. Optionally, the terminal can be a smartphone, tablet computer, laptop computer, or desktop computer, but it is not limited to these.

[0076] like Figure 1 The diagram shown is a schematic representation of an implementation environment provided by an embodiment of the present invention. (Refer to...) Figure 1 The implementation environment includes at least one terminal 102 and a server 101. The terminal 102 and the server 101 can be connected via a network, either wirelessly or via a wired connection, to complete data transmission and exchange.

[0077] Server 101 can be a standalone physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

[0078] Additionally, server 101 can also be a node server in a blockchain network. Blockchain is a novel application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms.

[0079] Terminal 102 can be a smartphone, tablet computer, laptop computer, desktop computer, smart speaker, smartwatch, etc., but is not limited to these. Terminal 102 and server 101 can be directly or indirectly connected via wired or wireless communication, and this embodiment of the invention does not impose any limitations.

[0080] For example, based on Figure 1 The implementation environment shown in this embodiment of the invention provides an mRNA sequence scoring method. The following description uses the application of this mRNA sequence scoring method in server 101 as an example. It can be understood that this mRNA sequence scoring method can also be applied in terminal 102.

[0081] Reference Figure 2 , Figure 2 This is an optional flowchart of the mRNA sequence scoring method provided in the embodiments of the present invention. The subject executing the mRNA sequence scoring method can be any of the aforementioned computer devices (including servers or terminals). Figure 2 The method may include, but is not limited to, steps S100 to S300.

[0082] Step S100: Obtain the mRNA sequence to be scored;

[0083] The mRNA sequence includes a non-coding region and a coding region. The non-coding region includes a 5' non-coding region located upstream of the start end of the coding region and a 3' non-coding region located downstream of the end end of the coding region.

[0084] Specifically, mRNA can be divided into different regions such as the 5' UTR (5' untranslated region), CDS (coding region), and 3' UTR (3' untranslated region). The UTR, or Untranslated Region, refers to a specific region outside the coding region in the genome. The UTR consists of two parts: the 5' UTR and the 3' UTR, located upstream of the start end and downstream of the end end of the coding region, respectively. The UTR plays an important role in gene expression regulation and mRNA stability.

[0085] Step S200: Extract key base sequences from the mRNA sequence;

[0086] The key base sequence includes the start codon context and the stop codon context. The start codon context represents the codon sequence with a preset window length at the start codon, and the stop codon context represents the codon sequence with a preset window length at the stop codon.

[0087] It should be noted that in some embodiments, step S200 may include the following steps: using the first nucleotide position of the start codon and the last nucleotide position of the stop codon as dual-center reference points, the mRNA sequence is normalized and aligned to obtain an aligned sequence; based on the aligned sequence, a sequence of a predetermined length upstream and downstream of the center corresponding to the start codon is truncated to form a start codon context; wherein, the predetermined length is half the length of a predetermined window; based on the aligned sequence, a sequence of a predetermined length upstream and downstream of the center corresponding to the stop codon is truncated to form a stop codon context; the start codon context and the stop codon context are marked and separated to form a key base sequence.

[0088] For example, in some specific implementations, the first nucleotide position of the start codon (AUG) (5'UTR-CDS boundary) and the last nucleotide position of the stop codon (UAA / UAG / UGA) (CDS-3'UTR boundary) are used as dual-center reference points to normalize and align all mRNA sequences. Based on this, 300bp segments are extracted upstream and downstream of each center to form two core analysis windows: the start codon context (-300bp to +300bp) and the stop codon context (-300bp to +300bp), thus constructing a core analysis region of 1200bp in total length (the beginning / end of the mRNA sequence is used for...). <s> / < / s> Mark the part that is not long enough. <pad>(The boundaries of the two core analysis windows are separated by NNN). Furthermore, this technology uses relative position encoding, where the position of each nucleotide is dynamically determined by its relative distance to the nearest center point. The relative distance between the two core windows is set to a large value, indicating a break. This encoding method can accurately identify the conserved sequence patterns and position-dependent features of the UTR-CDS connection region, effectively improving the model's ability to extract features from key regulatory elements of mRNA.

[0089] Specifically, this invention employs a "dual-center" normalization alignment based on the start / stop codons, ensuring that key functional regions of mRNA sequences of different lengths are positioned relatively consistently in the model input. This forces the model to focus on these conserved regions crucial for translation efficiency, reducing noise caused by sequence length differences and significantly improving the model's generalization ability and prediction accuracy for different sequences. This invention ensures that regardless of the original sequence, the model can accurately capture the core sequence context information required for the start and termination processes, directly addressing the problem of "insufficient consideration of sequence context information."

[0090] Step S300: Input the key base sequence into the target scoring model and process it to obtain the score of the mRNA sequence on the target index;

[0091] The target scoring model includes a multimodal RNA sequence encoding module and a large language model. The target scoring model is obtained by unsupervised pre-training based on transcription sample data and supervised fine-tuning based on the score label dataset corresponding to the target index.

[0092] It should be noted that the multimodal RNA sequence coding module includes one-dimensional coding units and two-dimensional coding units. In some embodiments, step S300 may include the following steps: inputting key base sequences into the multimodal RNA sequence coding module, and obtaining coding information by integrating base composition information and secondary structure features; wherein, the base composition information is obtained based on the one-dimensional coding unit, and the secondary structure features are obtained based on the two-dimensional coding unit; based on the coding information, the large language model is used to process and output the score of the mRNA sequence on the target indicator.

[0093] For example, in some specific embodiments, in view of RNA sequence characteristics and folding properties, the embodiments of the present invention employ a multimodal RNA sequence encoding method, which improves the model's ability to analyze RNA sequences by integrating base composition information and secondary structure features.

[0094] Specifically, this invention captures base composition (linear information of the sequence) through one-dimensional coding units and secondary structure features (spatial folding information of the sequence) through two-dimensional coding units, achieving deep fusion of different modal information of the mRNA sequence. The secondary structure of mRNA (such as hairpin structure) is a key factor affecting ribosome binding and translation process. This design enables the model to utilize both sequence and structural information simultaneously, greatly enhancing the model's ability to characterize the translation efficiency mechanism and resulting in more comprehensive and reliable predictions.

[0095] It should be noted that, in some embodiments, the encoding information obtained by integrating base composition information and secondary structure features may include the following steps: using a one-dimensional encoding unit to perform one-hot encoding on the key base sequence to obtain base composition information; based on the key base sequence, using a two-dimensional encoding unit to generate a two-dimensional base pairing matrix as a secondary structure feature based on the base complementary pairing rule; and integrating the base composition information and secondary structure features to obtain the encoding information.

[0096] For example, in some specific implementations, 1D (one-dimensional) one-hot encoding is used to characterize base composition information (e.g., A[1,0,0,0],U[0,1,0,0],C[0,0,1,0],G[0,0,0,1]); at the same time, 2D (two-dimensional) encoding is introduced to integrate base complementary pairing rules (Watson-Crick pairing rules) to generate a two-dimensional base pairing matrix (e.g., AU=2, GC=3, GU=0.8, representing the strength of different base pairings), enabling the model to automatically identify potential secondary structure features.

[0097] Specifically, embodiments of the present invention extract base composition information using "one-hot encoding," a simple and efficient feature representation method. Furthermore, embodiments of the present invention use a "two-dimensional base pairing matrix" to characterize the potential for complementary base pairing, providing direct and computable foundational data for the model to learn secondary structures. This combination provides the model with rich feature inputs containing both original sequence information and higher-order structural information, which is a key foundation for the model's success.

[0098] It should be noted that, in some embodiments, the process of integrating base composition information and secondary structure features to obtain encoded information may include the following steps: weighting the secondary structure features using an attention matrix; and fusing the weighted result into the base composition information through matrix multiplication to obtain encoded information.

[0099] For example, in some specific implementations, 2D encoding can be directly weighted with the attention matrix of the attention layer, and finally the weighted 2D information is fused into 1D encoding through matrix multiplication to obtain the encoding information that serves as the basis for scoring.

[0100] Specifically, this embodiment of the invention uses an "attention matrix" to weight secondary structure features, meaning the model can autonomously learn that the importance of secondary structures varies at different positions in the sequence (for example, strong hairpin structures near the start codon severely inhibit translation, while weaker structures in the middle of the CDS may have less impact). This dynamic, non-linear fusion method is more advanced than simple concatenation or addition, and can more intelligently integrate multimodal information, thereby making more accurate judgments.

[0101] It should be noted that in some embodiments, the method may further include the following steps: configuring an initial model based on a multimodal RNA sequence encoding module and a large language model; wherein, the multimodal RNA sequence encoding module includes one-dimensional encoding units and two-dimensional encoding units, the one-dimensional encoding units are used to process and obtain base composition information, and the two-dimensional encoding units are used to process and obtain secondary structure features; based on transcription sample data, the initial model is unsupervised pre-trained using a position-aware mask prediction task, so that the initial model learns the sequence rules of mRNA sequences, thereby obtaining a pre-trained model; wherein, the score label dataset corresponding to the target indicator includes multiple mRNA sequence samples and the score label of each mRNA sequence sample on the target indicator; based on the mRNA sequence samples and the score labels, the pre-trained model is subjected to supervised fine-tuning to obtain the target scoring model.

[0102] For example, in some specific embodiments, the present invention adopts a two-stage training paradigm of "unsupervised pre-training" + "supervised fine-tuning":

[0103] Unsupervised pre-training: It can utilize a large amount of publicly available transcriptome data (transcription sample data) that does not require manual annotation, allowing the model to first learn the basic syntax, general structure and general rules of mRNA sequences, solving the model initialization problem of "lack of sufficient training data" and laying a solid knowledge foundation for the model.

[0104] Supervised fine-tuning: Based on the pre-trained "generalist" model, fine-tuning is performed using a task-specific dataset, which may have a smaller amount of data but has accurate labels, to quickly make it an "expert" for that task. This method fully utilizes the advantages of data from different sources and can train high-performance models even on small sample labeled data.

[0105] It should be noted that, in some embodiments, unsupervised pre-training of the initial model based on transcription sample data through a position-aware mask prediction task may include the following steps: constructing mask probabilities based on two Gaussian distributions centered on the start and stop codons; masking a portion of the transcription sample sequence in the transcription sample data based on the mask probabilities to obtain Gaussian-weighted mask data; wherein, the two-dimensional coding unit integrates the secondary structural features of the sequence through attention mechanisms, gating fusion, and residual connections; and guiding the unsupervised pre-training of the initial model based on the Gaussian-weighted mask data and the structural features of the two-dimensional coding unit.

[0106] Exemplary examples, in some specific embodiments, the present invention employs a two-stage training strategy to optimize mRNA sequence modeling. The first stage involves unsupervised pre-training based on NCBI human transcription sample data, using an innovative position-aware mask prediction task to enable the model to grasp the basic patterns of mRNA. Specifically, this application can adjust certain regions to suit actual needs, for example, 10% to 20% of the region; this application does not impose specific limitations on this. Exemplary examples, the position-aware mask prediction task can mask 15% of the sequence region, using a Gaussian distribution centered on the start / stop codons, making the model focus more on the UTR-CDS connection region. The positions of the start / stop codons are derived from NCBI's annotation files, excluding sequences with a CDS region less than 100 or sequences with ambiguous annotations (e.g., CDS region length not a multiple of 3). Simultaneously, a two-dimensional base pairing matrix is ​​introduced to enhance the attention mechanism, integrating the secondary structure features of the sequence through gated fusion and residual connections. This pairing-biased attention mechanism can be used to calculate the correlation between any two bases, thereby deriving the long-range synergistic effect of two UTR-CDS connection regions. This stage enhances the model's ability to capture key regulatory elements of mRNA through Gaussian weighted masks and structural feature guidance, providing strong support for downstream tasks such as translation efficiency prediction in the second stage.

[0107] Specifically, traditional masked language models typically mask tokens randomly. This invention employs a Gaussian distribution centered on the start / stop codons to construct the mask probabilities, meaning the model is guided during pre-training to focus more on and predict these key functional regions and their surrounding tokens. This "position-aware" masking strategy makes the pre-training process more purposeful, allowing the model to learn and translate the most relevant sequence patterns more efficiently, accelerating convergence and improving final performance.

[0108] It should be noted that, in some embodiments, supervised fine-tuning of the pre-trained model based on mRNA sequence samples and score labels may include the following steps: inputting key base sequences extracted from mRNA sequence samples as input data into the pre-trained model so that the pre-trained model processes and outputs the predicted scores of the corresponding mRNA sequence samples; constructing a loss function based on the predicted scores and the score labels corresponding to the mRNA sequence samples; and using the loss function to fine-tune the parameters of the pre-trained model.

[0109] For example, in some specific implementations, the second stage involves supervised fine-tuning, optimizing model performance using task-specific labeled data based on pre-trained model parameters. This enables the model to recognize general sequence patterns while adapting to specific application requirements. This technique uses multiple experimental datasets to fine-tune a large language model (sequence-metric, supervised learning). The pre-trained large language model maps sequences to a high-dimensional space, adds an additional convolutional network as a decoding layer, and uses a regression prediction head to output predicted scores, obtaining a score for the given data scenario. Therefore, different datasets train different independent models to represent experimental metrics from different perspectives. The datasets used are from publicly available literature or websites, including ribosome load data measured under different cell states (normal / stressed) (polyribosome analysis experiments), translation efficiency data (paired Ribo-seq and RNA-seq experiments), protein expression data (mass spectrometry quantitative proteomics experiments, fluorescence reporter system experiments), and mRNA stability data (half-life determination experiments). During model evaluation, Spearman scores are used to assess the correlation between the model's predicted scores on the test set and the true scores on the dataset. The mRNA dataset containing synthetic sequences is divided into training, validation, and test sets in an 8:1:1 ratio. The mRNA dataset containing natural sequences is divided into training, validation (chromosome 10), and test (chromosome 15) sets based on their chromosome numbers.

[0110] Specifically, embodiments of the present invention compare the model's predicted scores with the actual experimentally measured score labels, and adjust the model parameters accordingly. This process ensures that the general knowledge learned by the pre-trained model can be effectively guided and corrected to accurately fit specific target indicators (such as protein expression levels), ultimately outputting reliable predicted scores. This is a necessary and effective step in transforming a general model into a dedicated scoring tool.

[0111] To explain in detail the principle of the technical solution of the present invention, the overall process of the present invention will be described below with reference to some specific embodiments. It is easy to understand that the following is an explanation of the technical principle of the present invention and should not be regarded as a limitation of the present invention.

[0112] First, it's important to note that existing mRNA sequence evaluation methods have significant technical limitations: Firstly, traditional algorithms only perform local analysis on short leader sequences (<30 nt) of the 5'UTR or 5'UTR+CDS, failing to reflect the functional characteristics of full-length mRNA. Secondly, most existing models focus only on a single region, neglecting the synergistic effect of the 5'UTR and 3'UTR in translation initiation, circularization, and termination processes, and also failing to assess the impact of different 5'UTR and 3'UTR combinations with CDS on translation performance. These limitations lead to significant discrepancies between predicted results and actual biological processes, making it difficult to guide practical mRNA design and optimization.

[0113] To address these issues, the technical solution of this invention innovatively develops the mRNA-UCS algorithm system (UTR-CDS scoring network, i.e., the framework of the target scoring model is detailed in [link to documentation]). Figure 3 This system, through an improved Transformer architecture, is the first to achieve co-modeling of the two key regions of mRNA (5'UTR-CDS and CDS-3'UTR), fully preserving the relative position information of the distal regions within a window of ±300bp between the start and stop codons.

[0114] In model design, mRNA-UCS focuses on key features of the UTR-CDS connection region. Based on the Transformer architecture, an innovative dual-center alignment strategy is employed: using the first nucleotide position of the start codon (AUG) (5'UTR-CDS boundary) and the last nucleotide position of the stop codon (UAA / UAG / UGA) (CDS-3'UTR boundary) as dual-center reference points, all mRNA sequences are normalized and aligned. Based on this, 300bp segments are extracted upstream and downstream of each center to form two core analysis windows: the start codon context (-300bp to +300bp) and the stop codon context (-300bp to +300bp), thus constructing a core analysis region of 1200bp in total length (the beginning / end of the mRNA sequence is used for...). <s> / < / s> Mark the part that is not long enough. <pad>(The boundaries of the two core analysis windows are separated by NNN). Furthermore, this technology uses relative position encoding, where the position of each nucleotide is dynamically determined by its relative distance to the nearest center point. The relative distance between the two core windows is set to a large value, indicating a break. This encoding method can accurately identify the conserved sequence patterns and position-dependent features of the UTR-CDS connection region, effectively improving the model's ability to extract features from key regulatory elements of mRNA.

[0115] Furthermore, considering RNA sequence characteristics and folding properties, this invention employs a multimodal RNA sequence encoding method. By integrating base composition information and secondary structure features, it enhances the model's ability to analyze RNA sequences: It uses 1D one-hot encoding (A[1,0,0,0],U[0,1,0,0],C[0,0,1,0],G[0,0,0,1]). In some specific application scenarios, other tokens can also be used to describe the breakpoints, start points, end points, padding points, and masking points of the mRNA sequence, corresponding to N, respectively. <s>，< / s> , <pad> ， <mask>The system represents base composition information; simultaneously, 2D encoding is introduced to integrate base complementary pairing rules (Watson-Crick pairing rules), generating a two-dimensional base pairing matrix (AU=2, GC=3, GU=0.8, representing the strength of different base pairings), enabling the model to automatically identify potential secondary structure features. The 2D encoding is directly weighted with the attention matrix of the attention layer, and finally, the weighted 2D information is fused into the 1D encoding through matrix multiplication.

[0116] The technical solution of this invention employs a two-stage training strategy to optimize mRNA sequence modeling. The first stage involves unsupervised pre-training based on NCBI human transcription sample data. An innovative position-aware masking prediction task enables the model to grasp the basic rules of mRNA. This task masks 15% of the sequence region, using a Gaussian distribution centered on the start / stop codons to make the model focus more on the UTR-CDS connection region. The start / stop codon positions are derived from NCBI's annotation files, excluding sequences with a CDS region less than 100 or ambiguous annotations (e.g., CDS region lengths not multiples of 3). Simultaneously, a two-dimensional base pairing matrix is ​​introduced to enhance the attention mechanism, integrating secondary structural features of the sequence through gated fusion and residual connections. This pairing-biased attention mechanism allows for the calculation of correlations between any two bases, thereby deriving the long-range synergistic effects of two UTR-CDS connection regions. This stage, guided by Gaussian weighted masking and structural features, improves the model's ability to capture key regulatory elements of mRNA, providing strong support for downstream tasks such as translation efficiency prediction in the second stage.

[0117] The second stage involves supervised fine-tuning, optimizing model performance using task-specific labeled data based on pre-trained model parameters. This ensures the model can recognize general sequence patterns while adapting to specific application needs. This technique uses multiple experimental datasets to fine-tune a large language model (sequence-metric, supervised learning). The pre-trained large language model maps sequences to a high-dimensional space, adds a convolutional network as a decoding layer, and uses a regression prediction head to output predicted scores, obtaining a score for the given data scenario. Therefore, different datasets train different independent models to represent experimental metrics from different perspectives. The datasets used are from publicly available literature or websites, including ribosome load data measured under different cell states (normal / stressed) (polyribosome analysis experiments), translation efficiency data (paired Ribo-seq and RNA-seq experiments), protein expression data (mass spectrometry quantitative proteomics experiments, fluorescence reporter system experiments), and mRNA stability data (half-life determination experiments). During model evaluation, Spearman scores are used to assess the correlation between the model's predicted scores on the test set and the true scores on the dataset. The mRNA dataset containing synthetic sequences is divided into training, validation, and test sets in an 8:1:1 ratio. The mRNA dataset containing natural sequences is divided into training, validation (chromosome 10), and test (chromosome 15) sets based on their chromosome numbers.

[0118] In some specific application scenarios, the technical solutions based on the embodiments of the present invention can be implemented as follows:

[0119] Implementation method one: Focusing on multiple regions of mRNA helps predict translation performance.

[0120] This invention collected six RNA large language models (RNAMSM, RNALM, UTRLM, RNABERT, RNAFM, UTRBERT). Through fine-tuning (supervised learning) on ​​eight translation-related benchmark tasks, it was found that the length of the input sequence to the model is positively correlated with the model's performance; the longer the input sequence, the better the model's performance (the average performance of the model on different benchmark tasks is referenced). Figure 4 The model's performance on different benchmark tasks is referenced. Figure 5 .

[0121] Specifically, Figure 4 In this study, the average results (SPR, Spearman coefficient) of six RNA large language models were compared across eight benchmark tasks with different training and testing lengths. The benchmark tasks were: translation initiation efficiency, translation efficiency, protein expression to mRNA expression ratio, ribosome recruitment efficiency, ribosome load, and mRNA degradation efficiency measured under normal / stress conditions. Figure 5 In this study, the performance of mRNA-UCS was compared with that of existing large language models. (Statistical tables labeled a and b: translation initiation efficiency measured by cells under normal / stress conditions; statistical tables labeled c and d: translation efficiency measured by cells under normal / stress conditions; statistical tables labeled e: ratio of protein expression to mRNA expression; statistical tables labeled f: ribosome recruitment efficiency; statistical tables labeled g: ribosome load; statistical tables labeled h: mRNA degradation efficiency).

[0122] Implementation method two: Benchmark testing based on public datasets and public LM algorithms:

[0123] This invention uses mRNA-UCS to compare with different large language model scoring algorithms on public datasets, and outperforms state-of-the-art methods (such as...) on eight downstream tasks. Figure 5 (As shown).

[0124] Implementation Method 3: Scoring and Verification of mRNA Sequences

[0125] This invention aims to verify the effectiveness of the mRNA-UCS method in assessing mRNA sequence characteristics and evaluate its correlation with protein expression levels. A set of known mRNA sequence samples were selected, from which the corresponding mRNA sequences and their corresponding expressed proteins were extracted from PC-9 cells (cells under stress), and their expression levels were measured. Using the mRNA-UCS method, each mRNA sequence was scored from multiple angles, such as translation initiation efficiency, translation efficiency, and degradation efficiency, specifically: TIE (stress), TE (stress), and Decay_Rate. Correlation analysis between the mRNA-UCS scores and protein expression levels revealed a high Spearman correlation coefficient. Furthermore, the direction of the correlation conformed to biological expectations, i.e., translation efficiency was positively correlated with protein expression levels, while degradation efficiency was negatively correlated with protein expression levels. The results are as follows: Figure 6 As shown.

[0126] Specifically, Figure 6 The results show the scores of mRNA sequences using the mRNA-UCS method across multiple dimensions, including TIE, TE, and Decay Rate. The scores are highly correlated with data obtained from protein expression experiments (extracting and measuring the amount of expressed protein from PC-9 cells), and this correlation aligns with biological expectations.

[0127] In summary, this invention, by considering the synergistic effects of multiple regions of the mRNA sequence, simultaneously incorporates distant sequence fragments into the model and combines them with scorers constructed for different downstream tasks. This enables multi-faceted evaluation (ribosome load, translation efficiency, protein expression levels, and mRNA stability measured under different cellular states (normal / stressed)), providing multiple reference indicators for screening high-quality mRNA sequences. Specifically, the technical breakthrough achieved by this invention lies in:

[0128] (1) Pioneering dual-center position encoding mechanism: By establishing a dynamic encoding system with start codon (AUG) and stop codon (UAA / UAG / UGA) as dual anchor points, the precise localization of mRNA sequences in key functional regions of UTR-CDS was achieved for the first time. This technology breaks through the limitations of traditional linear sequence modeling and deeply integrates biological prior knowledge such as codon boundary effects into the deep learning architecture.

[0129] (2) Innovative region-focused pre-training strategy: A pre-training method specifically for the UTR-CDS connection region was developed. Through a unique sequence truncation window design, two linearly distant connection regions (5'UTR-CDS and CDS-3'UTR) can be simultaneously presented in the same input window. This design enables the model to capture the synergistic effect (attention mechanism) of distant regulatory regions, laying the foundation for full-length mRNA function prediction.

[0130] (3) Pioneering Connector Region-Function Prediction System: A sequence-experimental index prediction model (mRNA-UCS system) based on the characteristics of dual connector regions was constructed. This system innovatively achieves: 1. Simultaneous analysis of the synergistic effect of two UTR-CDS connector regions, providing quantitative scores for different UTR and CDS combinations; 2. Outputting priority validation sequence ordering to accelerate the development process of mRNA vaccines.

[0131] Compared with the prior art, the embodiments of the present invention have at least the following beneficial effects:

[0132] A significant advantage of the technical solution of this invention lies in its comprehensive consideration of the mRNA translation mechanism. Given that the translation efficiency of mRNA is affected by the synergistic effect of multiple regions, and that the length of different regions of mRNA varies considerably, existing technologies often focus only on local regions, such as truncated 5' uncoding regions (5'UTR) or leader sequences with added coding sequences (CDS), to predict translation efficiency.

[0133] The technical solution of this invention incorporates longer sequence information in its design, enabling a more comprehensive and accurate evaluation of mRNA sequence effectiveness. Compared to methods that rely solely on local sequence information (such as those based only on CDS or 5'UTR), this method considers the synergistic effects of multiple regions of the mRNA sequence and constructs scorers in conjunction with different downstream tasks, achieving multi-angle evaluation (ribosome load, translation efficiency, protein expression levels, and mRNA stability measured under different cellular states (normal / stressed)). This provides multiple reference indicators for screening high-quality mRNA sequences.

[0134] Furthermore, the effectiveness of the technical solution in this invention is also reflected in its potential contributions to fields such as gene therapy and protein drug development. By pre-screening mRNA sequences with potentially high expression levels, this method helps reduce the blind spots and costs of experiments, thereby accelerating the research and development process and improving efficiency. Utilizing the advanced learning and generalization capabilities of large-scale language models and fine-tuning them with various experimental data, the scorer of this technology exhibits higher adaptability and flexibility, better meeting the needs of mRNA sequence evaluation in different scenarios.

[0135] In summary, the technical solution of this invention provides a scoring method for comprehensively evaluating mRNA sequence characteristics, and experiments have verified its high correlation with protein expression levels. This method not only provides a new technical means for mRNA sequence optimization and protein expression prediction, but also has the potential to promote technological progress and application development in related fields.

[0136] like Figure 7 As shown, this embodiment of the invention also provides an mRNA sequence scoring device 900, which can implement the above-described method. This device may include:

[0137] The sequence acquisition module 910 is used to acquire the mRNA sequence to be scored. The mRNA sequence includes a non-coding region and a coding region. The non-coding region includes a 5' non-coding region located upstream of the start end of the coding region and a 3' non-coding region located downstream of the end end of the coding region.

[0138] The sequence extraction module 920 is used to extract key base sequences from mRNA sequences. The key base sequences include start codon context and stop codon context. The start codon context represents the codon sequence at the start codon with a preset window length, and the stop codon context represents the codon sequence at the stop codon with a preset window length.

[0139] The scoring module 930 is used to input key base sequences into the target scoring model and process them to obtain the score of the mRNA sequence on the target indicator.

[0140] The target scoring model includes a multimodal RNA sequence encoding module and a large language model. The target scoring model is obtained by unsupervised pre-training based on transcription sample data and supervised fine-tuning based on the score label dataset corresponding to the target index.

[0141] In some embodiments, the apparatus may further include a model training module for performing the following operations:

[0142] The initial model is configured based on the multimodal RNA sequence coding module and the large language model;

[0143] The multimodal RNA sequence coding module includes a one-dimensional coding unit and a two-dimensional coding unit. The one-dimensional coding unit is used to process and obtain base composition information, and the two-dimensional coding unit is used to process and obtain secondary structure features.

[0144] Based on transcription sample data, the initial model was unsupervised pre-trained using a position-aware mask prediction task, so that the initial model could learn the sequence patterns of mRNA sequences and obtain a pre-trained model.

[0145] The target indicator's corresponding score label dataset includes multiple mRNA sequence samples and the score label of each mRNA sequence sample on the target indicator.

[0146] Based on mRNA sequence samples and score labels, the pre-trained model is fine-tuned in a supervised manner to obtain the target scoring model.

[0147] It is understood that the content of the above method embodiments is applicable to the present device embodiments. The specific functions implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0148] This invention also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the method described above. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0149] It is understood that the content of the above method embodiments is applicable to this device embodiment. The specific functions implemented by this device embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0150] like Figure 8 As shown, Figure 8 The hardware structure of an electronic device 1000 according to another embodiment is illustrated. The electronic device 1000 includes:

[0151] The processor 1001 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (aSIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present invention.

[0152] The memory 1002 can be implemented as a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RaM). The memory 1002 can store the operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1002 and is called and executed by the processor 1001.

[0153] Input / output interface 1003 is used to implement information input and output;

[0154] The communication interface 1004 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0155] Bus 1005 transmits information between various components of the device (e.g., processor 1001, memory 1002, input / output interface 1003, and communication interface 1004);

[0156] The processor 1001, memory 1002, input / output interface 1003 and communication interface 1004 are connected to each other within the device via bus 1005.

[0157] The electronic device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0158] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method.

[0159] It is understood that the content of the above method embodiments is applicable to this storage medium embodiment. The specific functions implemented in this storage medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0160] This invention also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0161] It is understood that the content of the above method embodiments is applicable to the embodiments of this program product. The specific functions implemented by the embodiments of this program product are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0162] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0163] The mRNA sequence scoring method, apparatus, electronic device, storage medium, and program product provided in this invention embodiment acquire an mRNA sequence to be scored. The mRNA sequence includes a non-coding region and a coding region. The non-coding region includes a 5' non-coding region upstream of the start end of the coding region and a 3' non-coding region downstream of the stop end of the coding region. Key base sequences are extracted from the mRNA sequence. The key base sequences include a start codon context and a stop codon context. The start codon context represents a codon sequence with a preset window length at the start codon, and the stop codon context represents a codon sequence with a preset window length at the stop codon. The key base sequences are input into a target scoring model to obtain the score of the mRNA sequence on a target indicator. The target scoring model includes a multimodal RNA sequence encoding module and a large language model. The target scoring model is obtained based on unsupervised pre-training of transcription sample data and supervised fine-tuning based on a score label dataset corresponding to the target indicator. This invention replaces conventional experimental procedures with a model, enabling instant batch scoring of massive mRNA sequences. This solves the problems of low throughput and time-consuming, labor-intensive individual testing inherent in traditional methods, meeting the high-efficiency requirements of practical applications. Furthermore, the model-based scoring method ensures all sequences are processed using the same standardized model, resulting in highly consistent and comparable outputs. This overcomes the challenges of incomparable results and inconsistent evaluation standards caused by significant differences in experimental conditions. Simultaneously, this invention employs "key base sequences" integrating key regions (start / stop codon context) of the uncoding region (UTR) and coding region (CDS) as input, and utilizes a large language model trained on massive datasets. This allows the model to comprehensively consider sequence context information, secondary structure, and other complex factors, overcoming the shortcomings of early bioinformatics methods, such as "simple models" and "insufficient consideration of sequence context information." Specifically, by using labeled data of specific indicators in the "supervised fine-tuning" stage, the same model framework can be adapted to the accurate prediction of different indicators, providing a feasible technical path for "integrating multiple experimental indicators" and overcoming the limitations of previous methods in effectively integrating multiple indicators.

[0164] The embodiments described in this invention are for the purpose of more clearly illustrating the technical solutions of the embodiments of this invention, and do not constitute a limitation on the technical solutions provided by the embodiments of this invention. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this invention are also applicable to similar technical problems.

[0165] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present invention, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0166] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0167] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0168] The terms "first," "second," "third," "fourth," etc. (if present) in the specification and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0169] It should be understood that in this invention, "at least one (item)" refers to one or more, and "more than" refers to two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0170] In the several embodiments provided by this invention, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0171] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0172] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0173] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0174] The preferred embodiments of the present invention have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present invention. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and spirit of the present invention should be within the scope of the claims of the present invention.< / mask> < / pad> < / pad> < / pad>

Claims

1. A method for scoring mRNA sequences, characterized in that, The method includes the following steps: Obtain the mRNA sequence to be scored, the mRNA sequence including a non-coding region and a coding region, the non-coding region including a 5' non-coding region located upstream of the start end of the coding region and a 3' non-coding region located downstream of the end end of the coding region; Key base sequences are extracted from the mRNA sequence, including a start codon context and a stop codon context. The start codon context represents a codon sequence of a predetermined window length at the start codon, and the stop codon context represents a codon sequence of the predetermined window length at the stop codon. The key base sequence is input into the target scoring model, and the mRNA sequence is processed to obtain the score of the target index. The target scoring model includes a multimodal RNA sequence encoding module and a large language model. The target scoring model is obtained based on unsupervised pre-training of transcription sample data and supervised fine-tuning based on the score label dataset corresponding to the target index. The step of extracting key base sequences from the mRNA sequence includes the following steps: Using the first nucleotide position of the start codon and the last nucleotide position of the stop codon as dual-center reference points, the mRNA sequence is normalized and aligned to obtain an aligned sequence.

2. The method according to claim 1, characterized in that, The extraction of key base sequences from the mRNA sequence further includes the following steps: Based on the alignment sequence, a sequence of a preset length upstream of the center and a sequence of the same preset length downstream of the center are extracted for the center corresponding to the start codon to form the start codon context; wherein, the preset length is half of the preset window length; Based on the alignment sequence, the stop codon context is formed by extracting a sequence of a preset length upstream of the center and a sequence of a preset length downstream of the center corresponding to the stop codon. The start codon context and the stop codon context are marked and separated to form the key base sequence.

3. The method according to claim 1 or 2, characterized in that, The multimodal RNA sequence coding module includes a one-dimensional coding unit and a two-dimensional coding unit. The step of inputting the key base sequence into the target scoring model and processing it to obtain the score of the mRNA sequence on the target indicator includes the following steps: The key base sequence is input into the multimodal RNA sequence encoding module, and the encoding information is obtained by integrating the base composition information and secondary structure features; The base composition information is obtained by processing the one-dimensional coding unit, and the secondary structure features are obtained by processing the two-dimensional coding unit. Based on the encoded information, the large language model is used to process and output the score of the mRNA sequence on the target indicator.

4. The method according to claim 3, characterized in that, The process of obtaining encoded information by integrating base composition information and secondary structure features includes the following steps: The key base sequence is one-hot encoded using the one-dimensional coding unit to obtain the base composition information; Based on the key base sequence, the two-dimensional coding unit generates a two-dimensional base pairing matrix as the secondary structure feature based on the base complementary pairing rule. The encoded information is obtained by integrating the base composition information and the secondary structure features.

5. The method according to claim 4, characterized in that, The process of integrating the base composition information and the secondary structure features to obtain the encoded information includes the following steps: The secondary structure features are weighted using an attention matrix; The weighted result is fused into the base composition information by matrix multiplication to obtain the encoded information.

6. The method according to claim 1, characterized in that, The method further includes the following steps: The initial model is configured based on the multimodal RNA sequence encoding module and the large language model; The multimodal RNA sequence coding module includes a one-dimensional coding unit and a two-dimensional coding unit. The one-dimensional coding unit is used to process and obtain base composition information, and the two-dimensional coding unit is used to process and obtain secondary structure features. Based on the transcription sample data, the initial model is unsupervised pre-trained using a position-aware mask prediction task, so that the initial model learns the sequence patterns of the mRNA sequence and obtains a pre-trained model. The target indicator's corresponding score label dataset includes multiple mRNA sequence samples and a score label for each mRNA sequence sample on the target indicator. Based on the mRNA sequence sample and the score label, the pre-trained model is fine-tuned in a supervised manner to obtain the target scoring model.

7. The method according to claim 6, characterized in that, The unsupervised pre-training of the initial model based on the transcribed sample data through a position-aware mask prediction task includes the following steps: The mask probability is constructed based on two Gaussian distributions centered on the start codon and the stop codon; Based on the masking probability, a portion of the transcription sample sequence in the transcription sample data is masked to obtain Gaussian weighted masked data. The two-dimensional coding unit integrates the secondary structure features of the sequence through attention mechanism, gated fusion and residual connection; The initial model is pre-trained unsupervised based on the Gaussian weighted mask data and the structural features of the two-dimensional coding unit.

8. The method according to claim 6, characterized in that, The supervised fine-tuning of the pre-trained model based on the mRNA sequence sample and the score label includes the following steps: The key base sequence extracted from the mRNA sequence sample is used as input data into the pre-trained model so that the pre-trained model can process and output the prediction score of the corresponding mRNA sequence sample. A loss function is constructed based on the predicted score and the score label corresponding to the mRNA sequence sample, and the parameters of the pre-trained model are fine-tuned using the loss function.

9. An mRNA sequence scoring device, characterized in that, The device includes: The sequence acquisition module is used to acquire the mRNA sequence to be scored. The mRNA sequence includes a non-coding region and a coding region. The non-coding region includes a 5' non-coding region located upstream of the start end of the coding region and a 3' non-coding region located downstream of the end end of the coding region. A sequence extraction module is used to extract key base sequences from the mRNA sequence. The key base sequences include a start codon context and a stop codon context. The start codon context represents a codon sequence of a preset window length at the start codon, and the stop codon context represents a codon sequence of the preset window length at the stop codon. The scoring module is used to input the key base sequence into the target scoring model and process it to obtain the score of the mRNA sequence on the target indicator. The target scoring model includes a multimodal RNA sequence encoding module and a large language model. The target scoring model is obtained based on unsupervised pre-training of transcription sample data and supervised fine-tuning based on the score label dataset corresponding to the target index. The step of extracting key base sequences from the mRNA sequence includes the following steps: Using the first nucleotide position of the start codon and the last nucleotide position of the stop codon as dual-center reference points, the mRNA sequence is normalized and aligned to obtain an aligned sequence.

10. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method according to any one of claims 1 to 8.

11. A computer storage medium storing a processor-executable program, which, when executed by a processor, is used to implement the method as described in any one of claims 1 to 8.

12. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 8.

Images