Codon optimization based on token classification

Abstract

The present disclosure relates to a codon optimization technique based on token classification. In a model training phase, a collected protein sequence is tokenized by using a single amino acid as a token, and a nucleic acid sequence encoding a protein and corresponding to the protein sequence is tokenized by using a single codon as a token; a plurality of different synonymous codons in the tokenized nucleic acid sequence encoding the protein and corresponding to a single amino acid in the tokenized protein sequence are sorted in a descending order of frequency of occurrence in an expression system of the protein, and a classification label is set, thereby training a natural language model. In a prediction phase of the nucleic acid sequence encoding the protein, a protein sequence to be subjected to codon optimization is tokenized. The tokenized protein sequence is input into the trained natural language model to predict a classification label of a synonymous codon corresponding to a single amino acid in the tokenized protein sequence. An optimized nucleic acid sequence is obtained on the basis of a mapping relationship between the synonymous codon and the classification label thereof.

Description

Codon optimization based on token classification

[0001] Cross-reference to Related Applications

[0002] This application claims priority to Chinese Patent Application No. 202410613348.1, filed May 16, 2024, the entire contents of which are incorporated herein by reference. TECHNICAL FIELD

[0003] The present disclosure relates to protein expression in biotechnology, and more specifically to methods for optimizing nucleic acid sequences encoding proteins. BACKGROUND

[0004] 1. Tokenization in natural language

[0005] Tokenization in natural language processing (NLP) refers to the process of dividing text (such as sentences) into smaller units (Tokens), typically words, phrases, or symbols, etc. (see Mielke, S. J., Alyafeai, Z., Salesky, E., Raffel, C., Dey, M., Gallé, M., Raja, A., Si, C., Lee, W. Y. and Sagot, B. (2021) Between words and characters: a brief history of open-vocabulary modeling and tokenization in nlp. ArXiv Prepr. ArXiv 211210508). Tokenization is the first step in natural language processing and is the foundation of text analysis; correct tokenization is crucial for subsequent tasks such as part-of-speech tagging, syntactic analysis, entity recognition, etc. In the field of biomedicine, the units (Tokens) of nucleic acid and protein sequences are often bases, amino acids, codons, and N-mer / N-gram (such as 2-6-mer) (see https: / / en.wikipedia.org / wiki / N-gram).

[0006] 2. Token Classification

[0007] Token Classification is a task in natural language processing that involves assigning a specific class or label to each unit (usually a word or punctuation mark) in a text. This process is a way of understanding and interpreting text at a finer level, often used to identify and classify different elements in the text. In token classification, each unit is given a label, which typically represents its semantic role, grammatical category, or other relevant properties. For example, in the task of Part-of-Speech tagging (POS tagging), units are classified as nouns, verbs, adjectives, etc. (see Chiche, A. and Yitagesu, B. (2022) Part of speech tagging: a systematic review of deep learning and machine learning approaches. J. Big Data, 9, 1-25). In the task of Named Entity Recognition (NER), units are labeled as entity types such as person names, location names, organization names, etc. (see Li, J., Sun, A., Han, J. and Li, C. (2020) A survey on deep learning for named entity recognition. IEEE Trans. Knowl. Data Eng., 34, 50-70). In the context of NER, the system attempts to find named entities in the text and classify them into predefined categories. For example, in the sentence "Apple was founded by Steve Jobs in California.", "Apple" (Apple Inc.) can be labeled as an organization (ORG), "Steve Jobs" as a person name (PERSON), and "California" as a location (LOC). Token classification is not limited to these tasks, it can also be used in sentiment analysis, next-word prediction in language models, specific term recognition in biomedical text analysis, and many other NLP applications.

[0008] The present disclosure attempts to apply natural language processing methods to the codon optimization of proteins in the biological field. It is desirable to propose a technique that can simplify the problem of codon optimization. SUMMARY

[0009] Considering that the number of synonymous codons for each amino acid is 1, 2, 3, 4, and 6 respectively, and the maximum number is only 6, which is a special nature, it can be tried to simplify the codon optimization problem into the Token Classification problem in natural language processing, that is, first tokenizing the protein sequence into single amino acid (1-mer) units (Tokens), and then classifying each amino acid unit. Similar to the codon optimization problem, the same amino acid can be encoded by different synonymous codons, and the same amino acid at different positions is allowed to be labeled with different classification tags due to the difference of the corresponding synonymous codons.

[0010] The present disclosure provides a codon optimization method. The method comprises: tokenizing the collected single amino acid sequences of the protein as the tokenization unit, and tokenizing the nucleotide sequence encoding the protein as the tokenization unit; sorting the synonymous codons of each amino acid in order of frequency from high to low in the expression system of the protein, and setting the corresponding classification label respectively; annotating the tokenized protein sequence with the classification label of the corresponding tokenization unit in the nucleic acid sequence encoding the protein in turn according to the tokenization unit, and taking the annotated protein sequence as a training set; training a natural language model through the training set; tokenizing the protein sequence to be codon-optimized; inputting the tokenized protein sequence to be codon-optimized into the trained natural language model to predict the classification label corresponding to each tokenization unit of the tokenized protein sequence to be codon-optimized; obtaining the optimized nucleic acid sequence of the protein sequence to be codon-optimized according to the mapping relationship between the different tokenization units of the nucleic acid sequence encoding the protein and the classification labels thereof. The nucleic acid sequence encoding the protein is optimized by the codon optimization method to improve the expression of the protein in the host cell.

[0011] According to a first aspect of the present disclosure, a training method of a natural language model for codon optimization is provided. The method comprises: tokenizing the collected protein sequences and their nucleic acid sequences encoding the protein respectively; setting classification labels for one or more different tokenization units in the tokenized nucleic acid sequence encoding the protein corresponding to each same tokenization unit in the tokenized protein sequence; annotating the tokenized protein sequence with the classification label of the corresponding tokenization unit in the nucleic acid sequence encoding the protein in turn according to the tokenization unit, and taking the annotated protein sequence as a training set; training a natural language model through the training set to obtain a trained natural language model.

[0012] Preferably, the collected protein sequences are tokenized as single amino acids as the tokenization unit, and the nucleic acid sequences encoding the protein are tokenized as single codons as the tokenization unit. For example, a single codon is a triple codon.

[0013] Preferably, the synonymous codons of each amino acid are sorted in order of frequency of occurrence in the protein expression system, and the corresponding classification labels are set respectively.

[0014] Preferably, the token units of the single amino acid are represented using word indices in the dictionary of the tokenizer. Alternatively, the token units of the single amino acid are represented using pre-trained protein models to extract amino acid features as word embedding vectors.

[0015] Preferably, the training of the natural language model by the training set can further include adding absolute or relative position encoding of the single amino acid in the protein sequence in the training set.

[0016] Preferably, a model suitable for long text prediction is selected as the natural language model.

[0017] Preferably, codon word embedding information extracted using nucleic acid language models is added in the training set.

[0018] The natural language model can be selected from RoFormer, Bert, RoBERTa, Longformer, BigBird, Phi, DistilBERT or XLNet; preferably BigBird.

[0019] According to a second aspect of the present disclosure, a natural language model for codon optimization is provided, which is trained by the method of the first aspect of the present disclosure.

[0020] According to a third aspect of the present disclosure, a codon optimization method is provided. The method comprises: tokenizing a protein sequence to be codon optimized; inputting the tokenized protein sequence to be codon optimized into the natural language model trained by the method of the first aspect of the present disclosure, and predicting the classification label corresponding to each token unit of the tokenized protein sequence to be codon optimized; obtaining an optimized nucleic acid sequence of the protein sequence to be codon optimized according to the mapping relationship between different token units of the nucleic acid sequence encoding the protein and their classification labels.

[0021] It is noted that the “protein sequence to be codon-optimized” mentioned in the present disclosure can be natural or artificially designed. In addition, for the “nucleic acid sequence encoding a protein” mentioned in the present disclosure, the nucleic acid sequence can also be natural, artificially synthesized, or codon-optimized. That is, the codons of the protein sequence, i.e., the codons of the nucleic acid sequence encoding the protein, can be optimized using the method of the present disclosure. In other words, the “protein sequence to be codon-optimized” can be a protein sequence that has not been subjected to nucleic acid encoding (from an amino acid sequence to a nucleic acid sequence), or a protein sequence that has been subjected to nucleic acid encoding. That is, the nucleic acid sequence encoding the protein, more specifically, the codons in the nucleic acid sequence encoding the protein corresponding to the amino acids in the protein sequence, can be optimized using the method of the present disclosure.

[0022] Preferably, the protein sequence to be codon-optimized is segmented with single amino acids as the segmentation unit. The segmentation unit of the nucleic acid sequence encoding the protein is a single codon. For example, a single codon is a triple codon.

[0023] Preferably, the classification tags are set by ordering the synonymous codons of each amino acid in order of frequency of occurrence in a protein expression system from high to low.

[0024] Preferably, the codon word embedding information extracted using the nucleic acid language model is input to the trained natural language model.

[0025] The natural language model is selected from RoFormer, Bert, RoBERTa, Longformer, BigBird, Phi, DistilBERT, or XLNet; preferably BigBird.

[0026] Preferably, the method according to the third aspect of the present disclosure can further comprise: optimizing the obtained nucleic acid sequence encoding the protein by performing synonymous codon substitution on the harmful site sequence, the variable splicing site sequence, and / or the gene synthesis difficulty region sequence. Preferably, each segmentation unit of the nucleic acid sequence encoding the protein corresponding to each segmentation unit of the protein sequence to be codon-optimized obtained according to the mapping relationship is sequentially combined to obtain an optimized nucleic acid sequence. More specifically, the codons corresponding to each amino acid of the protein sequence to be codon-optimized are sequentially combined to obtain an optimized nucleic acid sequence.

[0027] According to a fourth aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium for storing a computer program for optimizing a nucleic acid sequence encoding a protein. The computer program comprises instructions. The instructions, when executed by a processor of an electronic device, cause the electronic device to implement the training method of a natural language model for codon optimization according to the first aspect of the present disclosure or the codon optimization method according to the third aspect of the present disclosure.

[0028] According to a fifth aspect of the present disclosure, there is provided a system for optimizing a nucleic acid sequence encoding a protein. The system comprises: a processor; a memory; and a computer program. The computer program is stored in the memory and configured to be executed by the processor. The computer program comprises instructions for implementing the training method of a natural language model for codon optimization according to the first aspect of the present disclosure or the codon optimization method according to the third aspect of the present disclosure.

[0029] According to a sixth aspect of the present disclosure, there is provided a computer program product comprising a computer program for optimizing a nucleic acid sequence encoding a protein. The computer program, when executed by a processor, implements the training method of a natural language model for codon optimization according to the first aspect of the present disclosure or the codon optimization method according to the third aspect of the present disclosure.

[0030] According to a seventh aspect of the present disclosure, there is provided an isolated nucleic acid molecule comprising the optimized nucleic acid sequence obtained by the codon optimization method according to the third aspect of the present disclosure.

[0031] According to an eighth aspect of the present disclosure, there is provided a vector comprising the isolated nucleic acid molecule according to the seventh aspect of the present disclosure.

[0032] According to a ninth aspect of the present disclosure, there is provided a recombinant host cell comprising the isolated nucleic acid molecule according to the seventh aspect of the present disclosure or the vector according to the eighth aspect of the present disclosure.

[0033] According to a tenth aspect of the present disclosure, there is provided a method of expressing a protein in a host cell. The method comprises: (a) obtaining an optimized nucleic acid sequence for expressing the protein in the host cell using the codon optimization method according to the third aspect of the present disclosure; (b) synthesizing a nucleic acid molecule comprising the optimized nucleic acid sequence; (c) introducing the nucleic acid molecule into the host cell to obtain a recombinant host cell; and (d) culturing the recombinant host cell under conditions permitting expression of the protein from the optimized nucleic acid sequence.

[0034] The methods according to the present disclosure are suitable for codon optimization of proteins expressed in E. coli, mammalian, and the like systems.

[0035] The codon optimization technology involved in the present disclosure has the following beneficial technical effects: simplifying the solution of the codon optimization problem based on neural machine translation or question-answering tasks into a unit classification problem based on the unit classification problem, ensuring that the optimization effect (i.e., the protein expression amount in the host) is better than that of the existing codon optimization methods such as GenSmart, while the calculation speed is faster, the computing resources are reduced, and the throughput is expanded. BRIEF DESCRIPTION OF DRAWINGS

[0036] The present disclosure will be more fully understood from the following detailed description, taken in conjunction with the accompanying drawings, in which like reference numerals refer to like elements, in which:

[0037] FIG. 1A is a flowchart of a training method of a natural language model for codon optimization according to an embodiment of the present disclosure.

[0038] FIG. 1B is a flowchart of a codon optimization method based on unit classification of word segmentation according to an embodiment of the present disclosure.

[0039] FIG. 2 is a schematic diagram of the association between model training and model application in the codon optimization method based on unit classification of word segmentation according to an embodiment of the present disclosure.

[0040] FIG. 3 is a schematic diagram of unit classification of word segmentation for an example protein sequence.

[0041] FIG. 4 is an example SDS-PAGE gel electrophoresis diagram. DETAILED DESCRIPTION

[0042] Unless otherwise defined, the technical and scientific terms used in the present disclosure have the meanings commonly understood by one of ordinary skill in the art to which the present disclosure belongs.

[0043] The technical solutions of the present disclosure will be further described below by examples, and in conjunction with the drawings. Unless otherwise stated, the methods and materials described in the following examples are conventional products that can be purchased on the market. Those skilled in the art to which the present disclosure belongs will understand that the methods and materials described below are only exemplary and should not be considered as limiting the scope of the present disclosure.

[0044] Codon optimization is a process of changing codons in a nucleic acid sequence encoding a protein to other synonymous codons without changing the sequence of the encoded protein, based on the fact that there are two or more codons (i.e., synonymous codons) for the same amino acid. A codon refers to a group of every three adjacent nucleotides in a messenger RNA molecule that, during protein synthesis, represents a certain amino acid, which directs the cell to synthesize a specific amino acid. Different codons can encode the same amino acid, a phenomenon known as codon redundancy. Codon optimization takes advantage of this redundancy to change the codon composition of a gene to improve protein expression, stability, or other biological properties without changing the function of the protein.

[0045] Deep learning models perform exceptionally well on sequential data, so they have potential applications in various applications of codon optimization (see Goulet, D. R., Yan, Y., Agrawal, P., Waight, A. B., Mak, A. N. and Zhu, Y. (2022) Codon Optimization Using a Recurrent Neural Network. J. Comput. Biol.; Jain, R., Jain, A., Mauro, E., LeShane, K. and Densmore, D. (2021) ICOR: Improving codon optimization with recurrent neural networks. bioRxiv; Joiret, M., Leclercq, M., Lambrechts, G., Rapino, F., Close, P., Louppe, G. and Geris, L. (2023) Cracking the genetic code with neural networks. Front. Artif. Intell., 6, 59; Fu, H., Liang, Y., Zhong, X., Pan, Z., Huang, L., Zhang, H., Xu, Y., Zhou, W. and Liu, Z. (2020) Codon optimization with deep learning to enhance protein expression. Sci. Rep., 10, 1-9; Ding, Z., Guan, F., Xu, G., Wang, Y., Yan, Y., Zhang, W., Wu, N., Yao, B., Huang, H. and Tuller, T. (2022) MPEPE, a predictive approach to improve protein expression in E. coli based on deep learning. Comput. Struct. Biotechnol. J., 20, 1142-1153). By training models to recognize and exploit complex patterns in genetic sequences, deep learning can help more accurately predict the impact of codon changes on biological properties, driving the development of personalized genetic design and synthetic biology in biotechnology and medicine.

[0046] The codon optimization can be directly translated into neural machine translation or question-answering tasks, and then the common models for solving these two tasks are used for translation or answering of protein sequence to nucleic acid sequence. Common models include recurrent neural network (RNN), long short-term memory network (LSTM) and bidirectional long short-term memory network (BiLSTM), bidirectional encoder representation technology based on transformer (Bidirectional Encoder Representations from Transformers, Bert (see Devlin, J., Chang, M.-W., Lee, K. and Toutanova, K. (2018) Bert: Pre-training of deep bidirectional transformers for language understanding. ArXiv Prepr. ArXiv181004805)) and its variants (such as RoBerta (see Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L. and Stoyanov, V. (2019) Roberta: A robustly optimized bert pretraining approach. ArXiv Prepr. ArXiv190711692), ConvBert (see Jiang, Z.-H., Yu, W., Zhou, D., Chen, Y., Feng, J. and Yan, S. (2020) Convbert: Improving bert with span-based dynamic convolution. Adv. Neural Inf. Process. Syst., 33, 12837-12848), DeBerta (see He, P., Liu, X., Gao, J. and Chen, W. (2020) Deberta: Decoding-enhanced bert with disentangled attention. ArXiv Prepr. ArXiv200603654), DistilBert (see Sanh, V., Debut, L., Chaumond, J. and Wolf, T.text-to-text-transfer-transformer, T5 (see Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W. and Liu, P. J. (2020) Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res., 21, 5485-5551)) and variants thereof (such as ByT5 (see Xue, L., Barua, A., Constant, N., Al-Rfou, R., Narang, S., Kale, M., Roberts, A. and Raffel, C. (2022) ByT5: Towards a token-free future with pre-trained byte-to-byte models. Trans. Assoc. Comput. Linguist., 10, 291-306), LongT5 (see Guo, M., Ainslie, J., Uthus, D., Ontanon, S., Ni, J., Sung, Y.-H. and Yang, Y. (2021) LongT5: Efficient text-to-text transformer for long sequences. ArXiv Prepr. ArXiv 211207916) and the like), encoder-decoder type models (Encoder-Decoder, including Sequence-to-sequence models) and the like.

[0047] General method

[0048] In general, the present disclosure provides a codon optimization method based on classification of word units. FIG. 1A shows a flowchart of a training method of a natural language model for codon optimization according to an embodiment of the present disclosure. As shown in FIG. 1A, the training method 100A of a natural language model for codon optimization according to an embodiment of the present disclosure starts at step S110.

[0049] In step S110, the collected protein sequences and the nucleic acid sequences encoding the proteins are segmented respectively. Specifically, in a preferred embodiment, the collected protein sequences are segmented with single amino acid as the segmentation unit. On the other hand, the nucleic acid sequences encoding the proteins are segmented with single codon as the segmentation unit. In a preferred embodiment of the present disclosure, the single codon referred to herein is triple codon.

[0050] Then, in step S120, one or more different segmentation units in the segmented nucleic acid sequences encoding the proteins corresponding to each identical segmentation unit in the segmented protein sequences are assigned with classification labels. For example, one segmentation unit in the protein sequence can correspond to one or more different segmentation units in the nucleic acid sequence, and the one or more different segmentation units are assigned with different classification labels respectively, so that one segmentation unit in the protein sequence can correspond to one or more classification labels. However, for the collected protein sequences and the nucleic acid sequences encoding the proteins, the segmentation unit at a specific position in the protein sequence corresponds to only one classification label, i.e. only one of the multiple different nucleic acid segmentation units. In a preferred embodiment, the synonymous codons of each amino acid are sorted in the order of frequency of occurrence in the protein expression system from high to low, and the corresponding classification labels are assigned respectively. For example, the classification labels can be numbers from 1 to 6, and different labels imply the change of codon frequency: label 1 corresponds to the highest frequency of occurrence of the synonymous codon of a certain amino acid, and labels 2 to 6 correspond to the frequency of occurrence in descending order. For example, reference can be made to the detailed description in the embodiments below and the correspondence between amino acids and codons and labels in Table 1 below.

[0051] Those skilled in the art should understand that the same segmentation unit of the same amino acid at different positions in the protein sequence is the same, but the label category can be different (see Table 1). Different classification labels of the same segmentation unit in the protein sequence represent that the same amino acid can be encoded by synonymous codons with different frequencies of occurrence.

[0052] In addition, in a preferred embodiment, the segmentation unit of a single amino acid can be represented by the word index (input ids) in the dictionary of the tokenizer. Those skilled in the art should recognize that the natural language processing model itself can include the tokenizer word index function. For example, the input protein sequence into the natural language processing model, the model will change the 20 amino acids of the protein sequence into numerical values, such as 1, 2, …, 20.

[0053] Alternatively, the amino acid features can also be extracted by a pre-trained protein model as token embedding vectors to represent the tokenized units of individual amino acids. In this way, the training process of the codon optimization algorithm fully considers the implicit feature information of each amino acid in the protein.

[0054] On the other hand, for the encoding nucleic acid information, the codon token embedding information can be extracted by a nucleic acid language model as training information to further improve the training effect.

[0055] In step S130, the tokenized protein sequence is labeled with the classification label of the corresponding tokenized unit in the encoding protein nucleic acid sequence according to the tokenized unit, and the labeled protein sequence is used as the training set.

[0056] In step S140, the natural language model is trained by the training set to obtain the trained natural language model. In the training process, the absolute or relative position encoding of the single amino acid in the protein sequence can be added to the training set. Those skilled in the art should recognize that in an embodiment, the input protein sequence and its corresponding classification label into the model already implicitly contains absolute position information. In addition, a model suitable for long text prediction can be preferentially selected, for example, a model that has achieved excellent performance in long text evaluation, as the natural language model used for training.

[0057] According to a preferred embodiment of the present disclosure, the natural language model can be selected from RoFormer, Bert, RoBERTa, Longformer, BigBird, Phi, DistilBERT or XLNet. More preferably, the natural language model can be BigBird.

[0058] Those skilled in the art should understand that, relative to Bert and T5 (512 words), models such as BigBird and Longformer are suitable for long text training and prediction (such as greater than or equal to 1024 words).

[0059] FIG. IB shows a flowchart of a codon optimization method based on tokenized unit classification according to an embodiment of the present disclosure. As shown in FIG. IB, the codon optimization method 100B from step S150 to step S170 is a process of predicting the optimized nucleic acid sequence of the protein sequence to be codon optimized by the trained natural language model.

[0060] It should be noted that the "protein sequence to be codon optimized" described in steps S150 to S170 can be a protein sequence that has not been subjected to nucleic acid encoding (from an amino acid sequence to a nucleic acid sequence), or a protein sequence that has been subjected to nucleic acid encoding. That is, the nucleic acid sequence encoding the protein can be optimized using the method of the present disclosure, and more specifically, the codons in the nucleic acid sequence encoding the protein corresponding to the amino acids in the protein sequence can be optimized.

[0061] In step S150, the protein sequence to be codon optimized is segmented. Corresponding to the model training process, in a preferred embodiment, the protein sequence to be codon optimized is segmented with a single amino acid as the segmentation unit. Similarly, the representation of the segmentation unit can use the word index in the dictionary of the segmenter, or use the amino acid features extracted by the pre-trained protein model as the word embedding vector.

[0062] In step S160, the segmented protein sequence to be codon optimized is input into the trained natural language model to predict the classification label corresponding to each segmentation unit of the segmented protein sequence to be codon optimized. As previously described, those skilled in the art should recognize that the natural language processing model itself can include a segmenter word index function. For example, the protein sequence is input into the natural language processing model, which converts the 20 amino acids of the protein sequence into numerical values such as 1, 2, …, 20. As previously described, the classification label here can be arranged in order of frequency of occurrence in the protein expression system from high to low, or arranged in order of frequency of occurrence in the protein expression system from low to high, or arranged in any other order.

[0063] In step S170, the optimized nucleic acid sequence of the protein sequence to be codon optimized is obtained according to the mapping relationship between the different segmentation units of the nucleic acid sequence encoding the protein and their classification labels (for example, still referring to Table 1).

[0064] On the other hand, correspondingly, for the nucleic acid sequence encoding the protein, the codon word embedding information can be extracted using the nucleic acid language model, and input into the trained natural language model to further improve the effect of model application.

[0065] As previously described, the natural language model can be selected from RoFormer, Bert, RoBERTa, Longformer, BigBird, Phi, DistilBERT or XLNet. More preferably, the natural language model can be BigBird.

[0066] As mentioned above, BigBird and Longformer are both suitable for long text training and prediction (e.g., greater than or equal to 1024 words) relative to Bert and T5 (512 words).

[0067] In step S170, specifically, each token unit of the nucleic acid sequence of the encoded protein corresponding to the codon-optimized protein sequence obtained according to the mapping relationship is sequentially combined to obtain an optimized nucleic acid sequence. More specifically, the codons corresponding to each amino acid of the codon-optimized protein sequence are sequentially combined to obtain an optimized nucleic acid sequence.

[0068] Finally, the obtained nucleic acid sequence of the encoded protein can also be further optimized by performing synonymous codon substitution on the harmful site sequence, the variable splicing site sequence, and / or the gene synthesis difficulty region sequence.

[0069] FIGS. 1A and 1B depict the steps of the method of training a natural language model and applying the model for codon optimization of a protein sequence. The similarities and differences between the natural language model training and model application processes are further explained below with reference to FIG. 2.

[0070] FIG. 2 is a schematic diagram of the association between natural language model training and model application in the codon optimization method based on token unit classification according to an embodiment of the present disclosure. In FIG. 2, the left side of the middle dashed line shows the model training process, and the right side of the dashed line shows the process of model application.

[0071] All the processes on the left and right sides of the dashed line in FIG. 2 are actually the natural language model training method and the codon optimization method based on token unit classification according to an embodiment of the present disclosure shown in FIGS. 1A and 1B. As can be seen from FIG. 2, whether it is the model training or the actual application (prediction) process, the sequence needs to be tokenized, so as to split the sequence into various token units. This process has been described in detail in the foregoing (for example, FIG. 1 and the corresponding textual description).

[0072] More specifically, in the model training process, both the collected protein sequences and their encoded nucleic acid sequences need to be tokenized. Preferably, the tokenization unit of the protein sequence is a single amino acid, while the tokenization unit of the nucleic acid sequence encoding the protein is a single codon, for example, each triplet codon. The tokenization unit of the protein sequence corresponds to the tokenization unit of the nucleic acid sequence encoding the protein. Correspondingly, in the model application process, the protein sequence to be codon-optimized needs to be tokenized. Preferably, the tokenization unit of the protein sequence is a single amino acid. In the model training process, according to the correspondence between a single amino acid and synonymous codons, the classification label is annotated. As described above, the classification label here is used to distinguish multiple different synonymous codons that can correspond to the same amino acid, and preferably reflects the frequency of synonymous codons corresponding to the same amino acid in the protein expression system. The correspondence between amino acids, synonymous codons, and classification labels can form a lookup table (for example, see Table 1 below). Those skilled in the art should recognize that although Table 1 is a lookup table for E. coli, similarly there can also be tables for mammals and the like. For example, the Codon Usage Database (http: / / www.kazusa.or.jp / codon / ) also provides codon usage frequency tables for expression systems such as yeast (Pichia pastoris and Saccharomyces cerevisiae), insects (Spodoptera frugiperda), and mammalian cells (human HEK cells, black line hamster CHO cells, and mouse NSO cells). In fact, each amino acid (tokenization unit) of the protein sequence and its corresponding classification label form a combination, which constitutes the training set for training the model. Using a natural language model as the model used in the method of the present disclosure, in the training process, the classification labels obtained by using the protein sequence and all its amino acids (single amino acid is the tokenization unit) as input features, selecting a suitable natural language model as the model used in the present disclosure, comparing the classification labels obtained by inputting each amino acid in the collected protein sequence into the model in training with the classification labels in the training set, thereby updating the model parameters, thereby fully training the model, and thus obtaining the final trained natural language model.

[0073] For the trained model, it can be put into practical application of amino acid classification label prediction, i.e., the process to the right of the dashed line in FIG. 2. In the model application process, after the protein sequence to be codon-optimized is tokenized into single amino acids, the protein sequence is input into the model as an input feature, and the classification label corresponding to each amino acid in the protein sequence is predicted. Then, according to the amino acid, the predicted classification label, and the lookup table reflecting the correspondence between the amino acid, the synonymous codon, and the classification label, the codon corresponding to each amino acid in the protein sequence is obtained, and finally combined into the optimized nucleic acid sequence encoding the protein.

[0074] From the above description, it can be seen that the codon optimization technology of the present disclosure is proposed because the number of synonymous codons for each amino acid is 1, 2, 3, 4, 6, respectively, and the particularity of the maximum number being only 6. Thus, it can be attempted to simplify the codon optimization problem into the Token Classification problem in natural language processing, that is, first tokenizing the protein sequence into single amino acid (1-mer) units (Tokens), and then performing unit classification (Token Classification) on each amino acid unit. Similar to the codon optimization problem, that is, the same amino acid can be encoded by different synonymous codons, and the same amino acid at different positions is allowed to be marked with different classification labels due to the difference of the corresponding synonymous codons. Thus, the process of predicting codons by the model is simplified to predicting classification labels, and then a lookup table reflecting the correspondence between amino acids, synonymous codons and classification labels is used to obtain the codon corresponding to each amino acid in the protein sequence, and finally combined into a nucleic acid sequence encoding the protein.

[0075] Method detailed description

[0076] The present disclosure provides a codon optimization method based on token unit classification. The specific embodiments of the method can consist of the following stages.

[0077] 1. Stage one: preparing training set

[0078] According to the proteome, transcriptome, genome analysis results and annotation information of the protein expression host, the collected protein sequences and the nucleic acid sequences of the encoded proteins are collected. The collected protein sequences can be natural protein sequences, or artificially designed protein sequences. The protein data source can be a genomic coding sequence, such as the protein coding sequence (CDS) of the Escherichia coli genome; or a training set of other codon optimization algorithms, such as artificially selected house-keeping genes. According to the difference of the species of the protein expression host, the training set can be different. For an Escherichia coli expression host, 4322 CDS of Escherichia coli str. K-12 substr. MG1655 in the NCBI genome database can be selected as the training set. For a mammalian cell expression host, 41078 CDS or part of the CDS of Cricetulus griseus genome assembly CriGri_1.0 in the NCBI genome database can be selected as the training set; or 146566 CDS or part of the CDS of a human being can be selected as the training set. The collected protein sequences are tokenized with single amino acids as tokenization units, and the coding nucleic acids are tokenized with single codons as tokenization units. The host system of protein expression and the training set should be corresponding, such as selecting a CHO expression system, the proteins in the training set should be natural proteins from CHO, or the proteins and their nucleic acid sequences that are highly expressed in the CHO system.

[0079] 2. Stage two: preparing token classification labels (i.e. classification labels of synonymous codons)

[0080] According to the codon frequency of the protein coding sequence (CDS) of the expression host in the NCBI Genome (https: / / www.ncbi.nlm.nih.gov / genome / ) or Codon Usage Database (http: / / www.kazusa.or.jp / codon / ) database, the synonymous codons of each amino acid are sorted in order of frequency from high to low, corresponding to the classification labels 1-6, respectively. Different labels imply the high and low changes of codon frequency (label 1 corresponds to the highest frequency of synonymous codons of an amino acid, 2-6 correspond to the frequency in turn decreasing), as shown in Table 1 for the codon optimization of the codon token unit for the expression host of Escherichia coli (the codon frequency is from the Escherichia coli codon table of Codon Usage Database http: / / www.kazusa.or.jp / codon / cgi-bin / showcodon.cgi?species=37762, as described above, Codon Usage Database also includes the codon usage frequency tables of yeast (Pichia pastoris and Saccharomyces cerevisiae), insects (Spodoptera frugiperda), and mammalian cells (human HEK cells, black line hamster CHO cells, and mouse NSO cells) and other expression systems).

[0081] 3. Stage three: labeling the amino acid token units (Tokens) in the training set

[0082] The label of each codon of the protein coding sequence in the training set is taken as the classification label of the corresponding amino acid token unit (Token) in the protein sequence. Since each codon token unit (Token) can correspond to one of the six classification labels, the labeled amino acid token unit (Token) can appear in the following five cases:

[0083] (1) Amino acid token units (Tokens) M and W have only one classification label (i.e., the amino acid is encoded by only one codon, and the label is 1)

[0084] (2) Amino acid token units (Tokens) C, D, E, F, H, K, N, Q, and Y can be divided into two categories (i.e., the amino acid is encoded by one of the two synonymous codons, and the label is 1 or 2)

[0085] (3) Amino acid token units (Tokens) I can be divided into three categories (i.e., the amino acid is encoded by one of the three synonymous codons, and the label is 1, 2, or 3)

[0086] (4) Amino acid token unit (Token) A, G, P, T, V can be divided into four categories (i.e. the amino acid is encoded by one of the four synonymous codons, labeled 1, 2, 3 or 4)

[0087] (5) Amino acid token unit (Token) L, R, S can be divided into six categories (i.e. the amino acid is encoded by one of the six synonymous codons, labeled 1, 2, 3, 4, 5 or 6)

[0088] Figure 3 is a schematic diagram of token unit classification of an example protein sequence. Protein sequence MTEYK (SEQ ID NO: 6), the corresponding protein coding sequence is ATGACTGAATACAAA (SEQ ID NO: 7), the single amino acid in the protein sequence is taken as the token unit, the single triplet codon in the nucleotide sequence corresponding to the encoded protein is taken as the token unit of the nucleotide sequence, and the classification label according to the synonymous codon setting of the E. coli expression system in Table 1 is labeled. The same amino acid at different positions in the protein sequence can be classified into different token unit (Token) categories (e.g. labeled with different classification labels), and different amino acids can also be classified into the same token unit (Token) category (e.g. labeled with the same classification label, as shown in Figure 3 and Table 1). Different labels represent the same amino acid needs to be encoded by different frequencies of synonymous codons.

[0089] The individual amino acid token can be represented using the word index (input ids) in the tokenizer's dictionary or using a pretrained protein model (see Pretrained protein language model such as ProtTrans (see Elnaggar, A., Heinzinger, M., Dallago, C., Rehawi, G., Wang, Y., Jones, L., Gibbs, T., Feher, T., Angerer, C. and Steinegger, M. (2021) Prottrans: Toward understanding the language of life through self-supervised learning. IEEE Trans. Pattern Anal. Mach. Intel l, 44, 7112-7127), ESM2 (see Lin, Z., Akin, H., Rao, R., Hie, B., Zhu, Z., Lu, W., dos Santos Costa, A., Fazel-Zarandi, M., Sercu, T. and Candido, S. (2022) Language models of protein sequences at the scale of evolution enable accurate structure prediction. BioRxiv, 2022, 500902, Rives, A., Meier, J., Sercu, T., Goyal, S., Lin, Z., Liu, J., Guo, D., Ott, M., Zitnick, C. L. and Ma, J. (2021) Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences. Proc. Natl. Acad. Sci., 118, e2016239118), Ankh (see Elnaggar, A., Essam, H., Salah-Eldin, W., Moustafa, W., Elkerdawy, M., Rochereau, C. and Rost, B. (2023) Optimized protein language model unlocks general-purpose modelling. bioRxiv) Extracting amino acid features as token embedding vectors allows the training process of codon optimization algorithms to take into account the implicit feature information of each amino acid in the protein.

[0090] 4. Stage four: training the model

[0091] Models that can accomplish the Token Classification task in natural language processing include RoFormer (see Su, J., Ahmed, M., Lu, Y., Pan, S., Bo, W. and Liu, Y. (2024) Roformer: Enhanced transformer with rotary position embedding. Neurocomputing, 568, 127063), Bert (see Devlin, J., Chang, M.-W., Lee, K. and Toutanova, K. (2018) Bert: Pre-training of deep bidirectional transformers for language understanding. ArXiv Prepr. ArXiv 181004805), RoBERTa (see Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L. and Stoyanov, V. (2019) Roberta: A robustly optimized bert pretraining approach. ArXiv Prepr. ArXiv 190711692), Longformer (see Beltagy, I., Peters, M.E. and Cohan, A. (2020) Longformer: The long-document transformer. ArXiv Prepr. ArXiv 200405150), BigBird (see Zaheer, M., Guruganesh, G., Dubey, K.A., Ainslie, J., Alberti, C., Ontanon, S., Pham, P., Ravula, A., Wang, Q. and Yang, L. (2020) Big bird: Transformers for longer sequences. Adv. Neural Inf. Process. Syst., 33, 17283-17297), Phi (see Gunasekar, S., Zhang, Y., Aneja, J., Mendes, C.C.T., Del Giorno, A., Gopi, S., Javaheripi, M., Kauffmann, P., de Rosa, G.and Saarikivi, O. (2023) Textbooks Are All You Need. ArXiv Prepr. ArXiv 230611644), DistilBERT (see Sanh, V., Debut, L., Chaumond, J. and Wolf, T. (2019) DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter. ArXiv Prepr. ArXiv 191001108), XLNet (see Yang, Z., Dai, Z., Yang, Y., Carbonell, J., Salakhutdinov, R. R. and Le, Q. V. (2019) Xlnet: Generalized autoregressive pretraining for language understanding. Adv. Neural Inf. Process. Syst., 32) and so on (the model list can be referred to https: / / huggingface.co / docs / transformers / tasks / token_classification). The labeled training set (including the completed protein sequence and the classification label of each amino acid token) is used as input to train any model that can complete the token classification task.

[0092] During the model construction and training process, the length of the protein coding sequence, the size of the training set, the limitation of the computing resources, and the final performance and generalization ability of the model are comprehensively considered:

[0093] (1) Absolute or relative position encodings can be added (such as Rotary Position Embedding, RoPE (see Hebditch, M. and Warwicker, J. (2019) Charge and hydrophobicity are key features in sequence-trained machine learning models for predicting the biophysical properties of clinical-stage antibodies. PeerJ, 7, e8199) or Attention with Linear Biases, ALiBi (see Al-Khateeb, F., Dey, N., Soboleva, D. and Hestness, J. (2023) Position Interpolation Improves ALiBi Extrapolation. ArXiv Prepr. ArXiv 231013017)) or prefer models that have already shown superior performance in long text evaluations (such as Longformer and BigBird (see Tay, Y., Dehghani, M., Abnar, S., Shen, Y., Bahri, D., Pham, P., Rao, J., Yang, L., Ruder, S. and Metzler, D. (2020) Long range arena: A benchmark for efficient transformers. ArXiv Prepr. ArXiv 201104006, Qin, G., Feng, Y. and Van Durme, B. (2022) The nlp task effectiveness of long-range transformers. ArXiv Prepr. ArXiv 220207856));

[0094] (2) Selecting appropriate Optimizer and Dropout (see Labach, A., Salehinejad, H. and Valaee, S. (2019) Survey of dropout methods for deep neural networks. ArXiv Prepr. ArXiv 190413310); such as SM3 (see Anil, R., Gupta, V., Koren, T. and Singer, Y. (2019) Memory efficient adaptive optimization. Adv. Neural Inf. Process. Syst., 32), Adafactor (see Shazeer, N. and Stern, M. (2018) Adafactor: Adaptive learning rates with sublinear memory cost. In. PMLR, pp. 4596-4604), Lion (see Chen, X., Liang, C., Huang, D., Real, E., Wang, K., Pham, H., Dong, X., Luong, T., Hsieh, C.-J. and Lu, Y. (2024) Symbolic discovery of optimization algorithms. Adv. Neural Inf. Process. Syst., 36) or CAME (see Luo, Y., Ren, X., Zheng, Z., Jiang, Z., Jiang, X. and You, Y. (2023) CAME: Confidence-guided Adaptive Memory Efficient Optimization. ArXiv Prepr. ArXiv 230702047) etc. when memory capacity is limited; such as through Monte-Carlo Dropout (see Gal, Y. and Ghahramani, Z. (2016) Dropout as a bayesian approximation: Representing model uncertainty in deep learning. In. PMLR, pp. 1050-1059) or Concrete Dropout (see Gal, Y., Hron, J. and Kendall, A. (2017) Concrete dropout.Adv. Neural Inf. Process. Syst., 30) and the like discard method prevents model overfitting.

[0095] (3) The optimization of learning rate can use learning rate scheduler (learning rate scheduler such as OneCycleLR (see Smith, L. N. and Topin, N. (2019) Super-convergence: Very fast training of neural networks using large learning rates. In. SPIE, Vol. 11006, pp. 369-386)) or automatic adjustment method (such as MECHANIC (see Cutkosky, A., Defazio, A. and Mehta, H. (2024) Mechanic: A learning rate tuner. Adv. Neural Inf. Process. Syst., 36));

[0096] (4) The hyperparameter selection of the model can use grid search or Ray Tune (see Liaw, R., Liang, E., Nishihara, R., Moritz, P., Gonzalez, J. E. and Stoica, I. (2018) Tune: A research platform for distributed model selection and training. ArXiv Prepr. ArXiv180705118) using hyperparameter optimization framework with genetic algorithm. During the training process, the numerical value of the loss function of the model will gradually decrease and eventually reach a plateau; or by adjusting the hyperparameters, the loss function is minimized or decreased to reach a plateau, and the training ends when it no longer continues to decrease.

[0097] Finally, part of the data in the training set can be extracted as an independent test set to test the performance of the trained model.

[0098] 5、Stage five: use the model

[0099] The protein sequence to be codon optimized is subjected to tokenization operation with single amino acid as token unit, the tokenized protein sequence is input into the trained model in step four, and the classification label of each token unit is predicted; and the optimized coding sequence is obtained according to the classification label and the synonymous codon mapping relationship (such as Table 1).

[0100] 6. Stage six: filter harmful sites, variable splice sites and / or gene synthesis difficulty regions and secondary optimization

[0101] For the reported harmful motifs in the expressed protein, the restriction site sequences of the molecular cloning operation, and the region sequences that may generate variable splice sites, reduce or delete the potential risk by replacing the synonymous codons in these regions of the optimized sequence. For the regions in the optimized sequence that are predicted to seriously affect gene synthesis and lead to increased synthesis difficulty, consider secondary optimization processing by manual or automatic optimization methods.

[0102] Example embodiments

[0103] The test case uses the Bigbird model (see Zaheer, M., Guruganesh, G., Dubey, K. A., Ainslie, J., Alberti, C, Ontanon, S., Pham, P., Ravula, A., Wang, Q. and Yang, L. (2020) Big bird: Transformers for longer sequences. Adv. Neural Inf. Process. Syst., 33, 17283-17297) and the training data comes from 4322 CDS of Escherichia coli str. K-12 substr. MG1655 (downloaded at https: / / ftp.ncbi.nlm.nih.gov / genomes / refseq / bacteria / Escherichia_coli / reference / GCF_000005845.2_ASM584v2 / GCF_000005845.2_ASM584v2_cds_from_genomic.fna.gz) and uses the pre-trained protein model ProtTrans (see Elnaggar, A., Heinzinger, M., Dallago, C, Rehawi, G., Wang, Y., Jones, L., Gibbs, T., Feher, T., Angerer, C. and Steinegger, M. (2021) Prottrans: Toward understanding the language of life through self-supervised learning. IEEE Trans. Pattern Anal. Mach. Intell., 44, 7112-7127) to extract amino acid features as token embeddings vectors for single amino acid token units, with a token embedding vector length of 1024; see Table 1 for the classification of single amino acid tokens. Bigbird uses a sparse-attention based transformer, which reduces the computational resource requirements (including training time and memory consumption) while improving the processing capacity for long texts.In the embodiment, the Bigbird model uses similar absolute position encoding as RoBERTa (see Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L. and Stoyanov, V. (2019) Roberta: A robustly optimized bert pretraining approach. ArXiv Prepr. ArXiv190711692), the multi-head module of the attention mechanism is set to 8 heads, the hidden layer of the encoder (Transformer encoder) is set to 4 layers, the sample size (batch) of each batch during training is 16, the training period (epoch) is 100, the drop ratio is 0.1, the activation function of the hidden layer is gelu_new, the learning rate is 5e-5, the attention type is block_sparse, and the attention window size is 10. The optimization sequence is shown in Table 2.

[0104] Table 2: Comparison of protein sequences and nucleic acid sequences encoding proteins

[0105] GenSmart (see patent document WO2020024917A1) codon optimization tool performed very well in multiple evaluations (see Goulet, D.R., Yan, Y., Agrawal, P., Waight, A.B., Mak, A.N. and Zhu, Y. (2022) Codon Optimization Using a Recurrent Neural Network. J. Comput. Biol., Dreismann, A.K., McClements, M.E., Barnard, A.R., Orhan, E., Hughes, J.P., Lachmann, P.J. and MacLaren, R.E. (2021) Functional expression of complement factor I following AAV-mediated gene delivery in the retina of mice and human cells. Gene Ther., 28, 265-276, Ranaghan, M.J., Li, J.J., Laprise, D.M. and Garvie, C.W. (2021) Assessing optimal: inequalities in codon optimization algorithms. BMC Biol., 19, 1-13, Dogan, Y., Barese, C.N., Schindler, J.W., Yoon, J.K., Unnisa, Z., Guda, S., Jacobs, M.E., Oborski, C., Maiwald, T. and Clarke, D.L. (2022) Screening chimeric GAA variants in preclinical study results in hematopoietic stem cell gene therapy candidate vectors for Pompe disease. Mol. Ther.-Methods Clin. Dev., 27, 464-487, Gao, Z., Ravendran, S., Mikkelsen, N.S., Haldrup, J., Cai, H., Ding, X., Paludan, S.R., Thomsen, M.K., Mikkelsen, J.G. and Bak, R.O.(2022) A truncated reverse transcriptase enhances prime editing by split AAV vectors. Mol. Ther., Karaiyan, P., Chang, C.C.H., Chan, E.-S., Tey, B.T., Ramanan, R.N. and Ooi, C.W. (2022) In silico screening and heterologous expression of soluble dimethyl sulfide monooxygenases of microbial origin in Escherichia coli. Appl. Microbiol. Biotechnol., 106, 4523-4537, Reshetnikov, V.V., Chirinskaite, A.V., Sopova, J.V., Ivanov, R.A. and Leonova, E.I. (2022) Translational potential of base-editing tools for gene therapy of monogenic diseases. Front. Bioeng. Biotechnol., 10, 942440), so it is selected as a comparative algorithm. This example takes E. coli BL-21(DE3) as an expression system, and the expression of human protein GTPase KRas isoform b (NCBI Accession No. NP_004976.2) as a test case (see Table 2 for specific protein sequence information, wild-type nucleic acid sequence, and nucleic acid sequence optimized by two optimization methods. The final test sequence contains an added His tag coding sequence (CATCATCATCATCATCAT, SEQ ID NO: 5) at the 3' end to facilitate expression identification). The wild-type sequence and the sequence optimized by the two optimization methods of the protein in Table 2 are synthesized and connected to the expression vector pET-30a(+). Escherichia coli BL21(DE3) is transfected, and the corresponding positive clones are selected. The selected positive clones are cultured and fermented, and protein expression is induced by IPTG at 15°C for 16 hours. The expression effects of the wild-type sequence, the GenSmart codon tool optimized sequence, and the optimized sequence of the present disclosure are compared. After expression, SDS-PAGE is used to detect whole bacterial samples (see Figure 4), and the expression amounts are shown in Table 3.

[0106] Figure 4 is an example SDS-PAGE gel map. In the figure, each lane respectively represents:

[0107] Lane PC1 : BSA (1 μg)

[0108] Lane PC2: BSA (2 μg)

[0109] Lane M: Protein molecular weight marker

[0110] Lane NC: Cell lysate (5 μL) of cells transfected with plasmid containing wild type sequence, without IPTG induction

[0111] Lane 1: Cell lysate (5 μL) of cells transfected with plasmid containing wild type sequence, with IPTG induction, incubated at 15°C for 16 hours

[0112] Lane NC1: Cell lysate (5 μL) of cells transfected with plasmid containing GenSmart optimized sequence, without IPTG induction

[0113] Lane 2: Cell lysate (5 μL) of cells transfected with plasmid containing GenSmart optimized sequence, with IPTG induction, incubated at 15°C for 16 hours

[0114] Lane NC2: Cell lysate (5 μL) of cells transfected with plasmid containing optimized sequence of the disclosure, without IPTG induction

[0115] Lane 3: Cell lysate (5 μL) of cells transfected with plasmid containing optimized sequence of the disclosure, with IPTG induction, incubated at 15°C for 16 hours

[0116] The specific protein expression evaluation experiment procedure is as follows:

[0117] 1) Transformation

[0118] Take BL21(DE3) competent cells from the ultra-low temperature freezer, place on ice to thaw; add pET-30a(+) plasmid containing the protein expression sequence (100 ng) to the competent cells, gently blow and suck to mix thoroughly, place on ice for 30 minutes, heat shock in a water bath at 42°C for 90 seconds, and place on ice for 3 minutes. Add 100 μl of room temperature LB liquid medium, incubate in a shaker at 37°C, 200 rpm for 60 minutes. Mix the bacterial solution and then spread it on a kanamycin-resistant plate, invert the plate and incubate at 37°C overnight.

[0119] 2) Small-scale expression

[0120] Pick two single colonies respectively, inoculate into 4 ml of LB test tube containing 50 μg / ml of kanamycin, and incubate in a shaker at 37°C, 200 rpm. When the OD600 reaches 0.6-0.8, add 0.5 mM IPTG to the test tube respectively, and incubate at 15°C for 16 hours.

[0121] 3) Sample preparation and SDS

[0122] 3.1 450 μl of the above fermentation broth was centrifuged, the precipitate was taken and resuspended in 300 μl of lysis buffer (50 mM Tris-Hcl, 150 mM NaCl, 5% glycerol, pH 8.0) and lysed by ultrasonic for 1 minute.

[0123] 3.2 Whole cell sample: 100 μl of the lysis solution was mixed with 50 μl of 5x loading buffer and heated at 100 °C for 10 minutes and centrifuged at 15000 rpm for 5 minutes. 5 μl of the centrifuged lysis solution was used to detect the expression of protein by SDS-PAGE.

[0124] Table III: Comparison of expression of various sequences

[0125] Furthermore, nucleic acid language models such as DNABert (see Ji, Y., Zhou, Z., Liu, H. and Davuluri, R. V. (2021) DNABERT: pre-trained Bidirectional Encoder Representations from Transformers model for DNA-language in genome. Bioinformatics, 37, 2112-2120, Zhou, Z., Wu, W., Ho, H., Wang, J., Shi, L., Davuluri, R. V., Wang, Z. and Liu, H. (2024) DNABERT-S: Learning Species-Aware DNA Embedding with Genome Foundation Models. ArXiv Prepr. ArXiv 240208777), CodonBert (see Babjac, A. N., Lu, Z. and Emrich, S. J. (2023) CodonBERT: Using BERT for Sentiment Analysis to Better Predict Genes with Low Expression. In. pp. 1-6), RNABert (see Akiyama, M. and Sakakibara, Y. (2022) Informative RNA base embedding for RNA structural alignment and clustering by deep representation learning. NAR Genomics Bioinforma., 4, lqac012) and the like extract codon token embedding information as model training and input or will be used to further improve the performance of the model of the present disclosure.The skilled person will appreciate that the nucleic acid language model can be used for: genome-wide variant effect prediction, DNA cis-regulatory region prediction, DNA protein interaction prediction, DNA methylation (6mA, 4mC 5hmC) prediction, predicting RNA splice sites from DNA sequences, RNA 2D / 3D structure prediction, RNA structure alignment, RNA family clustering, predicting RNA splice sites from RNA sequences, predicting RNA N7-methylguanosine modification, predicting RNA 2'-O-methylation modification, multiple types of RNA modification prediction, predicting associations between miRNAs, IncRNAs and diseases, identifying IncRNAs IncRNA coding potential prediction, protein expression and mRNA degradation prediction, and the like.

[0126] Further, the person of ordinary skill in the art should appreciate that the method of the present disclosure can be implemented as a computer program. As described above in conjunction with the drawings, the method of the above-described embodiments is executed by one or more programs, and the instructions in the programs cause a computer or a processor to execute the algorithms described in conjunction with the drawings. These programs can be stored and provided to the computer or the processor using various types of non-transitory computer-readable media. The non-transitory computer-readable media include various types of tangible storage media. Examples of the non-transitory computer-readable media include magnetic recording media such as a floppy disk, a magnetic tape, and a hard disk drive, magneto-optical recording media such as a magneto-optical disk, CD-ROM (compact disc read-only memory), CD-R, CD-R / W, and semiconductor memory such as ROM, PROM (programmable ROM), EPROM (erasable PROM), flash ROM, and RAM (random access memory). Further, these programs can be provided to the computer by using various types of transitory computer-readable media. Examples of the transitory computer-readable media include electric signals, optical signals, and electromagnetic waves. The transitory computer-readable media can be used to provide the programs to the computer through wired communication paths such as electric wires and optical fibers or wireless communication paths.

[0127] For example, according to one embodiment of the present disclosure, a non-transitory computer-readable storage medium can be provided for storing a computer program for optimizing a nucleic acid sequence encoding a protein, the computer program including instructions that, when executed by a processor of an electronic device, cause the electronic device to implement the training method of a natural language model for codon optimization or the codon optimization method as described above.

[0128] In addition, according to the content of the present disclosure, a system for optimizing a nucleic acid sequence encoding a protein can also be provided, the system including: a processor; a memory; and a computer program. The computer program is stored in the memory and is configured to be executed by the processor. The computer program includes instructions for implementing the training method of a natural language model for codon optimization or the codon optimization method described above.

[0129] In addition, the present disclosure also relates to a computer program product comprising a computer program for optimizing a nucleic acid sequence encoding a protein, characterized in that the computer program, when executed by a processor, implements the training method of the natural language model for codon optimization or the codon optimization method as described above.

[0130] On the other hand, according to the above disclosure, the present disclosure can also have the following applications. For example, an isolated nucleic acid molecule comprising the optimized nucleic acid sequence obtained by the codon optimization method as described above can be provided.

[0131] A vector comprising the isolated nucleic acid molecule as described above can also be provided.

[0132] In addition, a recombinant host cell comprising the isolated nucleic acid molecule as described above or the vector as described above is provided.

[0133] In addition, according to an embodiment of the present disclosure, a method of expressing a protein in a host cell is provided, the method comprising: (a) obtaining an optimized nucleic acid sequence for expressing the protein in the host cell using the codon optimization method as described above; (b) synthesizing a nucleic acid molecule comprising the optimized nucleic acid sequence; (c) introducing the nucleic acid molecule into the host cell to obtain a recombinant host cell; and (d) culturing the recombinant host cell under conditions permitting expression of the protein from the optimized nucleic acid sequence.

[0134] Embodiments of the present disclosure are not limited to the above examples, and those of ordinary skill in the art can make various changes and improvements to the present disclosure in form and detail without departing from the spirit and scope of the present disclosure, and these are all considered to fall within the protection scope of the present disclosure.

Claims

1. A method for training a codon-optimized natural language model, the method comprising: The method comprises: According to the collected protein sequence and the nucleic acid sequence encoding the protein, respectively, the word segmentation is carried out; The classification label is set for one or more different segmentation units in the nucleic acid sequence encoding the protein corresponding to each same segmentation unit in the segmented protein sequence; The classification label of the corresponding segmentation unit in the nucleic acid sequence encoding the protein is marked on the segmented protein sequence according to the segmentation unit, and the marked protein sequence is used as a training set; The natural language model is trained by using the training set, and a trained natural language model is obtained.

2. The method of claim 1, wherein, The word segmentation is carried out according to the collected protein sequence and the nucleic acid sequence encoding the protein, respectively, which comprises: the protein sequence is segmented by taking a single amino acid as a segmentation unit, and the nucleic acid sequence encoding the protein is segmented by taking a single codon as a segmentation unit.

3. The method of claim 2, wherein, The synonymous codons of each amino acid are sorted in descending order of frequency in the expression system of the protein, and the corresponding classification label is set.

4. The method according to claim 2 or 3, characterized in that, The word index in the dictionary of the word segmenter is used to represent the segmentation unit of the single amino acid.

5. The method according to any one of claims 2-4, characterized in that, The amino acid features are extracted by using the pre-trained protein model as the word embedding vector to represent the segmentation unit of the single amino acid.

6. The method according to any one of claims 2-5, characterized in that, The natural language model is further trained by using the training set, which comprises: The absolute or relative position coding of the single amino acid in the protein sequence is added in the training set.

7. The method according to any one of claims 1 to 6, characterized in that, The natural language model is further trained by using the training set, which comprises: The model suitable for long text prediction is selected as the natural language model.

8. The method according to any one of claims 2-7, characterized in that, The natural language model is further trained by using the training set, which comprises: The codon word embedding information extracted by using the nucleic acid language model is added in the training set.

9. The method according to any one of claims 1-8, characterized in that, The natural language model is selected from RoFormer, Bert, RoBERTa, Longformer, BigBird, Phi, DistilBERT or XLNet; preferably, the natural language model is BigBird.

10. A codon-optimized natural language model trained by the method of any one of claims 1-9.

11. A method of codon optimization, characterized in that, The method comprises: The protein sequence to be codon-optimized is segmented; The segmented protein sequence to be codon-optimized is input into the natural language model trained according to the method of any one of claims 1-9, and the classification label corresponding to each segmentation unit of the segmented protein sequence to be codon-optimized is predicted; The optimized nucleic acid sequence of the protein sequence to be codon-optimized is obtained according to the mapping relationship between the different segmentation units of the nucleic acid sequence encoding the protein and the classification labels thereof.

12. The method of claim 11, wherein, The protein sequence to be codon-optimized is segmented by taking a single amino acid as a segmentation unit, and the segmentation unit of the nucleic acid sequence encoding the protein is a single codon.

13. The method of claim 12, wherein, The classification label is set by sorting the synonymous codons of each amino acid in descending order of frequency in the protein expression system.

14. The method according to claim 12 or 13, characterized in that, The inputting the codon-optimized protein sequence after word segmentation into the trained natural language model further comprises: The codon word embedding information extracted using the nucleic acid language model is input into the trained natural language model.

15. The method according to any one of claims 11-14, characterized in that, The natural language model is selected from RoFormer, Bert, RoBERTa, Longformer, BigBird, Phi, DistilBERT or XLNet; preferably BigBird.

16. The method according to any one of claims 11-15, characterized in that, The method further comprises: The obtained nucleic acid sequence encoding the protein is optimized by performing synonymous codon substitution on the sequence of the harmful site, the sequence of the alternative splicing site and / or the sequence of the gene synthesis difficulty region.

17. The method according to any one of claims 11-16, characterized in that, The mapping relationship between the different word segmentation units of the nucleic acid sequence encoding the protein and the classification tags thereof comprises: The optimized nucleic acid sequence is obtained by sequentially combining each word segmentation unit of the nucleic acid sequence encoding the protein corresponding to each word segmentation unit of the codon-optimized protein sequence obtained according to the mapping relationship.

18. The method of claim 17, wherein, The mapping relationship between the different word segmentation units of the nucleic acid sequence encoding the protein and the classification tags thereof comprises: The optimized nucleic acid sequence is obtained by sequentially combining the codons corresponding to each amino acid of the codon-optimized protein sequence.

19. A non-transitory computer-readable storage medium for storing a computer program for optimizing a nucleic acid sequence encoding a protein, the computer program comprising instructions which, when executed by a processor of an electronic device, cause the electronic device to implement the training method of the natural language model for codon optimization according to any one of claims 1-9 or the codon optimization method according to any one of claims 11-18.

20. A system for optimizing a nucleic acid sequence encoding a protein, the system comprising: a processor; a memory; and a computer program, wherein the computer program is stored in the memory and configured to be executed by the processor, the computer program comprising instructions for implementing the training method of the natural language model for codon optimization according to any one of claims 1-9 or the codon optimization method according to any one of claims 11-18.

21. A computer program product comprising a computer program for optimizing a nucleic acid sequence encoding a protein, characterized in that, The computer program is executed by the processor to implement the training method of the natural language model for codon optimization according to any one of claims 1-9 or the codon optimization method according to any one of claims 11-18.

22. An isolated nucleic acid molecule comprising the optimized nucleic acid sequence obtained by the codon optimization method according to any one of claims 11-18.

23. A vector comprising the isolated nucleic acid molecule of claim 22.

24. A recombinant host cell comprising the isolated nucleic acid molecule of claim 22 or the vector of claim 23.

25. A method of expressing a protein in a host cell, the method comprising: (a) using the codon optimization method of any one of claims 11-18 to obtain an optimized nucleic acid sequence that expresses the protein in the host cell; (b) synthesizing a nucleic acid molecule comprising the optimized nucleic acid sequence; (c) introducing the nucleic acid molecule into the host cell to obtain a recombinant host cell; and (d) culturing the recombinant host cell under conditions that allow expression of the protein from the optimized nucleic acid sequence.

Images