Training methods for biological models and processing methods for biological sequences
The biological model training method addresses the lack of multimodal protein-RNA sequence analysis by constructing and training models to capture their interaction, improving sequence understanding and prediction for applications in disease diagnosis and drug development.
Patent Information
- Application Number
- CN202411010942.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-26
- Publication Date
- 2024-12-27
- Estimated Expiration
- 2044-07-26
AI Technical Summary
Existing machine learning methods primarily focus on single-modal systems for protein sequences, neglecting the intrinsic relationship between protein and RNA sequences, which is crucial for fields like protein engineering and genetic engineering.
A biological model training method that constructs protein-RNA sequence pairs, performs word segmentation, and uses a multi-task training approach with a language model framework to capture the interaction between protein and RNA sequences, including masking and attention matrix techniques.
Enhances the understanding and prediction of protein-RNA interactions, enabling applications in disease diagnosis, drug development, and synthetic biology by optimizing sequence pairing and prediction capabilities.
Smart Images
Figure CN118538299B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of artificial intelligence, specifically machine learning, data processing technology, and bioinformatics, and more specifically to a biological model training method and a biological sequence processing method. Background Art
[0002] With the rapid development of bioinformatics and computational biology, protein sequence and messenger RNA (mRNA) coding sequence analysis has become one of the important means of studying biology. Proteins are the executors of most life functions, while mRNA is the information medium from gene to protein expression. Understanding and predicting the structure and function of protein sequences, and how they are translated from RNA sequences, are of great significance to the development of new drugs, treatment of genetic diseases, genetic engineering, and gene editing. Summary of the invention
[0003] In view of the above problems, the present invention provides a biological model training method and a biological sequence processing method.
[0004] According to a first aspect of the present invention, a method for training a biological model is provided, comprising: constructing a protein-ribonucleic acid sequence pair according to a preset input format based on a protein sequence data set and a RNA sequence data set, wherein the protein-ribonucleic acid sequence pair includes a preset prompt word; performing word segmentation processing on the protein sequence data set and the RNA sequence pair using a preset word segmenter to obtain a training sequence pair; inputting the training sequence pair into the biological model to obtain a predicted sequence pair corresponding to the training sequence pair; training the biological model according to the predicted sequence pair, the training sequence pair and the preset prompt word to obtain a trained biological model.
[0005] According to an embodiment of the present invention, the above-mentioned inputting the training sequence pair into the biological model to obtain the predicted sequence pair corresponding to the training sequence pair also includes: masking a preset proportion of words in the training sequence pair to obtain a masked training sequence pair; and inputting the above-mentioned masked training sequence pair into the biological model to obtain a predicted sequence pair.
[0006] According to an embodiment of the present invention, the above-mentioned masking processing of a preset proportion of words in the training sequence pair to obtain the masked training sequence pair also includes: selecting a preset proportion of words in the training sequence pair to obtain words to be masked; selecting a first preset sub-proportion of words in the above-mentioned words to be masked and replacing them with preset characters; selecting a second preset sub-proportion of words in the above-mentioned words to be masked and replacing them with random characters to obtain the masked training sequence pair.
[0007] According to an embodiment of the present invention, the biological model includes an attention matrix, which is composed of a first sub-matrix, a second sub-matrix, a third sub-matrix and a fourth sub-matrix, the first sub-matrix is located in the first quadrant, the second sub-matrix is located in the second quadrant, the third sub-matrix is located in the third quadrant and the fourth sub-matrix is located in the fourth quadrant; the method for constructing the attention matrix includes: using the prediction sequence located in the first half of the prediction sequence pair and the training sequence located in the first half of the training sequence pair to construct the second sub-matrix in the biological model, wherein the second sub-matrix is an all-zero matrix; using the prediction sequence located in the second half of the prediction sequence pair and the training sequence located in the second half of the training sequence pair to train the fourth sub-matrix in the biological model, the upper triangular area of the fourth sub-matrix is a non-zero matrix, and the lower triangular area is a zero matrix.
[0008] According to an embodiment of the present invention, a biological model is trained according to prediction sequence pairs, training sequence pairs and preset prompt words to obtain a trained biological model, which includes: multi-task training is performed on the biological model according to the above-mentioned prediction sequence pairs, training sequence pairs and preset prompt words to obtain a trained biological model, wherein the multi-task includes a bidirectional understanding task for protein sequences, a bidirectional understanding task for RNA sequences, a unidirectional generation task for protein sequences and a unidirectional generation task for RNA sequences.
[0009] Another aspect of the present invention provides a method for processing a biological sequence, comprising: obtaining a biological sequence to be processed, wherein the biological sequence to be processed includes a protein sequence to be processed or a ribonucleic acid sequence to be processed; performing word segmentation processing on the biological sequence to be processed using a preset word segmenter to obtain a biological sequence to be input; inputting the biological sequence to be input into a biological model, and generating a predicted sequence corresponding to the biological sequence to be input when it is determined that the biological sequence to be input includes a preset prompt word, wherein the sequence category of the predicted sequence is different from the sequence category of the biological sequence to be processed.
[0010] According to an embodiment of the present invention, in the case where it is determined that the biological sequence to be input includes a preset prompt word, generating a predicted sequence corresponding to the biological sequence to be input includes: when the biological sequence to be input is a protein sequence to be processed and the protein sequence to be processed includes a preset prompt word, generating a messenger RNA coding predicted sequence corresponding to the protein sequence to be processed; when the biological sequence to be input is a RNA sequence to be processed and the RNA sequence to be processed includes a preset prompt word, generating a protein predicted sequence corresponding to the RNA sequence to be processed.
[0011] According to an embodiment of the present invention, the method further includes: generating a high-dimensional feature matrix representing the biological sequence to be input when it is determined that the biological sequence to be input does not include a preset prompt word.
[0012] According to an embodiment of the present invention, the present invention provides a method for analyzing and modeling a protein sequence in combination with a RNA sequence. By utilizing a language model framework, a high-precision sequence modeling is performed on a single amino acid or single base level, thereby improving the understanding of the interaction between protein and messenger RNA. At the same time, the biological model based on biological sequences constructed by the present invention has a wide range of application scenarios, including but not limited to disease diagnosis, drug development, personalized medicine, and synthetic biology. The present invention can be used not only to develop new biological products, but also to improve the performance and production capacity of existing products. The construction process of the training model of the present invention not only strengthens the understanding of a single biological sequence, but also optimizes the pairing, interaction, and prediction capabilities between two sequences, providing a new direction for biotechnology research and medical applications. BRIEF DESCRIPTION OF THE DRAWINGS
[0013] The above contents and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:
[0014] Figure 1 A flow chart of a method for training a biological model according to an embodiment of the present invention is shown;
[0015] Figure 2 Shows a structural diagram of an attention matrix according to an embodiment of the present invention;
[0016] Figure 3 A flow chart of a biological model training method according to another embodiment of the present invention is shown;
[0017] Figure 4 A flow chart of a method for processing a biological sequence according to an embodiment of the present invention is shown;
[0018] Figure 5 A flow chart of a method for processing a biological sequence according to another embodiment of the present invention is shown;
[0019] Figure 6 A structural block diagram of a biological model training device according to an embodiment of the present invention is shown;
[0020] Figure 7 A structural block diagram of an apparatus for a biological sequence processing method according to an embodiment of the present invention is shown. DETAILED DESCRIPTION
[0021] Below, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present invention. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of embodiments of the present invention. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of concepts of the present invention.
[0022] The terms used herein are only for describing specific embodiments and are not intended to limit the present invention. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.
[0023] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.
[0024] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).
[0025] In the embodiments of the present invention, the collection, updating, analysis, processing, use, transmission, provision, disclosure, storage and other aspects of the data involved (for example, including but not limited to user personal information) are in compliance with the provisions of relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. In particular, necessary measures are taken for user personal information to prevent illegal access to user personal information data and maintain the security of user personal information and network security.
[0026] Currently, there are a lot of research on pattern recognition and prediction tools using biological sequences, especially using machine learning methods to model and analyze protein sequences. However, most methods focus on the study of protein sequences, ignoring the relationship between protein sequences and other biological macromolecules, such as DNA and RNA sequences. In addition, existing machine learning methods are often single-modal systems when dealing with these biological sequences.
[0027] Generating multimodal models of protein and RNA sequences is extremely valuable for fields such as protein engineering and genetic engineering. However, existing computational models have failed to provide a satisfactory solution. Therefore, a new method is needed that can capture the intrinsic relationship between protein and RNA sequences and is applicable to the language characteristics of the sequences of both biological macromolecules.
[0028] To achieve the above object, an embodiment of the present invention provides a biological model training method. Figure 1 A flow chart of a biological model training method according to an embodiment of the present invention is shown, including operations S101 to S104:
[0029] Operation S101: constructing a protein-ribonucleic acid sequence pair according to a protein sequence dataset and a ribonucleic acid sequence dataset in a preset input format, wherein the protein-ribonucleic acid sequence pair includes a preset prompt word.
[0030] Operation S102: using a preset word segmenter to perform word segmentation processing on the protein sequence data set and the RNA sequence pair to obtain a training sequence pair.
[0031] Operation S103: input the training sequence pair into the biological model to obtain a prediction sequence pair corresponding to the training sequence pair.
[0032] Operation S104: training a biological model according to the predicted sequence pairs, the training sequence pairs and the preset prompt words to obtain a trained biological model.
[0033] In operation S101, a protein sequence data set is provided by a protein database or a biological database, and protein sequence information is extracted for learning the representation of protein sequences; a RNA sequence data set is provided by a biological database, and RNA sequence information is extracted for learning the representation of RNA sequences; and the correspondence between protein sequence and RNA sequence data is extracted according to the amino acid-nucleotide codon pairing rule to construct protein-RNA sequence pairs.
[0034] In some embodiments of the present invention, the data formats of protein sequence, RNA sequence and constructed protein-RNA sequence pair are all in FASTA format.
[0035] In operation S101, the preset input format may be an input format corresponding to the model. For example, the preset format may include: "'<|start|>'+'protein sequence'+'<|end|>'+'<|prompt|>' +'ribonucleic acid sequence'+'<|end|>'" and "'<|start|>'+'ribonucleic acid sequence'+'<|end|>'+'<|prompt|>'+'protein sequence'+'<|end|>'"; where '<|prompt|>' is the preset prompt word.
[0036] In operation S102, the preset word segmenter may include a word segmenter for protein sequences and a word segmenter for messenger RNA. The word segmenter for protein sequences may be a word list with a single amino acid as a subword; the word segmenter for RNA sequences may be a word list with a single base as a subword, and the two word lists exist independently. The union of the protein sequence word segmenter and the RNA sequence word segmenter is used as the model total word segmenter, i.e., the preset word segmenter, so as to adapt to a wider range of downstream tasks.
[0037] In some embodiments of the present invention, the vocabulary size of protein sequences is 25 ("L", "A", "G", "V", "S", "E", "R", "T", "I", "D", "P", "K", "Q", "N", "F", "Y", "M", "H", "W", "C", "X", "B", "U", "Z", "O"), the vocabulary size of RNA sequences is 5 ("a", "u", "c", "g", "n",), and the size of special characters is 8 ("<|start|>", "<|end|>", "<|prompt|>", " <mask> ”," <pad> ”," <unkown>”, “.”, “-”), the final total vocabulary size is 38.
[0038] In operation S102, the training sequence pair is a sequence pair downloaded from a database, which has a known correspondence between a protein sequence and a messenger RNA, that is, a sequence pair in which three known messenger RNA bases correspond to one protein.
[0039] In operation S104, the trained bilingual model has two functions: understanding and generation. Depending on whether a preset prompt word is input, it is determined whether the model performs an understanding task or a generation task.
[0040] According to an embodiment of the present invention, the present invention provides a method for analyzing and modeling a protein sequence in combination with a RNA sequence. By utilizing a language model framework, a high-precision sequence modeling is performed on a single amino acid or single base level, thereby improving the understanding of the interaction between protein and messenger RNA. At the same time, the biological model based on biological sequences constructed by the present invention has a wide range of application scenarios, including but not limited to disease diagnosis, drug development, personalized medicine, and synthetic biology. The present invention can not only be used to develop new biological products, but also to improve the performance and production capacity of existing products. The construction process of the training model of the present invention not only strengthens the understanding of a single biological sequence, but also optimizes the pairing, interaction, and prediction capabilities between two sequences, providing a new direction for biotechnology research and medical applications; at the same time, it understands and generates protein and RNA sequences, providing a basis for downstream application docking.
[0041] According to an embodiment of the present invention, the step of inputting the training sequence pair into the biological model to obtain the predicted sequence pair corresponding to the training sequence pair further includes: performing masking processing on words of a preset proportion in the training sequence pair to obtain the masked training sequence pair;
[0042] The masked training sequence pairs are input into the biological model to obtain predicted sequence pairs.
[0043] According to an embodiment of the present invention, masking is performed on a preset proportion of words in a training sequence pair to obtain a masked training sequence pair, which may include: randomly selecting a preset proportion of words from the training sequence pair for masking, or selecting a preset proportion of words from a specific area in the training sequence pair for masking.
[0044] According to an embodiment of the present invention, the preset ratio is a preset ratio, for example, the preset ratio may be 15%, 20%, etc.
[0045] According to an embodiment of the present invention, the masking process is performed on the words of the preset ratio in the training sequence pair to obtain the masked training sequence pair, and the masked training sequence pair further includes:
[0046] Selecting words with a preset ratio in the training sequence pair to obtain words to be masked;
[0047] Selecting a first preset sub-proportion of words from the words to be masked and replacing them with preset characters;
[0048] A second preset sub-proportion of words in the words to be masked is selected and replaced with random characters to obtain the masked training sequence pair.
[0049] According to an embodiment of the present invention, the masking process may include randomly masking 15% of the subwords in the training sequence, i.e., a preset proportion; wherein, among the 15% of the words in the training sequence, a first preset sub-proportion, e.g., 80% of the words are replaced by <mask>special characters; replacing a second predetermined sub-proportion, e.g. 10%, of the words with random characters; and leaving the other 10% of the words unchanged.
[0050] According to an embodiment of the present invention, the biological model includes an attention matrix, the attention matrix is composed of a first sub-matrix, a second sub-matrix, a third sub-matrix and a fourth sub-matrix, the first sub-matrix is located in the first quadrant, the second sub-matrix is located in the second quadrant, the third sub-matrix is located in the third quadrant and the fourth sub-matrix is located in the fourth quadrant;
[0051] The construction method of the attention matrix includes:
[0052] Using the prediction sequence in the first half of the prediction sequence pair and the training sequence in the first half of the training sequence pair, a second submatrix in the biological model is constructed, wherein the second submatrix is an all-zero matrix;
[0053] The fourth submatrix in the biological model is trained using the prediction sequence in the second half of the prediction sequence pair and the training sequence in the second half of the training sequence pair. The upper triangular area of the fourth submatrix is a non-zero matrix and the lower triangular area is a zero matrix.
[0054] According to an embodiment of the present invention, the concatenated attention matrix is used as the attention matrix of the model.
[0055] Figure 2 A structural diagram of an attention matrix according to an embodiment of the present invention is shown.
[0056] like Figure 2 As shown, Figure 2 The upper right matrix is the first submatrix of the first quadrant, the upper left matrix is the second submatrix of the second quadrant, the lower left matrix is the third submatrix of the third quadrant, and the lower right matrix is the fourth submatrix of the fourth quadrant. The prefix decoder language model based on shared weights is used as the main framework of the model for multi-task training, and the control of bidirectional and unidirectional is achieved through a variable attention matrix. For the training of protein sequence-RNA sequence pairs, the method of diagonally splicing the upper triangular matrix with all zeros in the first half and the upper triangular matrix with -50000 in the second half is used as the attention matrix.
[0057] In some embodiments of the present invention, the torch.cat method of pytorch is used to construct the attention matrix in the attention layer, and four attention matrices are spliced into one attention matrix; except for the attention layer, the model network layer based on transformer is used. According to an embodiment of the present invention, when the model performs an understanding task, the attention matrix in the first half of the second quadrant in the upper left corner is called to achieve the same capability as the Bidirectional Encoder Representations from Transformer (BERT) model based on Transformer; when the model performs a generation task, the model calls the lower triangular attention matrix in the second half to achieve the same capability as the Generative Pre-Trained (GPT) model.
[0058] According to an embodiment of the present invention, a biological model is trained according to a prediction sequence pair, a training sequence pair and a preset prompt word, and the trained biological model is obtained, including:
[0059] The biological model is subjected to multi-task training according to the above-mentioned prediction sequence pairs, training sequence pairs and preset prompt words to obtain a trained biological model, wherein the multi-task includes a bidirectional understanding task for protein sequences, a bidirectional understanding task for RNA sequences, a unidirectional generation task for protein sequences and a unidirectional generation task for RNA sequences.
[0060] According to an embodiment of the present invention, the multi-task training of the model includes that the training task is a cloze task, and the framework can simultaneously unify bidirectional and unidirectional tasks.
[0061] According to an embodiment of the present invention, the bidirectional understanding task of protein sequences and messenger RNA is to obtain a representation vector. According to the constructed attention matrix, bidirectional understanding is achieved in the upper left second quadrant and the lower right third quadrant, and obtaining a representation vector is the bidirectional understanding task; the unidirectional generation task is to perform a unidirectional left-to-right autoregressive task on protein sequences and RNA sequences.
[0062] According to an embodiment of the present invention, the task can be subdivided into a bidirectional understanding Masked Learning Model (MLM) task for protein sequences, a bidirectional understanding MLM task for RNA sequences, a unidirectional generation causal language model (CLM) task for protein sequences, and a unidirectional generation CLM task for RNA sequences.
[0063] According to an embodiment of the present invention, the model multi-task training loss function can be expressed by formula (1):
[0064] ; (1)
[0065] Among them, L represents the loss value; M represents the set of masked words; m i represents the i-th masked word; θ represents the model topic parameter; θ 1 Represents the model prediction head parameters.
[0066] According to an embodiment of the present invention, when a preset prompt word is input, the trained biological model will execute the task of generating a corresponding RNA sequence and understanding when a protein sequence and a preset prompt word are input; when a RNA sequence and a preset prompt word are input, the trained biological model will execute the task of generating a corresponding protein sequence and understanding.
[0067] According to an embodiment of the present invention, without inputting a preset prompt word, a high-dimensional feature matrix of an amino acid sequence or a ribonucleic acid sequence can be obtained through a pre-trained model, and the matrix can be used for downstream classification tasks or regression tasks; the pre-trained model can capture the high-dimensional correlation between amino acid sequences and ribonucleic acid sequences, and can generate a sequence of another language modality from a known language modality through autoregression.
[0068] According to an embodiment of the present invention, the training method of the biological model may include:
[0069] The protein sequence-RNA sequence dataset is divided into a training set and a validation set.
[0070] The data loader loads the data in the training set in two data formats: '<|start|>' + 'protein sequence' + '<|end|>' + '<|prompt|>' + 'RNA sequence' + '<|end|>' and '<|start|>' + 'RNA sequence' + '<|end|>' + '<|prompt|>' + 'protein sequence' + '<|end|>', and uses gradient accumulation to control the iteration batch size.
[0071] Among them, pytorch is used as the main programming framework, and DeepSpeed is used as the main training acceleration framework for model training. DeepSpeed training model is used as the training framework; AdamW (Adam Weight DecayOptimizer) optimizer is used, and the learning rate is 2e -4 , warm-up steps are 3000, epsilon value is 1e -8 The data set is divided into training set and validation set at a ratio of 1000:1, and the gradient accumulation method is used for training. The mini batch size of a single card is 1, and the model parameters are accumulated and updated in 10 steps. The learning rate scheduling strategy adopts the cosine warm-up strategy.
[0072] Figure 3 A flow chart of a biological model training method according to another embodiment of the present invention is shown, including operations S301 to S307:
[0073] Operation S301: constructing a protein-ribonucleic acid sequence pair according to a protein sequence dataset and a ribonucleic acid sequence dataset in a preset input format, wherein the protein-ribonucleic acid sequence pair includes a preset prompt word.
[0074] Operation S302: Use a preset word segmenter to perform word segmentation processing on the protein sequence data set and the RNA sequence pair to obtain a training sequence pair.
[0075] Operation S303: Select a preset ratio of words in the training sequence to obtain words to be masked.
[0076] Operation S304: Select words of a first preset sub-proportion in the words to be masked and replace them with preset characters.
[0077] Operation S305: Select words of a second preset sub-proportion in the words to be masked and replace them with random characters to obtain masked training sequence pairs.
[0078] Operation S306: input the masked training sequence pair into the biological model to obtain a predicted sequence pair.
[0079] Operation S307: training the biological model according to the predicted sequence pair, the training sequence pair and the preset prompt words to obtain a trained biological model.
[0080] According to an embodiment of the present invention, a method for processing a biological sequence is provided.
[0081] Figure 4 A flow chart of a biological sequence processing method according to an embodiment of the present invention is shown, including operations S401 to S403.
[0082] In operation S401 : obtaining a biological sequence to be processed, wherein the biological sequence to be processed includes a protein sequence to be processed or a ribonucleic acid sequence to be processed.
[0083] In operation S402: a preset word segmenter is used to perform word segmentation on the biological sequence to be processed to obtain a biological sequence to be input.
[0084] In operation S403, the biological sequence to be input is input into the biological model, and when it is determined that the biological sequence to be input includes a preset prompt word, a predicted sequence corresponding to the biological sequence to be input is generated, wherein the sequence category of the predicted sequence is different from the sequence category of the biological sequence to be processed.
[0085] According to an embodiment of the present invention, a biological sequence is processed using a trained biological model, and the biological sequence to be processed includes a RNA sequence and a protein sequence. Before the biological sequence to be processed is input into the trained biological model, a word segmenter is used to perform word segmentation on the sequence, that is, a preset word segmentation table is used to convert the letters of the sequence into a language that can be understood by a computer.
[0086] According to an embodiment of the present invention, before the word segmentation process, the process also includes trimming the overlong biological sequence to be processed, and the trimming method is the same as the biological model training method.
[0087] According to the embodiments of the present invention, understanding the correspondence between protein sequences and RNA sequences helps to reveal the protein synthesis process, as well as the specific functions and mechanisms of protein in organisms, which is crucial to understanding the nature and laws of life activities; secondly, accurately predicting the correspondence between protein sequences and RNA sequences is of great significance for disease diagnosis and treatment. The occurrence and development of many diseases are related to the abnormality of protein function, and the abnormality of protein function is often closely related to the changes in its RNA sequence. By predicting the correspondence between protein sequences and RNA sequences, the molecular mechanism of disease occurrence can be revealed, providing new ideas and methods for early diagnosis and treatment of diseases; in addition, predicting the correspondence between protein sequences and RNA sequences also has important applications in drug development. The mechanism of action of drugs often involves binding and regulation with specific proteins, and the function of proteins is closely related to their RNA sequences. Therefore, by predicting the correspondence between protein sequences and RNA sequences, the mechanism of action of drugs can be better understood, the efficiency and success rate of drug development can be improved, and more possibilities are provided for biological research and application.
[0088] According to an embodiment of the present invention, in the case where it is determined that the biological sequence to be input includes a preset prompt word, generating a predicted sequence corresponding to the biological sequence to be input includes:
[0089] When the biological sequence to be input is a protein sequence to be processed and the protein sequence to be processed includes a preset prompt word, generating a predicted messenger RNA coding sequence corresponding to the protein sequence to be processed;
[0090] When the biological sequence to be input is a ribonucleic acid sequence to be processed and the ribonucleic acid sequence to be processed includes a preset prompt word, a protein prediction sequence corresponding to the ribonucleic acid sequence to be processed is generated.
[0091] According to some embodiments of the present invention, when preset prompt words are included, the biological model can output the corresponding predicted biological sequence and perform a two-way understanding task to generate a high-dimensional matrix, thereby achieving both generation and two-way understanding tasks.
[0092] According to an embodiment of the present invention, when the input is '<|start|>'+'protein sequence'+'<|end|>'+'<|prompt|>' + '<|end|>', because the input contains the preset prompt word '<|prompt|>', the model performs the generation task, i.e., outputs the corresponding predicted ribonucleic acid sequence; when the input is '<|start|>'+'ribonucleic acid'+'<|end|>'+'<|prompt|>' + '<|end|>', because the input contains the preset prompt word '<|prompt|>', the model performs the generation task, i.e., outputs the corresponding predicted protein sequence.
[0093] According to an embodiment of the present invention, the method further includes: generating a high-dimensional feature matrix representing the biological sequence to be input when it is determined that the biological sequence to be input does not include a preset prompt word.
[0094] According to an embodiment of the present invention, when the input is '<|start|>'+'protein sequence'+'<|end|>'+'<|end|>', since the input does not contain the preset prompt word, the model performs the understanding task, that is, outputs the high-dimensional feature matrix of the corresponding predicted RNA sequence; when the input is '<|start|>'+'ribonucleic acid'+'<|end|>' + '<|end|>', since the input does not contain the preset prompt word, the model performs the understanding task, that is, outputs the high-dimensional feature matrix of the corresponding predicted RNA sequence.
[0095] According to an embodiment of the present invention, the high-dimensional feature matrix of the biological sequence can be used for downstream classification tasks or regression tasks; the pre-trained model can capture the high-dimensional correlation between the amino acid sequence and the RNA sequence, and can generate a sequence of another language modality from a known language modality through autoregression.
[0096] According to an embodiment of the present invention, a trained biological model is used in the method for processing a biological sequence.
[0097] Figure 5 A flow chart of a biological sequence processing method according to another embodiment of the present invention is shown.
[0098] like Figure 5 As shown, the biological sequence processing method of this embodiment includes operations S501 to S507.
[0099] Operation S501: obtaining a biological sequence to be processed, wherein the biological sequence to be processed includes a protein sequence to be processed or a ribonucleic acid sequence to be processed.
[0100] Operation S502: Use a preset word segmenter to perform word segmentation on the biological sequence to be processed to obtain a biological sequence to be input.
[0101] Operation S503: Input the biological sequence to be input into the biological model, and determine whether the biological sequence to be processed contains the preset prompt word. If it is determined that the biological sequence to be processed contains the preset prompt word, perform operation S504; if it is determined that the biological sequence to be processed does not contain the preset prompt word, perform operation S507.
[0102] Operation S504: Determine whether the biological sequence to be processed containing the preset word is a protein sequence. If it is determined that the biological sequence to be processed is a protein sequence, perform operation S505; if it is determined that the biological sequence to be processed is not a protein sequence but a ribonucleic acid sequence, perform operation S506.
[0103] Operation S505: Generate a predicted messenger RNA coding sequence corresponding to the protein sequence to be processed.
[0104] Operation S506: Generate a predicted protein sequence corresponding to the RNA sequence to be processed.
[0105] Operation S507: Generate a high-dimensional feature matrix representing the biological sequence to be input.
[0106] Figure 6 A structural block diagram of a biological model training device according to an embodiment of the present invention is shown.
[0107] like Figure 6 As shown, the biological model training device 600 of this embodiment includes a construction module 610, a word segmentation module 620, an input module 630, and a training execution module 640.
[0108] The construction module 610 is used to construct a protein-ribonucleic acid sequence pair according to a preset input format based on the protein sequence data set and the ribonucleic acid sequence data set, wherein the protein-ribonucleic acid sequence pair includes a preset prompt word. In one embodiment, the construction module 610 can be used to perform the operation S101 described above, which will not be repeated here.
[0109] The word segmentation module 620 is used to perform word segmentation processing on the protein sequence data set and the RNA sequence pair using a preset word segmenter to obtain a training sequence pair. In one embodiment, the word segmentation module 620 can be used to perform the operation S102 described above, which will not be repeated here.
[0110] The input module 630 is used to input the training sequence pair into the biological model to obtain the predicted sequence pair corresponding to the training sequence pair. In one embodiment, the input module 630 can be used to perform the operation S103 described above, which will not be described in detail here.
[0111] The training execution module 640 is used to train the biological model according to the prediction sequence pair, the training sequence pair and the preset prompt word to obtain the trained biological model. In one embodiment, the training execution module 640 can be used to perform the operation S104 described above, which will not be repeated here.
[0112] According to an embodiment of the present invention, any multiple modules of the construction module 610, the word segmentation module 620, the input module 630 and the training execution module 640 can be combined into one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present invention, at least one of the construction module 610, the word segmentation module 620, the input module 630 and the training execution module 640 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware and firmware or in an appropriate combination of any of them. Alternatively, at least one of the construction module 610, the word segmentation module 620, the input module 630 and the training execution module 640 may be at least partially implemented as a computer program module, and when the computer program module is executed, the corresponding function may be executed.
[0113] Figure 7 A structural block diagram of a biological sequence processing device according to an embodiment of the present invention is shown.
[0114] like Figure 7 As shown, the biological sequence processing method and device 700 of this embodiment includes an acquisition module 710, a word segmentation module 720, and an execution module 730.
[0115] The acquisition module 710 acquires a biological sequence to be processed, wherein the biological sequence to be processed includes a protein sequence to be processed or a ribonucleic acid sequence to be processed. In one embodiment, the acquisition module 710 can be used to perform the operation S401 described above, which will not be described in detail here.
[0116] The word segmentation module 720 is used to perform word segmentation processing on the biological sequence to be processed using a preset word segmenter to obtain the biological sequence to be input. In one embodiment, the word segmentation module 720 can be used to perform the operation S402 described above, which will not be repeated here.
[0117] The execution module 730 is used to input the biological sequence to be input into the biological model, and when it is determined that the biological sequence to be input includes a preset prompt word, generate a predicted sequence corresponding to the biological sequence to be input, wherein the sequence category of the predicted sequence is different from the sequence category of the biological sequence to be processed. In one embodiment, the execution module 730 can be used to perform the operation S403 described above, which will not be repeated here.
[0118] According to an embodiment of the present invention, any multiple modules among the acquisition module 710, the word segmentation module 720 and the execution module 730 can be combined into one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present invention, at least one of the acquisition module 710, the word segmentation module 720 and the execution module 730 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware and firmware or in any appropriate combination of any of them. Alternatively, at least one of the acquisition module 710, the word segmentation module 720 and the execution module 730 can be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding function can be executed.
[0119] It will be appreciated by those skilled in the art that the features described in the various embodiments and / or claims of the present invention may be combined and / or combined in various ways, even if such combinations and / or combinations are not explicitly described in the present invention. In particular, the features described in the various embodiments and / or claims of the present invention may be combined and / or combined in various ways without departing from the spirit and teachings of the present invention. All of these combinations and / or combinations fall within the scope of the present invention.
[0120] The embodiments of the present invention are described above. However, these embodiments are only for the purpose of illustration, and are not intended to limit the scope of the present invention. Although each embodiment is described above, it does not mean that the measures in each embodiment cannot be used in combination. The scope of the present invention is defined by the attached claims and their equivalents. Without departing from the scope of the present invention, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present invention.< / mask> < / unkown> < / pad> < / mask>
Claims
1. A biological model training method, characterized in that: The method comprises: Constructing a protein-ribonucleic acid sequence pair according to a preset input format based on a protein sequence data set and a ribonucleic acid sequence data set, wherein the protein-ribonucleic acid sequence pair includes a preset prompt word; Using a preset word segmenter to perform word segmentation processing on the protein sequence data set and the RNA sequence pair to obtain a training sequence pair; Inputting the training sequence pair into a biological model to obtain a predicted sequence pair corresponding to the training sequence pair; and Training the biological model according to the prediction sequence pair, the training sequence pair and the preset prompt word to obtain a trained biological model; The biological model includes an attention matrix, which is composed of a first submatrix, a second submatrix, a third submatrix and a fourth submatrix, wherein the first submatrix is located in the first quadrant, the second submatrix is located in the second quadrant, the third submatrix is located in the third quadrant and the fourth submatrix is located in the fourth quadrant; The method for constructing the attention matrix includes: Using the prediction sequence in the first half of the prediction sequence pair and the training sequence in the first half of the training sequence pair, constructing the second sub-matrix in the biological model, wherein the second sub-matrix is an all-zero matrix; The fourth submatrix in the biological model is trained using the prediction sequence in the second half of the prediction sequence pair and the training sequence in the second half of the training sequence pair, wherein the upper triangular area of the fourth submatrix is a non-zero matrix and the lower triangular area is a zero matrix.
2. The method according to claim 1, characterized in that The step of inputting the training sequence pair into the biological model to obtain a predicted sequence pair corresponding to the training sequence pair comprises: Masking a preset proportion of words in the training sequence pair to obtain a masked training sequence pair; The masked training sequence pair is input into the biological model to obtain the predicted sequence pair.
3. The method according to claim 2, characterized in that The masking process is performed on the words of the preset proportion in the training sequence pair to obtain the masked training sequence pair, which includes: Selecting a preset ratio of words from the training sequence pair to obtain words to be masked; Selecting a first preset sub-proportion of words in the words to be masked and replacing them with preset characters; A second preset sub-proportion of words in the words to be masked is selected and replaced with random characters to obtain the masked training sequence pair.
4. The method according to claim 1, characterized in that: The step of training the biological model according to the prediction sequence pair, the training sequence pair and the preset prompt word to obtain the trained biological model comprises: The biological model is subjected to multi-task training according to the predicted sequence pair, the training sequence pair and the preset prompt words to obtain a trained biological model, wherein the multi-task includes a bidirectional understanding task for protein sequences, a bidirectional understanding task for RNA sequences, a unidirectional generation task for protein sequences and a unidirectional generation task for RNA sequences.
5. A method for processing a biological sequence, characterized in that: The method comprises: Acquiring a biological sequence to be processed, wherein the biological sequence to be processed includes a protein sequence to be processed or a ribonucleic acid sequence to be processed; Using a preset word segmenter to perform word segmentation on the biological sequence to be processed to obtain a biological sequence to be input; The biological sequence to be input is input into a biological model, and when it is determined that the biological sequence to be input includes a preset prompt word, a predicted sequence corresponding to the biological sequence to be input is generated, wherein the sequence category of the predicted sequence is different from the sequence category of the biological sequence to be processed, and the biological model is trained by the training method described in any one of claims 1 to 4.
6. The method according to claim 5, characterized in that In the case where it is determined that the biological sequence to be input includes a preset prompt word, generating a predicted sequence corresponding to the biological sequence to be input includes: In a case where the biological sequence to be input is a protein sequence to be processed and the protein sequence to be processed includes the preset prompt word, generating a predicted messenger RNA coding sequence corresponding to the protein sequence to be processed; In the case that the biological sequence to be input is a ribonucleic acid sequence to be processed and the ribonucleic acid sequence to be processed includes the preset prompt word, a protein prediction sequence corresponding to the ribonucleic acid sequence to be processed is generated.
7. The method according to claim 5, characterized in that The method further includes: generating a high-dimensional feature matrix representing the biological sequence to be input when it is determined that the biological sequence to be input does not include the preset prompt word.