Model training method, biological sequence task processing method and related device

Abstract

The invention provides a model training method, a biological sequence class task processing method and a related device, and the model training method comprises the steps: obtaining a first data set, and training a first neural network based on the first data set, thereby obtaining a biological sequence base model. Wherein the first data set comprises biological sequences of multiple sequence types. The biological sequences in the first data set have corresponding first sequence type markers and first sequence element markers, the first sequence type markers are used for indicating sequence types to which the corresponding biological sequences belong, and the first sequence element markers are used for indicating fragment types to which constituent fragments in the corresponding biological sequences belong. The biological sequence base model is used for constructing a sequence task model for executing a plurality of biological sequence tasks. According to the scheme, the biological sequence base model with good universality can be obtained, and the method can adapt to various different biological sequence tasks.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence (AI) technology, and in particular to a model training method, a method for processing biological sequence tasks, and related devices. Background Art

[0002] Biological sequences, such as genomes and ribonucleic acid (RNA), are important tools in the fields of bioinformatics, modern biological sciences, and biomedicine. There are many research directions based on biological sequences, including genome sequencing, RNA sequence analysis, RNA / protein structure prediction, RNA function annotation, RNA vaccine and nucleic acid drug design, etc.

[0003] At present, researchers have used AI models to assist in biological sequence-related research and design in order to reduce the complexity and time cost of the research and design process. However, there is a strong coupling between the current AI models and biological sequence tasks. Different AI models need to be trained separately for different biological sequence tasks, and the practicality and versatility of the models are poor. This is obviously not flexible and efficient enough for the diverse R&D needs and ever-changing drug design challenges in the field of biological sequences. Summary of the invention

[0004] The present application provides a model training method, a method for processing biological sequence tasks and related devices, which can obtain a biological sequence-based model with good versatility to adapt to a variety of different biological sequence tasks.

[0005] In a first aspect, the present application provides a model training method, the method comprising: obtaining a first data set, and then training a first neural network based on the first data set to obtain a biological sequence base model. The first data set includes biological sequences of multiple sequence types. The biological sequences in the first data set have corresponding first sequence type tags and first sequence element tags, the first sequence type tags are used to indicate the sequence type to which the corresponding biological sequence belongs, and the first sequence element tags are used to indicate the fragment type to which the constituent fragments in the corresponding biological sequence belong. The biological sequence base model is used to construct a sequence task model that performs multiple biological sequence tasks. The input data of the sequence task model has a second sequence type tag and / or a second sequence element tag, the second sequence type tag is used to indicate the sequence type to which the input data belongs in the above-mentioned multiple sequence types, and the second sequence element tag is used to indicate the fragment type to which the constituent fragments in the input data belong, and the above-mentioned multiple sequence types of biological sequences and the input data have constituent fragments of the same fragment type.

[0006] The biological sequence base model in this solution is obtained by training the first data set. Since the first data set contains multiple sequence types, and the biological sequences in the first data set have corresponding first sequence type tags and / or first sequence element tags, the biological sequence base model can learn the knowledge (semantics) of multiple sequence types themselves and the differences and connections between different sequence types, so that it can support multiple biological sequence tasks (downstream tasks), that is, based on this biological sequence base model, a sequence task model for performing multiple biological sequence tasks can be constructed, and the biological sequence base model has strong versatility.

[0007] Based on the first aspect, in a possible implementation scheme, the above-mentioned multiple sequence types include at least one of deoxyribonucleic acid (DNA) type, ribonucleic acid (RNA) type, and protein type, and the above-mentioned fragment type includes at least one of 5' untranslated region type, 3' untranslated region type, and coding region type. It should be understood that this scheme takes into account the central dogma of biology (DNA generates RNA through transcription, and RNA generates protein through translation), and thus collects biological sequences spanning multiple sequence types of DNA-RNA-Protein for model training, so that the biological sequence-based model learns the intrinsic distribution of genetic rules from the above-mentioned multiple sequence types, thereby having the ability to support multiple biological sequence tasks, and can provide technical support for the diversified research and development needs and changing drug design needs in the field of biological sequences.

[0008] Based on the first aspect, in a possible implementation, the above-mentioned biological sequence-based model includes an encoder and a decoder. It should be understood that since the biological sequence-based model adopts an encoder-decoder structure, the biological sequence-based model has the ability to generate biological sequences, thereby being able to support various generative biological sequence tasks, such as generating RNA sequences, generating coding regions, generating untranslated regions, and other generative tasks.

[0009] Based on the first aspect, in a possible implementation scheme, the above-mentioned multiple biological sequence tasks include discriminative tasks and / or generative tasks, the discriminative tasks include one or more of sequence expression tasks, sequence translation efficiency prediction tasks, sequence structure prediction tasks, and sequence functional attribute prediction tasks, and the generative tasks include sequence generation tasks. In other words, the biological sequence base model in this scheme is highly versatile, that is, it can support various discriminative biological sequence tasks (i.e., discriminative tasks, including classification tasks and / or regression tasks), and can also support various generative biological sequence tasks (i.e., generative tasks).

[0010] Based on the first aspect, in a possible implementation scheme, after the biological sequence base model is trained, the biological sequence base model and the model heads corresponding to multiple biological sequence tasks are trained based on the second data set to obtain a sequence task model. The second data set corresponds to the above-mentioned multiple biological sequence tasks. The sequence task model also corresponds to the above-mentioned multiple biological sequence tasks.

[0011] Optionally, the sequence task model can be one or more models. In the case where the sequence task model is one model, this one sequence task model is used to perform the above-mentioned multiple biological sequence tasks. In the case where the sequence task model is multiple models, each of these multiple models is used to perform part of the above-mentioned multiple biological sequence tasks. For example, multiple models are all constructed based on the biological sequence base model, that is, on the basis of the biological sequence base model, model heads corresponding to multiple biological sequence tasks are added, and then the model parameters are fine-tuned. These multiple models correspond to multiple biological sequence tasks one by one, and each model is only used to perform a corresponding biological sequence task, or each model is used to perform at least two biological sequence tasks among the above-mentioned multiple biological sequence tasks.

[0012] Based on the first aspect, in a possible implementation, when the biological sequence base model includes a decoder and the plurality of biological sequence tasks include discriminative tasks, the model head corresponding to the plurality of biological sequence tasks includes a multi-layer perceptron, which corresponds to the discriminative task. It should be understood that the decoder architecture is not suitable for discriminative tasks, so a small network such as a multi-layer perceptron is added to the model head here so that the sequence task model can adapt to the discriminative task.

[0013] Based on the first aspect, in a possible implementation scheme, after training to obtain a biological sequence base model, the biological sequence base model is trained based on multiple data sets to obtain a sequence task model. Among them, the above-mentioned multiple biological sequence tasks include discriminative tasks and generative tasks, the above-mentioned multiple data sets correspond to multiple biological sequence tasks one by one, and the biological sequence in each data set in the multiple data sets has a task tag, and the task tag is used to indicate the task corresponding to the corresponding biological sequence in the above-mentioned multiple biological sequence tasks and the natural language vocabulary associated with the task. It should be understood that for the data sets corresponding to various discriminative tasks and generative tasks, this scheme adds corresponding task tags to the data (i.e., biological sequences) in these data sets, so that these tasks can be unified into generative tasks, so that the biological sequence base model can be directly used as a sequence task model, avoiding the modification of the architecture of the biological sequence base model, increasing the versatility of the model, and the sequence task model can output the results of various discriminative tasks and generative tasks using a natural language generative method.

[0014] Based on the first aspect, in a possible implementation scheme, after obtaining the biological sequence base model, the training data is input into the biological sequence base model to obtain a sequence generation result, and then the biological sequence base model is updated according to the sequence generation result and the target output data corresponding to the training data, to obtain a first task model for performing a first biological sequence task, and the first task model belongs to the above-mentioned sequence task model. Wherein, the above-mentioned multiple biological sequence tasks include a first biological sequence task, the training data corresponds to the first biological sequence task, the training data has corresponding target output data, the training data and the target output data belong to different sequence types among the above-mentioned multiple sequence types, or the constituent segments in the training data and the constituent segments in the target output data belong to different segment types.

[0015] That is to say, in order to obtain a model capable of generating a biological sequence of a corresponding second sequence type according to a biological sequence of a first sequence type (the first sequence type and the second sequence type are two different sequence types among the above-mentioned multiple sequence types), the biological sequence base model can be fine-tuned using the training data (belonging to the first sequence type) and the target output data corresponding to the sequence data (belonging to the second sequence type) on the basis of the previously trained biological sequence base model, thereby obtaining a first task model, which has the ability to output a biological sequence of a corresponding second sequence type according to the biological sequence of the first sequence type, that is, the ability to generate sequences across sequence types. Similarly, in order to obtain a model capable of generating data of a corresponding second fragment type according to data of a first fragment type (the first fragment type and the second fragment type belong to different fragment types), the biological sequence base model can be fine-tuned using the training data (belonging to the first fragment type) and the target output data corresponding to the training data (outputting the second fragment type) on the basis of the previously trained biological sequence base model, thereby obtaining a first task model, which has the ability to output data of a corresponding second fragment type according to data of a first fragment type, that is, the ability to generate sequences across fragment types.

[0016] In a second aspect, the present application also provides a method for processing biological sequence tasks, the method comprising: obtaining input data, and then inputting the input data into a first task model, thereby obtaining output data corresponding to the input data. Wherein, the input data corresponds to the first biological sequence task. The input data has a second sequence type tag and / or a second sequence element tag, the second sequence type tag is used to indicate the sequence type to which the input data belongs among multiple sequence types, and the second sequence element tag is used to indicate the fragment type to which the constituent fragments in the input data belong. The first task model is used to perform the first biological sequence task, and the first task model is constructed based on the biological sequence base model. The biological sequence base model is obtained by training based on a first data set, and the first data set includes biological sequences of multiple sequence types. The biological sequences in the first data set have corresponding first sequence type tags and / or first sequence element tags, the first sequence type tags are used to indicate the sequence type to which the corresponding biological sequence belongs, and the first sequence element tags are used to indicate the fragment type to which the constituent fragments in the corresponding biological sequence belong. The input data and the biological sequences of the above-mentioned multiple sequence types have constituent fragments of the same fragment type.

[0017] Based on the second aspect, in a possible implementation, the multiple sequence types include at least one of a deoxyribonucleic acid type, a ribonucleic acid type, and a protein type, and the fragment type includes at least one of a 5' untranslated region type, a 3' untranslated region type, and a coding region type. Based on the second aspect, in a possible implementation, the biological sequence-based model includes an encoder and a decoder.

[0018] Based on the second aspect, in a possible implementation scheme, the above-mentioned first biological sequence type task includes discriminative tasks and / or generative tasks, the discriminative tasks include one or more of sequence expression tasks, sequence translation efficiency prediction tasks, sequence structure prediction tasks, and sequence functional attribute prediction tasks, and the generative tasks include sequence generation tasks.

[0019] Based on the second aspect, in a possible implementation scheme, the above-mentioned first task model is used to execute multiple biological sequence tasks, and these multiple biological sequence tasks include a first biological sequence task. The first task model includes a biological sequence base model and model heads corresponding to these multiple biological sequence tasks (used to output the results corresponding to these multiple biological sequence tasks).

[0020] Based on the second aspect, in a possible implementation scheme, when the biological sequence base model includes a decoder and the above-mentioned multiple biological sequence tasks include discriminative tasks, the model heads corresponding to the above-mentioned multiple biological sequence tasks include a multi-layer perceptron, and the multi-layer perceptron corresponds to the discriminative tasks.

[0021] Based on the second aspect, in a possible implementation scheme, the first task model is used to perform multiple biological sequence tasks, which include a first biological sequence task, and the multiple biological sequence tasks include discriminative tasks and generative tasks. The input data has a task tag, and the task tag is used to indicate the first biological sequence task. The above-mentioned output data includes natural language vocabulary corresponding to the first biological sequence task.

[0022] Based on the second aspect, in a possible implementation scheme, when the first biological sequence type task is a generative task, the above-mentioned output data and input data belong to different sequence types among multiple sequence types, or, the constituent fragments in the output data and the constituent fragments in the input data belong to different fragment types (but may be located in biological sequences of the same sequence type).

[0023] In a third aspect, the present application also provides a model training device, including an acquisition module and a training module. The acquisition module is used to acquire a first data set, wherein the first data set includes biological sequences of multiple sequence types, and the biological sequences in the first data set have corresponding first sequence type tags and first sequence element tags, the first sequence type tags are used to indicate the sequence type to which the corresponding biological sequence belongs, and the first sequence element tags are used to indicate the fragment type to which the constituent fragments in the corresponding biological sequence belong. The training module is used to train the first neural network based on the first data set to obtain a biological sequence base model, wherein the biological sequence base model is used to construct a sequence task model that performs multiple biological sequence tasks, and the input data of the sequence task model has a second sequence type tag and / or a second sequence element tag, the second sequence type tag is used to indicate the sequence type to which the input data belongs in multiple sequence types, the second sequence element tag is used to indicate the fragment type to which the constituent fragments in the input data belong, and the biological sequences of multiple sequence types and the input data have constituent fragments of the same fragment type.

[0024] The above-mentioned model training device may also include more or fewer units / modules, which are not specifically limited here. The model training device in the third aspect is specifically used to execute the method of any implementation scheme in the first aspect, which can be referred to the above introduction and will not be repeated here.

[0025] In a fourth aspect, the present application also provides a processing device for biological sequence tasks, including an acquisition module and a processing module. The acquisition module is used for the acquisition module, which is used to acquire input data, wherein the input data corresponds to a first biological sequence task, the input data has a second sequence type tag and / or a second sequence element tag, the second sequence type tag is used to indicate the sequence type to which the input data belongs among multiple sequence types, and the second sequence element tag is used to indicate the fragment type to which the component fragment in the input data belongs. The processing module is used to input the input data into a first task model to obtain output data corresponding to the input data, wherein the first task model is used to perform a first biological sequence task, the first task model is constructed based on a biological sequence base model, the biological sequence base model is obtained by training based on a first data set, the first data set includes biological sequences of multiple sequence types, the biological sequences in the first data set have corresponding first sequence type tags and / or first sequence element tags, the first sequence type tag is used to indicate the sequence type to which the corresponding biological sequence belongs, the first sequence element tag is used to indicate the fragment type to which the component fragment in the corresponding biological sequence belongs, and the input data and biological sequences of multiple sequence types have component fragments of the same fragment type.

[0026] The above-mentioned biological sequence task processing device may also include more or fewer units / modules, which are not specifically limited here. The biological sequence task processing device in the fourth aspect is specifically used to execute the method of any implementation scheme in the second aspect, which can be referred to the above introduction and will not be repeated here.

[0027] In a fifth aspect, the present application further provides a computing device cluster, comprising at least one computing device, each computing device comprising a processor and a memory. The processor of the at least one computing device is used to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the method of any implementation scheme in the first aspect or the second aspect.

[0028] In a sixth aspect, the present application also provides a computer-readable storage medium comprising computer program instructions. When the above-mentioned computer program instructions are executed by a computing device cluster (including at least one computing device), the computing device cluster executes a method as in any implementation scheme in the first aspect or the second aspect.

[0029] In a seventh aspect, the present application further provides a computer program product comprising instructions. When the instructions are executed by a computing device cluster (including at least one computing device), the computing device cluster executes the method of any one of the embodiments of the first aspect or the second aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for describing the embodiments.

[0031] Figure 1 It is a flowchart of a model training method provided in an embodiment of the present application;

[0032] Figure 2 is a schematic diagram of adding a first sequence type tag to a biological sequence provided by an embodiment of the present application;

[0033] Figure 3 It is a schematic diagram of adding a first sequence element label to a constituent fragment of a biological sequence provided in an embodiment of the present application;

[0034] Figure 4 It is a schematic diagram of adding a first sequence type label and a first sequence element label to a biological sequence provided by an embodiment of the present application;

[0035] Figure 5 It is a schematic diagram of building a sequence task model based on a biological sequence base model provided in an embodiment of the present application;

[0036] Figure 6 is another schematic diagram of building a sequence task model based on a biological sequence base model provided in an embodiment of the present application;

[0037] Figure 7 It is a schematic diagram of adding a multi-task model head based on a Transformer network provided in an embodiment of the present application;

[0038] Figure 8 is another schematic diagram of building a sequence task model based on a biological sequence base model provided in an embodiment of the present application;

[0039] Fig. 9 is a schematic diagram of outputting the results of multiple biological sequence tasks in natural language provided by an embodiment of the present application;

[0040] Fig.10 It is a flowchart of a method for processing biological sequence tasks provided in an embodiment of the present application;

[0041] Fig.11 This is a schematic diagram of a design of a 3UTR sequence generation task provided in an embodiment of the present application;

[0042] Fig.12 This is a schematic diagram of a design of a 5UTR sequence generation task provided in an embodiment of the present application;

[0043] Fig.13 It is a structural schematic diagram of a model training device provided in an embodiment of the present application;

[0044] Fig.14It is a structural schematic diagram of a biological sequence task processing device provided in an embodiment of the present application;

[0045] Fig.15 is a schematic diagram of the structure of a computing device provided in an embodiment of the present application;

[0046] Fig.16 is a schematic diagram of a computing device cluster provided in an embodiment of the present application;

[0047] Fig.17 It is a schematic diagram of a scenario in which two computing devices interact with each other according to an embodiment of the present application. DETAILED DESCRIPTION

[0048] In order to facilitate understanding of the embodiments of the present application, some terms and concepts involved in the embodiments of the present application are briefly introduced below.

[0049] 1. Token: In the large language model (LLM), it refers to the smallest unit (smallest semantic unit) in the text. Usually a token can be a word, a punctuation mark, a number, a symbol, etc.

[0050] 2. Prompt: A way to start a model. It is a piece of text or sentence that guides the machine learning model to generate output of a specific type, topic or format. In the field of natural language processing, a prompt usually consists of a question or task description.

[0051] 3. Genome: In biology, it refers to a complete set of genetic material in a cell or virus particle of a particular species (containing all the genetic information of an organism). The genome includes genes and non-coding regions. More precisely, the genome of an organism refers to the complete deoxyribonucleic acid (DNA) sequence in a set of chromosomes.

[0052] 4. Ribonucleic Acid (RNA): A type of linear macromolecule composed of ribonucleotides that carries information about gene expression. RNA has three subtypes: messenger RNA (mRNA), transfer RNA (tRNA) and ribosomal RNA (rRNA). RNA in nature is usually single-stranded and is usually generated by transcription from DNA. The four most basic bases are adenine (A), uracil (U), guanine (G) and cytosine (C).

[0053] 5. Protein: A long chain composed of one or more amino acid residues.

[0054] 6. Coding Region / Coding DNA Sequence / Coding Sequence, CDS: The portion of DNA or RNA that is composed of exons and encodes proteins. The boundaries of this region start from the start codon near the 5' end and end at the stop codon near the 3' end. The coding region of mRNA is located between the 5' untranslated region and the 3' untranslated region, which is also part of the exons.

[0055] 7. Untranslated Region (UTR): A segment located at both ends of the coding sequence of the mRNA chain. If it is located at the 5' end, it is called the 5' untranslated region, and if it is located at the 3' end, it is called the 3' untranslated region.

[0056] The following describes an embodiment of the model training method provided by this application.

[0057] See also Figure 1 , Figure 1 It is a flowchart of a model training method provided in an embodiment of the present application, including steps S101 to S102.

[0058] S101. Acquire a first data set, wherein the first data set includes biological sequences of M sequence types, where M is a positive integer greater than 1, and the biological sequences in the first data set have corresponding first sequence type tags and first sequence element tags, wherein the first sequence type tag is used to indicate the sequence type to which the corresponding biological sequence belongs, and the first sequence element tag is used to indicate the fragment type to which the constituent fragments in the corresponding biological sequence belong.

[0059] The number of sequence types (i.e., M value) and the specific sequence types are not limited in the embodiments of the present application, and can be divided according to actual application requirements. For example, the above-mentioned M sequence types include at least one of deoxyribonucleic acid type, RNA type, and protein type, and the fragment type includes at least one of 5' untranslated region, 3' untranslated region, and coding region. Optionally, in addition to the biological sequences of the above-mentioned sequence types, the first data set can also include sequence data such as CDS sequence, 5UTR sequence, 3UTR sequence, etc.

[0060] In a possible implementation scheme, an unlabeled biological sequence dataset may be first obtained, where the biological sequence dataset includes biological sequences of M sequence types, but these biological sequences do not yet have corresponding first sequence type tags and first sequence element tags. Then, corresponding first sequence type tags and first sequence element tags are added to the biological sequences in the biological sequence dataset, thereby obtaining a first dataset.

[0061] For example, biological sequences of various sequence types without annotations can be collected from genome databases such as T2T, NCBI, and EMBL, such as DNA sequences, RNA sequences, and protein sequences, and annotation information corresponding to the above biological sequences can be obtained from the genome database. The annotation information describes the sequence type to which each biological sequence belongs and which component fragments are contained in each biological sequence. In addition, the annotation information may also include information such as gene location, function, and known mutations. Then, based on the annotation information, the biological sequences in the unannotated biological sequence data set are annotated (by manual and / or machine annotation), and the corresponding first sequence type tags and first sequence element tags are added to obtain the first annotated data set. It should be understood that by adding corresponding first sequence type tags and first sequence element tags to biological sequences of various types and levels such as unannotated DNA, RNA, and proteins, and then performing model training based on these labeled biological sequences (see step S102), the biological sequence-based model can learn the genetic information flow characteristics of the central dogma in biology (DNA generates RNA through transcription, and RNA generates protein through translation), so that it can adapt to more biological sequence tasks and support downstream tasks across different sequence types.

[0062] The following is an example of how to add the first sequence type mark and the first sequence element mark.

[0063] For example, Figure 2 Four examples of adding the first sequence type tag to biological sequences are given:

[0064] The protein sequence in Example 1 is a biological sequence belonging to the protein type. Protein_START and Protein_END are added to the beginning and end of the protein sequence respectively as the first sequence type marker, and the first sequence type marker is used to indicate the sequence type to which the protein sequence belongs;

[0065] The DNA sequence in Example 2 is a biological sequence belonging to the DNA type. DNA_START and DNA_END are added to both ends of the DNA sequence as the first sequence type marker, and the first sequence type marker is used to indicate the sequence type to which the DNA sequence belongs.

[0066] The protein sequence in Example 3 is a biological sequence belonging to the protein type. Protein_token is added to the head of the protein sequence as a first sequence type tag, and the first sequence type tag is used to indicate the sequence type to which the protein sequence belongs;

[0067] The DNA sequence in Example 4 is a biological sequence belonging to the DNA type. DNA_token is added to the end of the DNA sequence as a first sequence type tag, and the first sequence type tag is used to indicate the sequence type to which the DNA sequence belongs.

[0068] For example, Figure 3 Five examples of adding first sequence element tags to constituent fragments in biological sequences are given:

[0069] The 5UTR sequence in Example 5 is one of the constituent fragments of a certain RNA sequence. The 5UTR sequence belongs to the 5' untranslated region type. 5UTR_START and 5UTR_END are added to the first and last ends of the 5UTR sequence as the first sequence element markers, respectively. The first sequence element markers are used to indicate the fragment type to which the 5UTR sequence belongs.

[0070] The CDS sequence in Example 6 is one of the constituent fragments of a DNA sequence. The CDS sequence belongs to the coding region type. CDS_START and CDS_END are added to the beginning and the end of the CDS sequence respectively as the first sequence element markers, and the first sequence element markers are used to indicate the fragment type to which the CDS sequence belongs.

[0071] The 3UTR sequence in Example 7 is one of the constituent fragments of a certain RNA sequence. The 3UTR sequence belongs to the 3' untranslated region type. 3UTR_START and 3UTR_END are added to the first and last ends of the 3UTR sequence as the first sequence element markers, respectively. The first sequence element markers are used to indicate the fragment type to which the 3UTR sequence belongs.

[0072] The 3UTR sequence in Example 8 is one of the constituent fragments of a certain RNA sequence. The 3UTR sequence belongs to the 3' untranslated region type. 3UTR_token is added to the head end of the 3UTR sequence as the first sequence element marker, and the first sequence element marker is used to indicate the fragment type to which the 3UTR sequence belongs;

[0073] The 3UTR sequence in Example 9 is one of the constituent fragments of a certain RNA sequence. The 3UTR sequence belongs to the 3' untranslated region type. 3UTR_token is added to the tail end of the 3UTR sequence as the first sequence element marker, which is used to indicate the fragment type to which the 3UTR sequence belongs.

[0074] For example, Figure 4 An example of adding a first sequence type tag and a first sequence element tag to a biological sequence is given. Figure 4 The biological sequence in is a certain RNA sequence, and the RNA sequence includes a 5UTR sequence, a CDS sequence and a 3UTR sequence. First, as shown in step ①, RNA_START and RNA_END are added to the first and last ends of the RNA sequence as the first sequence type marker, and the first sequence type marker indicates that the RNA sequence belongs to the RNA type. Then, as shown in step ②, the corresponding first sequence element marker is added to each component fragment in the RNA sequence to indicate the fragment type to which each component fragment belongs: 5UTR_START and 5UTR_END are added to the first and last ends of the 5UTR sequence in the RNA sequence to indicate that the 5UTR sequence belongs to the 5' untranslated region type; CDS_START and CDS_END are added to the first and last ends of the CDS sequence in the RNA sequence to indicate that the CDS sequence belongs to the coding region type; 3UTR_START and 3UTR_END are added to the first and last ends of the 3UTR sequence in the RNA sequence to indicate that the 3UTR sequence belongs to the 3' untranslated region type.

[0075] It should be noted that Figure 2 to Figure 4 The tag names, addition positions, and addition quantities of various first sequence type tags and first sequence element tags appearing in the text are only examples and do not constitute specific limitations on the embodiments of the present application. In actual application scenarios, other tag names, other addition positions, and / or other addition quantities may also be used. Figure 4 The order of adding the first sequence type mark and the first sequence element mark is only an example. In actual application scenarios, the first sequence type mark can be added first and then the first sequence element mark, or the first sequence element mark can be added first and then the first sequence type mark, or the first sequence type mark and the first sequence element mark can be added at the same time.

[0076] S102. Train the first neural network based on the first data set to obtain a biological sequence base model, wherein the biological sequence base model is used to construct a sequence task model that performs N biological sequence tasks, N is a positive integer greater than 1, and the input data of the sequence task model has a second sequence type tag and / or a second sequence element tag, the second sequence type tag is used to indicate the sequence type to which the input data belongs among the M sequence types, the second sequence element tag is used to indicate the fragment type to which a constituent fragment in the input data belongs, and the biological sequences of the M sequence types and the input data have constituent fragments of the same fragment type.

[0077] It should be understood that since the first data set is used to train the biological sequence-based model, the first data set includes biological sequences of M sequence types, and the biological sequences of the M sequence types have constituent fragments of the same fragment type as the above-mentioned input data. Therefore, after training with the first data set, the biological sequence-based model has learned the relevant knowledge of the constituent fragments in the input data, and thus has the ability to process the input data.

[0078] Optionally, the above input data can be a biological sequence of a sequence type such as a DNA type, an RNA type or a protein type, each of which contains multiple component fragments; the input data can also be a CDS sequence, a 5UTR sequence, a 3UTR sequence, and it can be considered that these sequences only contain one component fragment. For example, a CDS sequence contains a component fragment which is itself, and the fragment type to which the component fragment belongs is the CDS type.

[0079] Regarding the type of the first neural network, the embodiment of the present application does not make specific restrictions. It should be understood that after pre-training the first neural network with the first data set, a trained first neural network is obtained, and the trained first neural network is the biological sequence-based model in the embodiment of the present application, so the biological sequence-based model is the same type as the first neural network. For example, the first neural network can be an artificial neural network of the type of a transformer network, a variational autoencoder (VAE), a long short-term memory (LSTM), etc., and can also be other large language models, which are not specifically limited in the embodiment of the present application.

[0080] Among them, the Transformer model is a deep learning model based on the self-attention mechanism, which is mainly used to process sequence data. The model includes an encoder and a decoder, forming an encoder-decoder architecture. The encoder is a stacked structure containing multiple identical layers, each of which has two sublayers: the first sublayer is the self-attention mechanism layer, and the model calculates the attention weight of each element in the input sequence to other elements to better capture the pattern in the sequence; the second sublayer is a feedforward neural network, which is a simple fully connected neural network used to further process the results of the self-attention layer. Each sublayer is followed by a residual connection and layer normalization, which helps the training and deep learning of the model. The decoder is also a stacked structure, containing multiple identical layers, but each layer has three sublayers: the first sublayer is the self-attention mechanism layer, which is similar to the self-attention mechanism layer in the encoder, but a mask is added to prevent attention to future positions, thereby ensuring the autoregressive nature of decoding; the second sublayer is the encoder-decoder attention layer, which focuses on the output of the encoder, allowing the decoder to use all the information in the input sequence; the third sublayer is a feedforward neural network, which is the same as the feedforward neural network in the encoder, used to further process information, and each sublayer of the decoder is also followed by residual connections and layer normalization. VAE is a generative model, and the model structure of VAE also includes an encoder and a decoder, which will not be introduced in detail here.

[0081] It should be understood that biological sequence tasks refer to tasks related to biological sequences, and the present application embodiment does not limit the number of biological sequence tasks (i.e., N value) and specific task types. Optionally, the above-mentioned N biological sequence tasks may include discriminative tasks and / or generative tasks.

[0082] Discriminant means that by learning the boundaries of different categories (different sequence types and / or fragment types) through the study of the data set, new data can be classified and / or regressed. The discriminant task includes classification tasks and / or regression tasks. The value range of the output result of the classification task is a discrete category (discrete value), while the value range of the output result of the regression task is a continuous value. For example, the discriminant task includes one or more of the sequence expression task, sequence translation efficiency prediction task, sequence structure prediction task, and sequence functional attribute prediction task. Among them, the sequence expression task requires the prediction of the expression level of the input biological sequence, such as in the field of mRNA vaccine research, the expression level of the mRNA vaccine (a kind of RNA sequence) is required to be predicted. The sequence translation efficiency prediction task requires the prediction of the translation efficiency of the input biological sequence, such as predicting the translation efficiency of a certain RNA sequence. The sequence structure prediction task requires the prediction of the structure of the input biological sequence (such as secondary structure, tertiary structure, etc.), such as predicting the secondary structure of a certain RNA sequence and predicting the tertiary structure of a certain protein sequence. The task of predicting sequence functional attributes requires predicting the function of the input biological sequence, such as predicting what function each position in a certain RNA sequence has.

[0083] Generative refers to the ability to generate new data through learning from data sets. Generative tasks include sequence generation tasks, which require the output of a new sequence that meets the requirements based on the input sequence. For example, sequence biology tasks include UTR sequence generation tasks, such as generating a corresponding 3UTR sequence based on an input CDS sequence, or generating a corresponding 5UTR sequence based on an input CDS sequence. For another example, sequence generation tasks include sequence translation tasks and / or sequence reverse translation tasks. The sequence translation task requires the translation of the input RNA sequence into the corresponding protein sequence, and the sequence reverse translation task requires the translation of the input protein sequence into the corresponding CDS sequence or RNA sequence (3UTR sequence + CDS sequence + 5UTR sequence).

[0084] Optionally, the first neural network includes an encoder and a decoder, and accordingly, the biological sequence-based model obtained by training the first data set also includes an encoder and a decoder. Since the biological sequence-based model includes a decoder, and the decoder has sequence generation capability, the biological sequence-based model can support generative tasks, and a sequence task model for performing generative tasks can be constructed based on the biological sequence-based model, thereby providing sequence design support for biological sequence researchers.

[0085] It should be noted that Figure 1 The model training method may further include the following step S103, that is, after the biological sequence base model is obtained through training in step S102, a sequence task model may be constructed through step S103.

[0086] S103. Training the biological sequence base model based on the second data set to obtain a sequence task model, wherein the second data set corresponds to N biological sequence tasks.

[0087] The data volume, training rounds, and data sources of the second data set are not specifically limited in the present embodiment. For example, data corresponding to various biological sequence tasks can be collected from national project libraries such as gene expression databases, single cell expression databases, and protein expression databases, and data corresponding to various biological sequence tasks can also be collected from independent research databases such as RNA secondary structure databases, thereby forming the second data set used in the present embodiment.

[0088] Optionally, the sequence task model used to perform N biological sequence tasks can be one model or multiple models. In the case where the sequence task model is one model, this one model is used to perform the above-mentioned N biological sequence tasks, that is, the sequence task model at this time is a multi-task model. In the case where the sequence task model includes multiple models, each of these multiple models is used to perform at least one of the above-mentioned N biological sequence tasks, and each of these multiple models can be a single-task model or a multi-task model, and the tasks corresponding to any two models in these multiple models may or may not overlap. For the convenience of description, the following text mainly describes the sequence task model as a multi-task model as an example, but it does not mean that it can only be done in this way.

[0089] In one embodiment, the biological sequence base model and the model heads (artificial neural network, the network structure is not specifically limited) corresponding to N biological sequence tasks are trained based on the second data set to obtain a sequence task model. The second data set corresponds to N biological sequence tasks. The model heads corresponding to the N biological sequence tasks are used to output the output results corresponding to the N sequence tasks.

[0090] For example, Figure 5As shown, N model heads are added on the basis of the biological sequence base model, and the N model heads correspond to N biological sequence tasks one by one, and each model head is used to output the output result of a corresponding biological sequence task. The second data set includes data corresponding to N biological sequence tasks, and the biological sequence base model + N model heads are trained based on the second data set, so as to obtain a sequence task model through training. For example, the data corresponding to biological sequence task 1 in the second data set is input into the biological sequence base model, and then the output of the biological sequence base model is injected into the model head corresponding to biological sequence task 1, and then the output result of biological sequence task 1 is output, and the biological sequence base model and / or the model head corresponding to biological sequence task 1 are updated based on the output result of the biological sequence task 1 (update model parameters). Similarly, the data corresponding to biological sequence task N in the second data set is input into the biological sequence base model, and then the output of the biological sequence base model is injected into the model head corresponding to biological sequence task N, and then the output result of biological sequence task N is output, and the biological sequence base model and / or the model head corresponding to biological sequence task N are updated based on the output result of the biological sequence task N.

[0091] For example, Figure 6 As shown, a model head is added on the basis of the biological sequence base model, and the model head corresponds to N biological sequence tasks, and is used to output the output result of each task in the N biological sequence tasks. The second data set includes data corresponding to the N biological sequence tasks, and the biological sequence base model + 1 model head is trained based on the second data set, that is, the model is fine-tuned for specific downstream tasks (N biological sequence tasks), so as to train and obtain a sequence task model. For example, the data corresponding to the biological sequence task 1 in the second data set is input into the biological sequence base model, and then the output of the biological sequence base model is injected into the model head, and then the output result of the biological sequence task 1 is output, and the biological sequence base model and / or the model head are updated based on the output result of the biological sequence task 1. Similarly, the data corresponding to the biological sequence task N in the second data set is input into the biological sequence base model, and then the output of the biological sequence base model is injected into the model head, and then the output result of the biological sequence task N is output, and the biological sequence base model and / or the model head are updated based on the output result of the biological sequence task N.

[0092] It should be noted that Figure 5 and Figure 6The method of adding a model head is only an example and does not constitute a specific limitation on the embodiments of the present application. In actual application scenarios, in addition to adding one model head (corresponding to N biological sequence tasks) or N model heads (one-to-one corresponding to N biological sequence tasks) to the biological sequence base model, 2 to N-1 model heads can also be added to the biological sequence base model, each model head corresponds to at least one biological sequence task, and at least one model head corresponds to two biological sequence tasks. One or more classification tasks can also correspond to one model head, and one or more regression tasks can correspond to another model head.

[0093] Optionally, when the biological sequence base model includes a decoder and the above-mentioned N biological sequence tasks include discriminative tasks, the model heads corresponding to the N biological sequence tasks include a multi-layer perceptron (MLP), which corresponds to the discriminative task. Among them, the multi-layer perceptron is a feedforward neural network, consisting of multiple layers, each layer consisting of many neurons (also called nodes), and the multi-layer perceptron abstracts the input data through multiple layers of nonlinear transformation. It should be understood that since the decoder architecture is not suitable for discriminative tasks, a network such as a multi-layer perceptron is added here so that the sequence task model can adapt to discriminative tasks.

[0094] For example, Figure 7 As shown, it is assumed that the biological sequence base model specifically adopts a Transformer network, which includes K layers of encoders and K layers of decoders, where K is a positive integer, and the embodiment of the present application does not specifically limit the value of K. After training with the first data set, Figure 7 The Transformer network in has learned the differences and connections between M types of sequences. At this point, the second data set corresponding to various downstream tasks (i.e., various biological sequence tasks) can be used to fine-tune the Transformer network to obtain a sequence task model suitable for various downstream tasks.

[0095] Assume that the N biological sequence tasks mentioned above include generative tasks and discriminative tasks. Generative tasks include sequence generation tasks, and discriminative tasks include classification tasks and regression tasks. Since the Transformer network has a decoder structure, it has sequence generation capabilities and can support sequence generation tasks. For sequence generation tasks, there is no need to change the structure of the Transformer network. The Transformer network is trained using the data corresponding to the sequence generation task. The Transformer network outputs the result corresponding to the sequence generation task, that is, Figure 7 The autoregression prediction results in .

[0096] In order to adapt to the discriminative tasks such as classification and regression tasks, Figure 7 As shown in the figure, N model heads corresponding to biological sequence tasks are added to the Transformer network (i.e. Figure 7 The multi-task model head in the Transformer network is used as the input of the multi-task model head. The output of the Transformer network decoder is injected into the fully connected layer, which corresponds to the MLP described above. The MLP is added to adapt to the discriminative task. The fully connected layer can directly output the prediction result of the regression task. The output of the fully connected layer is then processed by the flexible maximum transfer function to output the prediction result of the classification task. It should be noted that Figure 7 The structure of the multi-task model head in is only an example and does not constitute a specific limitation.

[0097] In another embodiment, there is no need to add a model header to the biological sequence-based model, and the biological sequence-based model is directly trained based on the second data set to obtain a sequence task model, wherein the second data set corresponds to N biological sequence-based tasks.

[0098] For example, Figure 8 As shown, after the biological sequence base model is obtained through pre-training with the first data set, there is no need to add a model header to the biological sequence base model, and the biological sequence base model is directly trained based on the second data set (i.e., fine-tuned on the downstream biological sequence tasks), so as to train and obtain the sequence task model. For example, the data corresponding to the biological sequence task 1 in the second data set is input into the biological sequence base model, the output result of the biological sequence task 1 is output, and the biological sequence base model is updated (the model parameters are updated) based on the output result of the biological sequence task 1. Similarly, the data corresponding to the biological sequence task N in the second data set is input into the biological sequence base model, the output result of the biological sequence task N is output, and the biological sequence base model is updated based on the output result of the biological sequence task N.

[0099] Optionally, after obtaining the biological sequence base model, the training data is input into the biological sequence base model to obtain a sequence generation result, and then the biological sequence base model is updated (model parameters are updated) according to the sequence generation result and the target output data corresponding to the training data, to obtain a first task model for performing the first biological sequence task, and the first task model belongs to the above-mentioned sequence task model. Wherein, the above-mentioned N biological sequence tasks include the first biological sequence task, the training data corresponds to the first biological sequence task, the training data has corresponding target output data, the training data and the target output data belong to different sequence types among the above-mentioned M sequence types, or the constituent segments in the training data and the constituent segments in the target output data belong to different segment types.

[0100] That is to say, in order to obtain a model capable of generating a biological sequence of a corresponding second sequence type according to a biological sequence of a first sequence type (the first sequence type and the second sequence type are two different sequence types among the above-mentioned multiple sequence types), the biological sequence base model can be fine-tuned using the training data (belonging to the first sequence type) and the target output data corresponding to the sequence data (belonging to the second sequence type) on the basis of the previously trained biological sequence base model, thereby obtaining a first task model, which has the ability to output a biological sequence of a corresponding second sequence type according to the biological sequence of the first sequence type, that is, the ability to generate sequences across sequence types. Similarly, in order to obtain a model capable of generating data of a corresponding second fragment type according to data of a first fragment type (the first fragment type and the second fragment type belong to different fragment types), the biological sequence base model can be fine-tuned using the training data (belonging to the first fragment type) and the target output data corresponding to the training data (outputting the second fragment type) on the basis of the previously trained biological sequence base model, thereby obtaining a first task model, which has the ability to output data of a corresponding second fragment type according to data of a first fragment type, that is, the ability to generate sequences across fragment types.

[0101] Optionally, the second data set includes N data sets, and the N data sets correspond to N biological sequence tasks one by one. The biological sequences in each data set in the N data sets have corresponding task tags, and the task tags are used to indicate the tasks corresponding to the corresponding biological sequences in the N biological sequence tasks and the natural language vocabulary corresponding to the tasks. The biological sequence base model is trained based on the above N data sets to obtain a sequence task model. It should be understood that for different data sets corresponding to generative tasks and discriminative tasks, by adding corresponding task tags to the data in these data sets, these tasks can be unified into generative tasks (which can avoid model architecture modifications and increase model versatility), so that the sequence task model can output the results corresponding to these tasks in the form of natural language.

[0102] For example, assume that the second data set includes data corresponding to several biological sequence tasks, including a sequence expression task, a sequence reverse translation task (a type of sequence generation task), a UTR sequence generation task (another type of sequence generation task), and a secondary structure prediction task, and the data in the second data set has corresponding task tags (not shown in the figure), and the task tags indicate the task corresponding to the data and the natural language vocabulary corresponding to the task, so that the sequence task model can use these natural language vocabulary to output the results of various biological sequence tasks in the form of natural language.

[0103] like Fig. 9As shown, a piece of data corresponding to the sequence expression task (required to predict the expression level of the biological sequence) in the second data set is: CCCAGATCTCTTCAGTTTTTATGCCTCATTCTGTGAAAATTGCTGTAGTCTCTTCCAGTT (a DNA sequence), and the data has a corresponding task tag (not shown in the figure), the task tag indicates that the data corresponds to the sequence expression task, and the task tag indicates the natural language vocabulary corresponding to the sequence expression task (can be set, the embodiment of the present application is not specifically limited). The data with the task tag is input into the sequence task model for training. The sequence task model can determine the biological sequence class task corresponding to the data and the natural language vocabulary corresponding to the task according to the task tag. Finally, the sequence task model outputs the sequence expression result corresponding to the data: [2.0, 5.0, 3.0, 10.0, 5.5]. It can be seen that the sequence expression result is a result output in natural language form, which is easy for users to understand. Similarly, other data corresponding to the sequence expression task (required to predict the translation efficiency of the biological sequence) in the second data set can also be used to train the sequence task model, which will not be introduced in detail here.

[0104] like Fig. 9 As shown, a piece of data corresponding to the sequence reverse translation task (required to reverse translate the protein sequence) in the second data set is: MASTSRDVIAGRGIHSKVKS (protein amino acid sequence), and the data has a corresponding task tag (not shown in the figure), and the task tag indicates the biological sequence task corresponding to the data and the natural language vocabulary corresponding to the task (can be set, and the embodiment of the present application is not specifically limited). The data with the task tag is input into the sequence task model for training. The sequence task model can determine the biological sequence task corresponding to the data and the natural language vocabulary corresponding to the task according to the task tag. Finally, the sequence task model outputs the reverse translation result corresponding to the data, that is, determines what kind of CDS sequence the protein sequence may be translated from: AUGGCUAGCACGUCCCGAGUGAUAGUUGCCGGCCGGAUGGCUCAUAGCAAAGUGAAAAGUUAGC (a CDS sequence).

[0105] like Fig. 9As shown, a piece of data corresponding to the UTR sequence generation task in the second data set is: MASTSRDVIAGRGIHSKVKS (a protein sequence), and the data has a corresponding task tag (not shown in the figure), and the task tag indicates the biological sequence task corresponding to the data and the natural language vocabulary corresponding to the task (which can be set, and the embodiment of the present application is not specifically limited). The data with the task tag is input into the sequence task model for training, and the sequence task model can determine the biological sequence task corresponding to the data and the natural language vocabulary corresponding to the task according to the task tag. Finally, the sequence task model outputs the sequence generation result corresponding to the data: [5UTR_START]GACGCCAACGCAGGCGTAGCGCTCCGGT[5UTR_END][CDS_START]AUGGCUAGCAC GUCCCGAGUGAUAGUUGCCGGCCGGAUGGCUCAUAGCAAAGUGAAAAGUUAGC[CDS_END][3UTR_START]CATCAGGAGACATACTTTTTTGGGGTATAATGAGATGTTCACAGTTAAATACAGA[3UTR_END]. It can be seen that the sequence generation result includes 5UTR sequence + CDS sequence + 3UTR sequence. Among them, Fig. 9 The 5UTR sequence in is GACGCCAACGCAGGCGTAGCGCTCCGGT, and [5UTR_END] and [5UTR_START] at both ends of the 5UTR sequence are sequence element markers, which are used to indicate that the fragment type to which the 5UTR sequence belongs is a 5UTR type (the marker name and addition position of the sequence type marker are only used as examples and do not constitute a specific limitation); Fig. 9 The CDS sequence in is AUGGCUAGCACGUCCCGAGUGAUAGUUGCCGGCCGGAUGGCUCAUAGCAAAGUGAAAAGUUAGC, and [CDS_START] and [CDS_END] at both ends of the CDS sequence are sequence element markers, which are used to indicate that the fragment type to which the CDS sequence belongs is a CDS type; Fig. 9 The 3UTR sequence in is CATCAGGAGACATACTTTTTTGGGGTATAATGAGATGTTCACAGTTAAATACAGA, and [3UTR_END] and 3UTR_START] at both ends of the 3UTR sequence are sequence element markers, which are used to indicate that the fragment type to which the 3UTR sequence belongs is the 3UTR type.

[0106] like Fig. 9As shown, a piece of data corresponding to the secondary structure prediction task in the second data set is: GCCTCATTCAGTCTCTTCCA (a DNA sequence), and the data has a corresponding task tag (not shown in the figure), and the task tag indicates the biological sequence task corresponding to the data and the natural language vocabulary corresponding to the task (can be set, and the embodiment of the present application does not make specific limitations, such as setting it in the model vocabulary table). The data with the task tag is input into the sequence task model for training. The sequence task model can determine the biological sequence task corresponding to the data and the natural language vocabulary corresponding to the task according to the task tag. Finally, the sequence task model outputs the secondary structure prediction result corresponding to the data as: (..((..)..). The small brackets here represent pairing, and the dot represents unpairing.

[0107] It should be noted that, in addition to using the second data set to train a biological sequence base model to obtain a model for performing N biological sequence tasks, the biological sequence base model can also be trained using the data corresponding to each biological sequence task to obtain multiple independent models. Each of the above multiple independent models is constructed based on the biological sequence base model, and each model is obtained after training with the data corresponding to a corresponding biological sequence task. Optionally, each of the above multiple independent models is used to perform at least one of the above N biological sequence tasks, and each model can be a single-task model or a multi-task model. The single-task model is used to perform one biological sequence task, and the multi-task model is used to perform at least two biological sequence tasks.

[0108] In summary, the biological sequence base model in this scheme is obtained by training the first data set. Since the first data set contains multiple sequence types, and the biological sequences in the first data set have corresponding first sequence type tags and / or first sequence element tags, the biological sequence base model can learn the knowledge of multiple sequence types themselves and the differences and connections between different sequence types, so that it can support multiple biological sequence tasks (downstream tasks), that is, based on this biological sequence base model, a sequence task model for performing multiple biological sequence tasks can be constructed. The biological sequence base model has strong versatility, especially it can support biological sequence tasks across different sequence types (with cross-sequence type design capabilities). It should also be noted that the deployment method of the biological sequence base model and the sequence task model is not specifically limited in the embodiment of the present application. For example, the biological sequence base model can be deployed on a cloud platform, and then the user can use the data corresponding to one or more biological sequence tasks on the cloud platform to fine-tune the biological sequence base model, so as to obtain a task model that meets the user's needs, and the task model is used to perform the above-mentioned one or more biological sequence tasks. Task model. For another example, the sequence task model can be deployed in the cloud, and then an application programming interface (API) can be provided to the user, through which the user can call the sequence task model to perform the corresponding biological sequence task. It should be understood that in addition to the online method, the user can also download the biological sequence base model and / or the sequence task model to the local computer, so that the user can freely fine-tune the model or deploy it locally.

[0109] Based on the above sequence task model, an embodiment of the method for processing biological sequence tasks provided by the present application is introduced below.

[0110] See also Fig.10 , Fig.10 It is a flowchart of a method for processing biological sequence tasks provided in an embodiment of the present application, including steps S1001 to S1002.

[0111] S1001. Obtain input data, wherein the input data corresponds to a first biological sequence type task, and the input data has a second sequence type tag and / or a second sequence element tag, the second sequence type tag is used to indicate the sequence type to which the input data belongs among M sequence types, and the second sequence element tag is used to indicate the fragment type to which a constituent fragment in the input data belongs.

[0112] For the above M sequence types, see Figure 1 The relevant introduction of step S101 is not repeated here.

[0113] S1002. Input the input data into the first task model to obtain output data corresponding to the input data, wherein the first task model is used to perform a first biological sequence task, and the first task model is constructed based on a biological sequence base model, and the biological sequence base model is obtained by training based on a first data set.

[0114] For the contents of the first dataset, see Figure 1 The relevant introduction in step S101 is not repeated here.

[0115] The specific type of the first biological sequence task is not limited in this embodiment of the present application.

[0116] Optionally, the first biological sequence type task includes a discriminative task and / or a generative task, the discriminative task includes one or more of a sequence expression task, a sequence translation efficiency prediction task, a sequence structure prediction task, and a sequence functional attribute prediction task, and the generative task includes a sequence generation task.

[0117] It should be understood that Figure 1 The embodiment provides a training method for a sequence task model for performing N biological sequence tasks. Optionally, the sequence task model may be a multi-task model for performing the above-mentioned N biological sequence tasks, in which case the first task model is the sequence task model; the sequence task model may also include multiple models, each model is used to perform at least one biological sequence task among the N biological sequence tasks, each model may be a single-task model or a multi-task model, in which case the first task model is a model corresponding to the first biological sequence task in the sequence task model, the first task model is used to perform the first biological sequence task, and the first biological sequence task may include at least one biological sequence task among the N biological sequence tasks.

[0118] For example, suppose the first biological sequence task is a sequence function attribute prediction task, which requires predicting whether the input biological sequence can cause an immune response. Therefore, this task belongs to a classification task, and the output value range is discrete value / discrete category. Assume that the input data is a certain RNA sequence, and this RNA sequence is input into the first task model for executing the first biological sequence task. The first task model includes the biological sequence base model and the model head corresponding to the first biological sequence task. The method of adding the model head can refer to Figure 5 Subsequently, the model head corresponding to the first biological sequence task outputs output data (discrete value) corresponding to the RNA sequence, and the output data indicates whether the RNA sequence can induce an immune response.

[0119] For another example, suppose that the first biological sequence task is a sequence reverse translation task (one of the sequence generation tasks), which requires the protein sequence to be reverse translated into a corresponding CDS sequence, so the task belongs to a generative task. Assume that the biological sequence base model part in the first task model used to perform the first biological sequence task includes a decoder structure, so the biological sequence base model has sequence generation capabilities, and there is no need to add a model header corresponding to the first biological sequence task in the first task model. Assuming that the input data is a protein sequence, the protein sequence is input into the first task model, and the first task model generates one or more different CDS sequences, each CDS representing a possible reverse translation result of the protein sequence. Regarding the number of output CDS sequences, the embodiment of the present application does not make specific restrictions, and can be set and adjusted according to actual application requirements.

[0120] Optionally, the first task model is used to perform multiple biological sequence tasks, that is, the first task model is a multi-task model. The multiple biological sequence tasks can be some or all of the N biological sequence tasks mentioned above. The multiple biological sequence tasks include the first biological sequence task. In this case, the first task model includes the biological sequence base model and the model headers corresponding to the multiple biological sequence tasks. For the specific method of adding a model header based on the biological sequence base model, please refer to Figure 1 The relevant introduction of step S103 is not repeated here.

[0121] Optionally, when the biological sequence base model includes a decoder and the plurality of biological sequence tasks include discriminative tasks, the model heads corresponding to the plurality of biological sequence tasks include a multilayer perceptron, which corresponds to the discriminative tasks. For more information about multilayer perceptrons, see Figure 1 The relevant introduction of step S103 is not repeated here.

[0122] Optionally, the first task model is used to perform multiple biological sequence tasks, that is, the first task model is a multi-task model. These multiple biological sequence tasks can be some or all of the N biological sequence tasks mentioned above. These multiple biological sequence tasks include the first biological sequence task. At this time, the input data of the first task model has a task tag, and the task tag is used to indicate the first biological sequence task. The corresponding output data includes the natural language vocabulary corresponding to the first biological sequence task. For other contents of the task tag, please refer to the relevant introduction in step S103, which will not be repeated here.

[0123] Optionally, when the first biological sequence task is a generative task, the input data and output data of the first task model belong to different sequence types among the M sequence types (i.e., cross-sequence types), or the constituent fragments in the output data and the constituent fragments in the input data belong to different fragment types (i.e., cross-fragment types).

[0124] For example, it is assumed that the first biological sequence task is a sequence generation task, which requires generating a corresponding full-length RNA sequence (including 5UTR, CDS and 3UTR) according to a given protein sequence. It is assumed that the biological sequence base model part in the first task model used to perform the first biological sequence task includes a decoder structure, so the biological sequence base model has the ability to generate sequences, and there is no need to add the model head corresponding to the first biological sequence task in the first task model. It is assumed that the input data is a protein sequence, and the sequence type to which the protein sequence belongs is the protein type. The protein sequence is input into the first task model, and the output data of the first task model is the RNA sequence corresponding to the protein sequence, and the sequence type to which the RNA sequence belongs is the RNA type. The RNA sequence includes a 5UTR sequence, a CDS sequence and a 3UTR sequence, and these three sequences are all constituent fragments in the RNA sequence, and each of these three sequences has a corresponding sequence element tag, and the sequence element tag is used to indicate the fragment type to which the constituent fragment in the RNA sequence belongs. For example, the sequence element mark corresponding to the above 5UTR sequence is used to indicate that the 5UTR sequence belongs to the 5UTR type, the sequence element mark corresponding to the above CDS sequence is used to indicate that the CDS sequence belongs to the CDS type, and the sequence element mark corresponding to the above 3UTR sequence is used to indicate that the 3UTR sequence belongs to the 3UTR type.

[0125] For another example, assuming that the first biological sequence task is a 3UTR sequence generation task, after the biological sequence base model is obtained by training based on the first data set in step S102, the biological sequence base model is executed Fig.11 The fine-tuning design shown is to use the data corresponding to the first biological sequence task to train the biological sequence base model (the training method can refer to Figure 1 ), thereby obtaining a first task model for performing the first biological sequence task. The data corresponding to the first biological sequence task has Fig.11 In the form shown, each data includes a CDS sequence and a corresponding 3UTR sequence, and the CDS sequence and 3UTR sequence in each data are respectively marked with a corresponding second sequence element: Fig.11 CDS_START and CDS_END are the second sequence element tags corresponding to the CDS sequence, which are used to indicate that the CDS sequence belongs to the CDS type. Fig.11The 3UTR_START in is the second sequence element marker corresponding to the 3UTR sequence, which is used to indicate that the 3UTR sequence belongs to the 3UTR type.

[0126] Then in the inference phase, design Fig.11 The inference prompt words in the form of CDS tag (i.e., the second sequence element tag for indicating the CDS type), CDS sequence and 3UTR tag (i.e., the second sequence element tag for indicating the 3UTR type). The inference prompt is used as input data and injected into the first task model trained previously, and the first task model generates the corresponding 3UTR sequence. Optionally, the 3UTR sequence output by the first task model has a second sequence element tag 3UTR_END, which is used to indicate that the 3UTR sequence belongs to the 3UTR type.

[0127] Regarding the number of 3UTR sequences output by the first task model, the embodiment of the present application does not make any specific limitation and can be set according to actual application requirements. Fig.11 An example is given, where the first task model is input Fig.11 The first task model generates 10 3UTR sequences, which are sorted according to their similarity with the target 3UTR sequence. Among them, the first 3UTR sequence has the highest similarity with the target 3UTR sequence, reaching 94.51%.

[0128] For another example, assuming that the first biological sequence task is a 5UTR sequence generation task, after obtaining a biological sequence base model based on the first data set in step S102, the biological sequence base model is executed Fig.12 The fine-tuning design shown is to use the data corresponding to the first biological sequence task to train the biological sequence base model (the training method can refer to Figure 1 ), thereby obtaining a first task model for performing the first biological sequence task. The data corresponding to the first biological sequence task has Fig.12 In the form shown, each data includes a CDS sequence and a corresponding 5UTR sequence, and the CDS sequence and 5UTR sequence in each data are respectively marked with a corresponding second sequence element: Fig.12 CDS_START and CDS_END are the second sequence element tags corresponding to the CDS sequence, which are used to indicate that the CDS sequence belongs to the CDS type. Fig.12 The 5UTR_START in is the second sequence element marker corresponding to the 5UTR sequence, which is used to indicate that the 5UTR sequence belongs to the 5UTR type.

[0129] Then in the inference phase, design Fig.12The reasoning prompt words in the form of CDS tag (i.e., the second sequence element tag for indicating the CDS type), CDS sequence and 5UTR tag (i.e., the second sequence element tag for indicating the 5UTR type). The reasoning prompt is used as input data and injected into the first task model trained previously, and the first task model generates the corresponding 5UTR sequence. Optionally, the 5UTR sequence output by the first task model has a second sequence element tag 5UTR_END, which is used to indicate that the 5UTR sequence belongs to the 5UTR type.

[0130] Regarding the number of 5UTR sequences output by the first task model, the embodiment of the present application does not make any specific limitation and can be set according to actual application requirements. Fig.12 An example is given, where the first task model is input Fig.12 The reasoning prompt word 2, then the first task model generates 8 5UTR sequences, and here these 8 5UTR sequences are sorted according to the similarity with the target 5UTR sequence. Among them, the first 5UTR sequence has the highest similarity with the target 5UTR sequence, reaching 95.3%. It can be seen that in this example, the CDS sequence is given, and the model generates the corresponding 5UTR sequence. In fact, the CDS sequence + 3UTR sequence can also be given, and the model generates the corresponding 5UTR sequence. The embodiment of this application is not specifically limited.

[0131] In summary, the first task model in the embodiment of the present application is constructed based on the biological sequence base model, and the biological sequence base model is obtained by training the first data set. Since the first data set contains multiple sequence types, and the biological sequences in the first data set have corresponding first sequence type tags and / or first sequence element tags, the biological sequence base model learns the knowledge of multiple sequence types themselves and the differences and connections between different sequence types, so that it can well support various biological sequence tasks (including the first biological sequence tasks), and then the first task model for performing the first biological sequence task can be constructed based on the biological sequence base model.

[0132] The following describes how to execute Figure 1 Model training device and method for executing Fig.10 A method for processing biological sequence tasks.

[0133] See also Fig.13 , Fig.13 It is a structural diagram of a model training device 1300 provided in an embodiment of the present application, including an acquisition module 1301 and a training module 1302.

[0134] The acquisition module 1301 is used to acquire a first data set. The first data set includes biological sequences of M sequence types, where M is a positive integer greater than 1. The biological sequences in the first data set have corresponding first sequence type tags and first sequence element tags. The first sequence type tag is used to indicate the sequence type to which the corresponding biological sequence belongs, and the first sequence element tag is used to indicate the fragment type to which the constituent fragments in the corresponding biological sequence belong. For other contents of the first data set, please refer to Figure 1 The relevant introduction in step S101 is not repeated here.

[0135] The training module 1302 is used to train the first neural network based on the first data set to obtain a biological sequence-based model. The biological sequence-based model is used to construct a sequence task model for performing N biological sequence-based tasks, where N is a positive integer greater than 1. For N biological sequence-based tasks, see Figure 1 The relevant introduction in step S102 is not repeated here. The input data of the sequence task model has a second sequence type tag and / or a second sequence element tag, the second sequence type tag is used to indicate the sequence type to which the input data belongs among the M sequence types, and the second sequence element tag is used to indicate the fragment type to which the component fragment in the input data belongs, and the biological sequence of the M sequence types and the input data have component fragments of the same fragment type.

[0136] Optionally, the M sequence types include at least one of a deoxyribonucleic acid type, a ribonucleic acid type, and a protein type, and the fragment type includes at least one of a 5' untranslated region type, a 3' untranslated region type, and a coding region type.

[0137] Optionally, the biological sequence based model includes an encoder and a decoder.

[0138] Optionally, the above-mentioned N biological sequence tasks include discriminative tasks and / or generative tasks, the discriminative tasks include one or more of sequence expression tasks, sequence translation efficiency prediction tasks, sequence structure prediction tasks, and sequence functional attribute prediction tasks, and the generative tasks include sequence generation tasks.

[0139] Optionally, after the training module 1302 obtains the biological sequence base model through training, the training module 1302 is further used to: train the biological sequence base model and the model heads corresponding to N biological sequence tasks based on the second data set to obtain the sequence task model. The second data set corresponds to the N biological sequence tasks.

[0140] Optionally, when the biological sequence base model includes a decoder and the N biological sequence class tasks include discriminative tasks, the model head corresponding to the N biological sequence class tasks includes a multi-layer perceptron, which corresponds to the discriminative task.

[0141] Optionally, after the training module 1302 obtains the biological sequence base model through training, the training module 1302 is further used to: train the biological sequence base model based on N data sets to obtain a sequence task model. The N data sets correspond to the N biological sequence class tasks one by one, and the biological sequence in each of the N data sets has a corresponding task tag, and the task tag is used to indicate the task corresponding to the corresponding biological sequence in the N biological sequence class tasks and the natural language vocabulary corresponding to the task.

[0142] Optionally, after obtaining the biological sequence base model, the training module 1302 is further used to: input the training data into the biological sequence base model to obtain a sequence generation result, and then update the biological sequence base model according to the sequence generation result and the target output data corresponding to the training data, to obtain a first task model for performing the first biological sequence task, and the first task model belongs to the above-mentioned sequence task model. The above-mentioned N biological sequence tasks include a first biological sequence task, the training data corresponds to the first biological sequence task, the training data has corresponding target output data, the training data and the target output data belong to different sequence types among the above-mentioned M sequence types, or the constituent segments in the training data and the constituent segments in the target output data belong to different segment types.

[0143] It should be noted that the acquisition module 1301 and the training module 1302 can be implemented by software, or can be implemented by hardware, or can be implemented by both software and hardware. Exemplarily, the implementation of the training module 1302 is described below by taking the training module 1302 as an example. Similarly, the implementation of the other modules mentioned above can refer to the implementation of the training module 1302.

[0144] As an example of a software functional unit, the training module 1302 may include code running on a computing instance. Among them, the computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Furthermore, the above-mentioned computing instance may be one or more. For example, the training module 1302 may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code can be distributed in the same region (region) or in different regions. Furthermore, the multiple hosts / virtual machines / containers used to run the code can be distributed in the same availability zone (AZ) or in different AZs, each AZ including one data center or multiple data centers with close geographical locations. Among them, usually a region can include multiple AZs.

[0145] Similarly, multiple hosts / virtual machines / containers used to run the code can be distributed in the same virtual private cloud (VPC) or in multiple VPCs. Usually, a VPC is set up in a region. For cross-region communication between two VPCs in the same region and between VPCs in different regions, a communication gateway needs to be set up in each VPC to achieve interconnection between VPCs through the communication gateway.

[0146] As an example of a hardware functional unit, the training module 1302 may include at least one computing device, such as a server, etc. Alternatively, the training module 1302 may also be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0147] The multiple computing devices included in the training module 1302 can be distributed in the same region or in different regions. The multiple computing devices included in the training module 1302 can be distributed in the same AZ or in different AZs. Similarly, the multiple computing devices included in the training module 1302 can be distributed in the same VPC or in multiple VPCs. The multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0148] In other embodiments, both the acquisition module 1301 and the training module 1302 can be used to execute Figure 1 Any steps of the method of the embodiment, the steps that each of the above modules is responsible for implementing can be specified as needed, and they are respectively implemented Figure 1 The different steps are used to realize all the functions of the model training device 1300.

[0149] It should also be noted that the above-mentioned model training device 1300 is used to perform Figure 1 For details, please refer to the method of the embodiment. Figure 2 The relevant description is not repeated here. Fig.13The model training device 1300 is only used as an example to illustrate the division of the above-mentioned functional modules. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the model training device 1300 can be divided into other different functional modules to complete all or part of the functions described above.

[0150] See also Fig.14 , Fig.14 It is a structural diagram of a biological sequence task processing device 1400 provided in an embodiment of the present application, including an acquisition module 1401 and a processing module 1402.

[0151] The acquisition module 1401 is used to acquire input data. The input data corresponds to the first biological sequence type task, and the input data has a second sequence type tag and / or a second sequence element tag, the second sequence type tag is used to indicate the sequence type to which the input data belongs among the M sequence types, and the second sequence element tag is used to indicate the fragment type to which the component fragments in the input data belong. For the above-mentioned M sequence types, please refer to Figure 1 The relevant introduction in step S101 is not repeated here.

[0152] The processing module 1402 is used to input the input data into the first task model to obtain output data corresponding to the input data. The first task model is used to perform the first biological sequence type task. The first task model is constructed based on the biological sequence base model, and the biological sequence base model is obtained by training based on the first data set. The first data set includes biological sequences of M sequence types, and the biological sequences in the first data set have corresponding first sequence type tags and / or first sequence element tags. The first sequence type tag is used to indicate the sequence type to which the corresponding biological sequence belongs, and the first sequence element tag is used to indicate the fragment type to which the constituent fragments in the corresponding biological sequence belong. The input data and the biological sequences of the M sequence types have constituent fragments of the same fragment type.

[0153] Optionally, the M sequence types include at least one of a deoxyribonucleic acid type, a ribonucleic acid type, and a protein type, and the fragment type includes at least one of a 5' untranslated region type, a 3' untranslated region type, and a coding region type. Optionally, the biological sequence-based model includes an encoder and a decoder.

[0154] Optionally, the first biological sequence type task includes a discriminative task and / or a generative task, the discriminative task includes one or more of a sequence expression task, a sequence translation efficiency prediction task, a sequence structure prediction task, and a sequence functional attribute prediction task, and the generative task includes a sequence generation task.

[0155] Optionally, the first task model is used to execute multiple biological sequence tasks, the multiple biological sequence tasks include the first biological sequence task, and the first task model includes a biological sequence base model and model heads corresponding to the multiple biological sequence tasks.

[0156] Optionally, when the biological sequence base model includes a decoder and the above-mentioned multiple biological sequence tasks include discriminative tasks, the model heads corresponding to the above-mentioned multiple biological sequence tasks include a multi-layer perceptron, which corresponds to the discriminative tasks.

[0157] Optionally, the first task model is used to perform multiple biological sequence tasks, the multiple biological sequence tasks include a first biological sequence task, the input data has a task tag, the task tag is used to indicate the first biological sequence task, and the output data includes a natural language vocabulary corresponding to the first biological sequence task.

[0158] Optionally, when the first biological sequence task is a generative task, the output data and input data of the first task model belong to different sequence types among the M sequence types, or the constituent fragments in the output data and the constituent fragments in the input data belong to different fragment types.

[0159] It should be noted that the acquisition module 1401 and the processing module 1402 can be implemented by software, or by hardware, or by both software and hardware. Fig.13 Related introduction of .

[0160] It should also be noted that the above-mentioned biological sequence task processing device 1400 is used to execute Fig.10 For details, please refer to the method of the embodiment. Fig.10 The relevant description is not repeated here. Fig.14 The processing device 1400 for biological sequence tasks is only used as an example to illustrate the division of the above-mentioned functional modules. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the processing device 1400 for biological sequence tasks can be divided into other different functional modules to complete all or part of the functions described above.

[0161] See also Fig.15 The present application also provides a computing device 1500, including a bus 1502, a processor 1504, a memory 1506, and a communication interface 1508. The processor 1504, the memory 1506, and the communication interface 1508 communicate with each other through the bus 1502. The computing device 1500 may be a server, a terminal device, etc., and the embodiments of the present application do not specifically limit it, and the embodiments of the present application do not limit the number of processors and memories in the computing device 1500.

[0162] The bus 1502 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.15 The bus 1502 may include a path for transmitting information between various components of the computing device 1500 (eg, the memory 1506, the processor 1504, and the communication interface 1508).

[0163] The processor 1504 may include any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0164] The memory 1506 may include a volatile memory, such as a random access memory (RAM). The processor 1504 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0165] The memory 1506 stores executable program codes. The processor 1504 executes the executable program codes to respectively implement Fig.13 The functions of the acquisition module 1301 and the training module 1302 in the present application are realized. Figure 1 Alternatively, the processor 1504 executes the executable program code to respectively implement Fig.14 The functions of the acquisition module 1401 and the processing module 1402 in the present invention are implemented Fig.10 A method for processing biological sequence tasks according to an embodiment.

[0166] The communication interface 1508 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 1500 and other devices or communication networks.

[0167] The embodiment of the present application also provides a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smart phone.

[0168] like Fig.16 As shown, the computing device cluster includes at least one computing device 1500. The memory 1506 in one or more computing devices 1500 in the computing device cluster may store the same Figure 1 or Fig.10 The method instruction.

[0169] In some possible implementations, the memory 1506 of one or more computing devices 1500 in the computing device cluster may also store the memory 1506 for executing the above Figure 1 or Fig.10 In other words, the combination of one or more computing devices 1500 can jointly execute the method for Figure 1 or Fig.10 The method instruction.

[0170] It should be noted that the memory 1506 in different computing devices 1500 in the computing device cluster may store different instructions, respectively used to execute Fig.13 Part of the functions of the model training device 1300, that is, the instructions stored in the memory 1506 in the different computing devices 1500 can be implemented Fig.13 Alternatively, the memory 1506 in different computing devices 1500 in the computing device cluster may store different instructions, respectively, for executing Fig.14 Part of the functions of the processing device 1400 for biological sequence tasks, that is, the instructions stored in the memory 1506 of the different computing devices 1500 can be implemented Fig.14 The functions of one or more modules in the acquisition module 1401 and the processing module 1402.

[0171] In some possible implementations, one or more computing devices in the computing device cluster may be connected via a network, which may be a wide area network or a local area network. Fig.17 A possible implementation is shown. Fig.17As shown, two computing devices 1500A and 1500B are connected via a network. Specifically, they are connected to the network via a communication interface in each computing device. In this type of possible implementation, the memory 1506 in the computing device 1500A stores instructions for executing the functions of the acquisition module 1301. At the same time, the memory 1506 in the computing device 1500B stores instructions for executing the functions of the training module 1302.

[0172] It should be understood that Fig.17 The functions of the computing device 1500A shown in FIG. 1 may also be jointly completed by multiple computing devices 1500. Similarly, the functions of the computing device 1500B may also be jointly completed by multiple computing devices 1500.

[0173] The present application embodiment also provides another computing device cluster. The connection relationship between the computing devices in the computing device cluster can be similar to that of Fig.17 The connection mode of the computing device cluster. The difference is that the memory 1506 in one or more computing devices 1500 in the computing device cluster may store the same memory for executing the above Figure 1 or Fig.10 The method instruction.

[0174] In some possible implementations, the memory 1506 of one or more computing devices 1500 in the computing device cluster may also store a program for executing the executable program. Figure 1 or Fig.10 In other words, the combination of one or more computing devices 1500 can jointly execute instructions for executing the method that can be implemented. Figure 1 or Fig.10 The method instruction.

[0175] The present application also provides a computer-readable storage medium. The computer-readable storage medium may be any available medium that can be stored by a computing device or a data storage device such as a data center that includes one or more available media. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk). The computer-readable storage medium includes instructions that instruct a computing device cluster (including at least one computing device) to execute Figure 1 Example or Fig.10 Method of embodiment.

[0176] The present application also provides a computer program product including instructions. The computer program product may be software or a program product including instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on at least one computing device, the at least one computing device executes Figure 1 Example or Fig.10 Method of embodiment.

[0177] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of the present invention.

Claims

1. A model training method, characterized in that: The method comprises: Acquire a first data set, wherein the first data set includes biological sequences of multiple sequence types, and the biological sequences in the first data set have corresponding first sequence type tags and first sequence element tags, wherein the first sequence type tag is used to indicate the sequence type to which the corresponding biological sequence belongs, and the first sequence element tag is used to indicate the fragment type to which the constituent fragments in the corresponding biological sequence belong; The first neural network is trained based on the first data set to obtain a biological sequence base model, wherein the biological sequence base model is used to construct a sequence task model that performs multiple biological sequence tasks, and the input data of the sequence task model has a second sequence type tag and / or a second sequence element tag, the second sequence type tag is used to indicate the sequence type to which the input data belongs among the multiple sequence types, and the second sequence element tag is used to indicate the fragment type to which a component fragment in the input data belongs, and the biological sequences of the multiple sequence types and the input data have component fragments of the same fragment type.

2. The method according to claim 1, characterized in that: The multiple sequence types include at least one of a deoxyribonucleic acid type, a ribonucleic acid type, and a protein type, and the fragment type includes at least one of a 5' untranslated region type, a 3' untranslated region type, and a coding region type.

3. The method according to claim 1 or 2, characterized in that: The biological sequence-based model includes an encoder and a decoder.

4. The method according to any one of claims 1 to 3, characterized in that The multiple biological sequence tasks include discriminative tasks and / or generative tasks, the discriminative tasks include one or more of sequence expression tasks, sequence translation efficiency prediction tasks, sequence structure prediction tasks, and sequence functional attribute prediction tasks, and the generative tasks include sequence generation tasks.

5. The method according to any one of claims 1 to 4, characterized in that After obtaining the biological sequence-based model, the method further comprises: The biological sequence base model and the model heads corresponding to the multiple biological sequence tasks are trained based on a second data set to obtain the sequence task model, wherein the second data set corresponds to the multiple biological sequence tasks.

6. The method according to claim 5, characterized in that In the case where the biological sequence base model includes a decoder and the multiple biological sequence class tasks include discriminative tasks, the model heads corresponding to the multiple biological sequence class tasks include a multi-layer perceptron, and the multi-layer perceptron corresponds to the discriminative tasks.

7. The method according to any one of claims 1 to 4, characterized in that After obtaining the biological sequence-based model, the method further comprises: The biological sequence base model is trained based on multiple data sets to obtain the sequence task model, wherein the multiple data sets correspond one-to-one to the multiple biological sequence class tasks, and the biological sequence in each of the multiple data sets has a corresponding task tag, and the task tag is used to indicate the task to which the corresponding biological sequence corresponds in the multiple biological sequence class tasks and the natural language vocabulary corresponding to the task.

8. The method according to any one of claims 1 to 4, characterized in that After obtaining the biological sequence-based model, the method further comprises: Inputting training data into the biological sequence-based model to obtain a sequence generation result, wherein the multiple biological sequence tasks include a first biological sequence task, the training data corresponds to the first biological sequence task, the training data has corresponding target output data, the training data and the target output data belong to different sequence types among the multiple sequence types, or the constituent segments in the training data and the constituent segments in the target output data belong to different segment types; The biological sequence base model is updated according to the sequence generation result and the target output data to obtain a first task model for executing the first biological sequence task, and the first task model belongs to the sequence task model.

9. A method for processing biological sequence tasks, characterized in that: The method comprises: Acquire input data, wherein the input data corresponds to a first biological sequence task, and the input data has a second sequence type tag and / or a second sequence element tag, the second sequence type tag is used to indicate the sequence type to which the input data belongs among multiple sequence types, and the second sequence element tag is used to indicate the fragment type to which a component fragment in the input data belongs; Input data is input into a first task model to obtain output data corresponding to the input data, wherein the first task model is used to perform the first biological sequence task, the first task model is constructed based on a biological sequence base model, and the biological sequence base model is obtained by training based on a first data set, the first data set includes biological sequences of the plurality of sequence types, the biological sequences in the first data set have corresponding first sequence type tags and / or first sequence element tags, the first sequence type tag is used to indicate the sequence type to which the corresponding biological sequence belongs, the first sequence element tag is used to indicate the fragment type to which the constituent fragments in the corresponding biological sequence belong, and the input data and the biological sequences of the plurality of sequence types have constituent fragments of the same fragment type.

10. The method according to claim 9, characterized in that The multiple sequence types include at least one of a deoxyribonucleic acid type, a ribonucleic acid type, and a protein type, and the fragment type includes at least one of a 5' untranslated region type, a 3' untranslated region type, and a coding region type.

11. The method according to claim 9 or 10, characterized in that: The biological sequence-based model includes an encoder and a decoder.

12. The method according to any one of claims 9 to 11, characterized in that The first biological sequence task includes a discriminative task and / or a generative task, wherein the discriminative task includes one or more of a sequence expression task, a sequence translation efficiency prediction task, a sequence structure prediction task, and a sequence functional attribute prediction task, and the generative task includes a sequence generation task.

13. The method according to any one of claims 9 to 12, characterized in that The first task model is used to execute multiple biological sequence tasks, the multiple biological sequence tasks include the first biological sequence task, and the first task model includes the biological sequence base model and model heads corresponding to the multiple biological sequence tasks.

14. The method according to claim 13, characterized in that In the case where the biological sequence base model includes a decoder and the multiple biological sequence class tasks include discriminative tasks, the model heads corresponding to the multiple biological sequence class tasks include a multi-layer perceptron, and the multi-layer perceptron corresponds to the discriminative tasks.

15. The method according to any one of claims 9 to 12, characterized in that The first task model is used to perform multiple biological sequence tasks, the multiple biological sequence tasks include the first biological sequence task, the input data has a task tag, the task tag is used to indicate the first biological sequence task, and the output data includes a natural language vocabulary corresponding to the first biological sequence task.

16. The method according to any one of claims 9 to 15, characterized in that In the case where the first biological sequence task is a generative task, the output data and the input data belong to different sequence types among the multiple sequence types, or the constituent fragments in the output data and the constituent fragments in the input data belong to different fragment types.

17. A model training device, characterized in that: include: an acquisition module, configured to acquire a first data set, wherein the first data set includes biological sequences of multiple sequence types, and the biological sequences in the first data set have corresponding first sequence type tags and first sequence element tags, wherein the first sequence type tags are used to indicate the sequence type to which the corresponding biological sequence belongs, and the first sequence element tags are used to indicate the fragment type to which the constituent fragments in the corresponding biological sequence belong; A training module is used to train the first neural network based on the first data set to obtain a biological sequence base model, wherein the biological sequence base model is used to construct a sequence task model that performs multiple biological sequence tasks, and the input data of the sequence task model has a second sequence type tag and / or a second sequence element tag, the second sequence type tag is used to indicate the sequence type to which the input data belongs among the multiple sequence types, and the second sequence element tag is used to indicate the fragment type to which the constituent fragments in the input data belong, and the biological sequences of the multiple sequence types and the input data have constituent fragments of the same fragment type.

18. A biological sequence task processing device, characterized in that: include: An acquisition module, used for acquiring input data, wherein the input data corresponds to a first biological sequence type task, and the input data has a second sequence type tag and / or a second sequence element tag, wherein the second sequence type tag is used for indicating the sequence type to which the input data belongs among multiple sequence types, and the second sequence element tag is used for indicating the fragment type to which a constituent fragment in the input data belongs; A processing module is used to input input data into a first task model to obtain output data corresponding to the input data, wherein the first task model is used to perform the first biological sequence task, the first task model is constructed based on a biological sequence base model, and the biological sequence base model is obtained by training based on a first data set, the first data set includes biological sequences of the plurality of sequence types, the biological sequences in the first data set have corresponding first sequence type tags and / or first sequence element tags, the first sequence type tags are used to indicate the sequence type to which the corresponding biological sequence belongs, the first sequence element tags are used to indicate the fragment type to which the constituent fragments in the corresponding biological sequence belong, and the input data and the biological sequences of the plurality of sequence types have constituent fragments of the same fragment type.

19. A computing device cluster, characterized in that: It includes at least one computing device, each computing device includes a processor and a memory, and the processor of the at least one computing device is used to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the method as described in any one of claims 1-8 or 9-16.

20. A computer-readable storage medium, characterized in that: The method comprises computer program instructions. When the computer program instructions are executed by a computing device cluster, the computing device cluster performs the method as claimed in any one of claims 1-7 or 8-15.

21. A computer program product comprising instructions, characterized in that When the instructions are executed by a computing device cluster, the computing device cluster executes the method as described in any one of claims 1-8 or 9-16.

Images