Method for predicting signal peptides, method for constructing prediction model, device and computing equipment

Abstract

The present application relates to the field of computer technology, and specifically provides a method for predicting signal peptides, a method for constructing a prediction model, an apparatus, and a computing device. The method includes obtaining a target protein sequence; using the target protein sequence to generate a first input sequence and inputting it into a prediction model to obtain a predicted signal peptide sequence; the first input sequence includes the context representing the target protein sequence; wherein the prediction model is obtained by training a pre-trained language model using a training set, and the training set includes a second input sequence; the second input sequence includes the context representing the signal peptide sequence and the context representing the protein sequence. The method can generate a signal peptide sequence in the context using the protein sequence in the context and improve the accuracy of signal peptide prediction.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and specifically provides a method for predicting signal peptides, a method for constructing a prediction model, an apparatus and a computing device. Background Art

[0002] Since the Google team first proposed the self-attention mechanism in 2017 and created a new natural language processing model Transformer on this basis, a large number of powerful general model architectures based on the Transformer architecture have been born for representation learning or as generative models. Transformer is mainly composed of multiple encoders (Encoder) and multiple decoders (Decoder) stacked together. Based on Transformer, the most mainstream frameworks BERT (Bidirectional Encoder Representations from Transformers) and GPT (Generative Pre-trained Transformer) have evolved, which use the encoder part and decoder part of Transformer respectively. The biggest difference between BERT and GPT frameworks is that BERT uses the encoder part of Transformer as a multi-head self-attention mechanism, which can be calculated in parallel, while GPT uses the decoder part of Transformer as a masked multi-head attention. When generating text, the model can only see the previous input, and the subsequent input is masked, so it can only be decoded step by step from front to back. During pre-training, BERT will randomly mask some input text, and then let the model predict these masked texts based on the previous and next texts. This type of model is also called a masked language model, which has the ability of bidirectional encoding, but the ability to generate text is weak. During pre-training, GPT uses a unidirectional encoding method and can only predict the following text based on the previous text. This type of model is also called a generative language model.

[0003] A signal peptide is a short peptide sequence with a coding length of 15 to 30 amino acids at the N-terminus of a protein. It is used to guide the secretion and movement of newly synthesized proteins. After the protein is transported, the signal peptide is cut off by the signal peptidase. In the industry, before the target protein is expressed, the target protein to be produced often has no natural signal peptide or the expression level of its natural signal peptide is low. It is usually necessary to delete, replace or add the signal peptide, generally by screening signal peptides of similar species or evolutionarily close species to increase the expression level of the target protein. Currently, there is little research on the generation of signal peptides.

[0004] Therefore, under the above background, how to utilize the protein sequence below to generate the signal peptide sequence above and improve the efficiency is a technical problem to be solved. Summary of the invention

[0005] The present application provides a method for predicting signal peptides, a method for constructing a prediction model, an apparatus and a computing device, which can generate the signal peptide sequence above using the protein sequence below.

[0006] In order to achieve the above purpose, this application adopts the following technical solutions:

[0007] The first aspect of the present invention provides a method for predicting a signal peptide, comprising:

[0008] Obtain target protein sequence;

[0009] Generate a first input sequence using the target protein sequence and input it into a prediction model to obtain a predicted signal peptide sequence; the first input sequence includes the following content representing the target protein sequence;

[0010] The prediction model is obtained by training a pre-trained language model using a training set, and the training set includes a second input sequence; the second input sequence includes a context representing a signal peptide sequence and a context representing a protein sequence.

[0011] The current generative language model is a one-way generation model that can only predict the following context based on the previous context, but not the previous context based on the following context. In the case of less labeled context data (signal peptide sequence and protein sequence pairs), this application uses a small amount of labeled data to fine-tune the pre-trained language model containing a large amount of unlabeled data, thereby increasing the model's ability to generate the previous context using the information in the following context, and achieving the ability to directly generate the previous context using the information in the following context while keeping the input text structure unchanged. In addition, this application is the first to apply this method and system to the task of generating signal peptides.

[0012] As a possible implementation manner of the first aspect, the pre-trained language model includes a pre-trained bidirectional masked language model.

[0013] As a possible implementation manner of the first aspect, the word length of the preceding text and the word length of the following text are respectively fixed lengths.

[0014] As a possible implementation of the first aspect, the above in the second input sequence is generated by right-aligned padding or truncation using the signal peptide sequence; and / or,

[0015] The following in the second input sequence is generated by padding or truncating the protein sequence based on left alignment; and / or,

[0016] The following in the first input sequence is generated by padding or truncating the target protein sequence based on left alignment.

[0017] As a possible implementation manner of the first aspect, the method further includes: performing a mask operation on the above partial sequence in the second input sequence.

[0018] As a possible implementation of the first aspect, the loss function used to train the pre-trained language model includes:

[0019]

[0020] Among them, x i represents the ith sequence in the second input sequence, s represents the actual word length of the signal peptide sequence corresponding to the above sequence, and k represents the set word length of the above sequence.

[0021] As a possible implementation of the first aspect, the step of using the target protein sequence to generate a first input sequence and inputting it into a prediction model to obtain a predicted signal peptide sequence includes:

[0022] generating the following text representing the target protein sequence according to the target protein sequence;

[0023] Adding the preceding context represented by the mask before the following context to form the first input sequence;

[0024] The first input sequence is input into the prediction model to obtain a predicted signal peptide sequence.

[0025] The second aspect of the present application provides a method for predicting a signal peptide, comprising:

[0026] Obtain target protein sequence;

[0027] Generate the following text representing the target protein sequence according to the target protein sequence to obtain a first input sequence;

[0028] Inputting the first input sequence into a prediction model to obtain a predicted signal peptide sequence;

[0029] The prediction model is obtained by training a pre-trained language model using a training set, wherein the training set includes a second input sequence; the second input sequence includes a context representing a signal peptide sequence and a context representing a protein sequence, and a partial sequence in the context representing the signal peptide sequence is masked;

[0030] The loss functions used to train the pre-trained language model include:

[0031]

[0032] Among them, x irepresents the i-th sequence in the second input sequence, s represents the actual word length of the signal peptide sequence corresponding to the above sequence, and k represents the set word length of the above sequence;

[0033] The pre-trained language model includes a pre-trained bidirectional masked language model;

[0034] The word length of the preceding text and the word length of the following text are respectively fixed lengths;

[0035] The above in the second input sequence is generated by right-aligned padding or truncation using the signal peptide sequence; and / or,

[0036] The following in the second input sequence is generated by padding or truncating the protein sequence based on left alignment; and / or,

[0037] The following in the first input sequence is generated by padding or truncating the target protein sequence based on left alignment.

[0038] The third aspect of the present application provides a method for constructing a prediction model, comprising:

[0039] Obtaining a sequence pair consisting of a signal peptide sequence and a protein sequence;

[0040] Generate a second input sequence according to the sequence pair consisting of the signal peptide sequence and the protein sequence; the second input sequence includes an upper context representing the signal peptide sequence and a lower context representing the protein sequence;

[0041] The pre-trained language model is trained using a training set to obtain a prediction model, wherein the training set includes the second input sequence.

[0042] As a possible implementation manner of the third aspect, the pre-trained language model includes a pre-trained bidirectional masked language model.

[0043] As a possible implementation manner of the third aspect, the word length of the preceding text and the word length of the following text are respectively fixed lengths.

[0044] As a possible implementation of the third aspect, the above in the second input sequence is generated by right-aligned padding or truncation using the signal peptide sequence; and / or,

[0045] The following in the second input sequence is generated by padding or truncating the protein sequence based on left alignment.

[0046] As a possible implementation manner of the third aspect, the method further includes: performing a mask operation on the above partial sequence in the second input sequence.

[0047] As a possible implementation of the third aspect, the loss function used for training the pre-trained language model includes:

[0048]

[0049] Among them, x i represents the ith sequence in the second input sequence, s represents the actual word length of the signal peptide sequence corresponding to the above sequence, and k represents the set word length of the above sequence.

[0050] The fourth aspect of the present application provides a device for predicting a signal peptide, comprising:

[0051] An acquisition unit, used for acquiring a target protein sequence;

[0052] A processing unit, configured to generate a first input sequence from the target protein sequence and input it into a prediction model to obtain a predicted signal peptide sequence; the first input sequence includes a subtext representing the target protein sequence;

[0053] The prediction model is obtained by training a pre-trained language model using a training set, and the training set includes a second input sequence; the second input sequence includes a context representing a signal peptide sequence and a context representing a protein sequence.

[0054] As a possible implementation manner of the fourth aspect, the pre-trained language model includes a pre-trained bidirectional masked language model.

[0055] As a possible implementation manner of the fourth aspect, the word length of the preceding text and the word length of the following text are respectively fixed lengths.

[0056] As a possible implementation of the fourth aspect, the above in the second input sequence is generated by right-aligned padding or truncation using the signal peptide sequence; and / or,

[0057] The following in the second input sequence is generated by padding or truncating the protein sequence based on left alignment; and / or,

[0058] The following in the first input sequence is generated by padding or truncating the target protein sequence based on left alignment.

[0059] As a possible implementation manner of the fourth aspect, the method further includes: performing a mask operation on the above partial sequence in the second input sequence.

[0060] As a possible implementation of the fourth aspect, the loss function used to train the pre-trained language model includes:

[0061]

[0062] Among them, x i represents the ith sequence in the second input sequence, s represents the actual word length of the signal peptide sequence corresponding to the above sequence, and k represents the set word length of the above sequence.

[0063] As a possible implementation of the fourth aspect, the step of generating a first input sequence from the target protein sequence and inputting the first input sequence into a prediction model to obtain a predicted signal peptide sequence includes:

[0064] generating the following text representing the target protein sequence according to the target protein sequence;

[0065] Adding the preceding context represented by the mask before the following context to form the first input sequence;

[0066] The first input sequence is input into the prediction model to obtain a predicted signal peptide sequence.

[0067] A fifth aspect of the present application provides a device for constructing a prediction model, comprising:

[0068] An acquisition unit, used for acquiring a sequence pair consisting of a signal peptide sequence and a protein sequence;

[0069] A sequence generating unit, used to generate a second input sequence according to the sequence pair consisting of the signal peptide sequence and the protein sequence; the second input sequence includes an upper context representing the signal peptide sequence and a lower context representing the protein sequence;

[0070] A training unit is used to train the pre-trained language model using a training set to obtain a prediction model, wherein the training set includes the second input sequence.

[0071] As a possible implementation manner of the fifth aspect, the pre-trained language model includes a pre-trained bidirectional masked language model.

[0072] As a possible implementation manner of the fifth aspect, the word length of the preceding text and the word length of the following text are respectively fixed lengths.

[0073] As a possible implementation of the fifth aspect, the above in the second input sequence is generated by right-aligned padding or truncation using the signal peptide sequence; and / or,

[0074] The following in the second input sequence is generated by padding or truncating the protein sequence based on left alignment.

[0075] As a possible implementation manner of the fifth aspect, the method further includes: performing a mask operation on the above partial sequence in the second input sequence.

[0076] As a possible implementation of the fifth aspect, the loss function used for training the pre-trained language model includes:

[0077]

[0078] Among them, x i represents the ith sequence in the second input sequence, s represents the actual word length of the signal peptide sequence corresponding to the above sequence, and k represents the set word length of the above sequence.

[0079] A sixth aspect of the present application provides a computing device, including:

[0080] processor, and

[0081] A memory having program instructions stored thereon, wherein when the program instructions are executed by the processor, the processor executes any of the methods for predicting signal peptides of the first aspect or the method for predicting signal peptides of the second aspect, or the processor executes any of the methods for constructing a prediction model of the third aspect.

[0082] The seventh aspect of the present application provides a computer-readable storage medium, which stores a computer program, and the computer program is suitable for being loaded and executed by a processor, so that a computer device having the processor performs any of the methods for predicting signal peptides in the first aspect, or the method for predicting signal peptides in the second aspect, or any of the methods for constructing a prediction model in the third aspect.

[0083] The eighth aspect of the present application provides a computer program product, which includes computer instructions, and when the computer instructions are executed by a processor, implement any method for predicting signal peptides described in the first aspect, or the method for predicting signal peptides described in the second aspect, or any method for constructing a prediction model described in the third aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0084] The following further illustrates the various features of the present application and the connections between the various features with reference to the accompanying drawings. The accompanying drawings are all exemplary, and some features are not shown in actual proportion. Some drawings may omit features that are conventional in the field involved in the present application and are not necessary for the present application, or additional features that are not necessary for the present application may be shown. The combination of the various features shown in the accompanying drawings is not intended to limit the present application. In addition, throughout this specification, the same figure numerals refer to the same content. The specific description of the drawings is as follows:

[0085] Figure 1 is a flow chart of the method for predicting signal peptides provided in the first embodiment of the present application;

[0086] Figure 2 is a schematic diagram of a prediction model provided in the second embodiment of the present application;

[0087] Figure 3 is a flowchart of a prediction model construction method provided in the third embodiment of the present application;

[0088] Figure 4 is a schematic structural diagram of a prediction signal peptide device provided in the ninth embodiment of the present application;

[0089] Figure 5 is a schematic structural diagram of a device for constructing a prediction model provided in the tenth embodiment of the present application;

[0090] Figure 6 A structural schematic diagram of a computing device provided in the eleventh embodiment of the present application. DETAILED DESCRIPTION

[0091] The technical solution provided by the present application is further described below with reference to the accompanying drawings and examples. It should be understood that the system structure and business scenarios provided in the examples of the present application are mainly to illustrate the possible implementation methods of the technical solution of the present application and should not be interpreted as the only limitation on the technical solution of the present application. It is known to those of ordinary skill in the art that with the evolution of the system structure and the emergence of new business scenarios, the technical solution provided by the present application is also applicable to similar technical problems.

[0092] The scheme for predicting signal peptides provided in the embodiments of the present application includes a method for predicting signal peptides, a method for constructing a prediction model, corresponding devices, computing equipment, computer-readable storage media, and computer program products. Since the principles of solving the problems in these technical solutions are the same or similar, some repetitions may not be repeated in the introduction of the following specific embodiments, but it should be regarded as that these specific embodiments have been referenced to each other and can be combined with each other.

[0093] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of this application. In the event of any inconsistency, the meaning described in this specification or the meaning derived from the contents recorded in this specification shall prevail. In addition, the terms used herein are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application. In order to accurately describe the technical content in this application and to accurately understand the present invention, the following explanations or definitions are given to the terms used in this specification before describing the specific embodiments:

[0094] (1) Pre-training and fine-tuning: Pre-training refers to a pre-trained model or the process of pre-training a model; fine-tuning refers to the process of applying a pre-trained model to one's own dataset and adapting the parameters to one's own dataset.

[0095] (2) Token: refers to the basic unit that makes up a word, also called a word unit, identifier, etc.

[0096] (3) Special words: words that express specific functions, such as <sos> 、 <bos> 、 <go>The word segment represents the beginning of a sequence and can be used <eos>Represents the end of a sequence, which can be used <pad>As a fill character to fill the gaps in the sequence, you can use <mask>To fill the masked sequence, you can use <sep>As a separator between two sequences, etc. The function represented by a specific word segment is based on the definition during model training.

[0097] In a signal peptide prediction method, the implementation method provided is: generate a signal peptide database from a large number of human immunoglobulin (Ig) heavy chains (HC) and light chains (LC). The databases of heavy chain signal peptides and light chain signal peptides are clustered according to sequence similarity, and finally the heavy chain signal peptides are divided into 8 categories and the light chain signal peptides are divided into 2 categories, which are used to analyze the effect of signal peptides on expression. The optimized signal peptide can increase the yield of one of the proteins by two times compared with the original signal peptide. This method does not involve how to predict signal peptides based on protein sequences.

[0098] In another signal peptide prediction method, the implementation method provided is: extract the already labeled signal peptide-protein pairing data in Uniprot, including a total of about 40,000 data that have been experimentally verified and predicted. After separating the signal peptide sequence from the mature protein sequence, the mature protein sequence is used as the input of the Encoder in the Transformer, and the signal peptide is the output annotation of the Transformer Decoder to train the Transformer model. Finally, the signal peptide sequence generated based on the mature protein sequence in the test set has a recovery rate of about 35%. The generated signal peptide replaces the signal peptide of the industrial enzyme itself, and more than 50% of the signal peptides can detect enzyme activity. This method uses the protein sequence as input and the signal peptide as a whole as an annotation to directly train the Transformer, and the prediction accuracy (i.e., recovery rate) of the trained model is about 35%.

[0099] This application proposes another scheme for predicting signal peptides. The prediction model used is obtained by fine-tuning the pre-trained language model, and it is the first time to predict the upstream signal peptide sequence based on the downstream mature protein sequence, and the prediction accuracy can be increased to about 45%. The method provided in this application can be used for signal peptide screening of target proteins, and the predicted signal peptide sequence can be used as the signal peptide of the target protein or replace the natural signal peptide of the target protein, so as to improve the secretion efficiency of the target protein, increase the expression amount of the target protein, etc.

[0100] The various embodiments of the present application will be described in detail below.

[0101] like Figure 1 The first embodiment of the present application provides an example of a method for predicting a signal peptide, comprising the following steps:

[0102] S11: Obtain target protein sequence.

[0103] Wherein, the target protein sequence refers to the protein sequence of the signal peptide to be predicted. In some embodiments, the target protein sequence is a mature protein sequence. In some embodiments, the target protein sequence can be obtained from protein sequence databases such as NCBI and Uniprot.

[0104] In addition, in the process of verifying the prediction model of the present application, the target protein sequence can be a target protein sequence that has been annotated with a signal peptide. In this regard, in the embodiments of the verification process, the target protein sequence can be obtained through protein sequence databases such as NCBI and Uniprot.

[0105] S12: Use the target protein sequence to generate a first input sequence input into a prediction model to obtain a predicted signal peptide sequence; the first input sequence includes the context representing the target protein sequence; wherein the prediction model is obtained by training a pre-trained language model using a training set, and the training set includes a second input sequence; the second input sequence includes the context representing the signal peptide sequence and the context representing the protein sequence.

[0106] Among them, the target protein sequence, signal peptide sequence and protein sequence can all be composed of one or more amino acid characters. One amino acid character can be represented by one letter, and different amino acid characters can be represented by different letters.

[0107] This method can use a small amount of protein data with annotated signal peptides to fine-tune a pre-trained language model containing a large amount of protein data without annotation information. The fine-tuned prediction model can have the ability to generate the previous context using the following context.

[0108] In some embodiments, the target protein sequence, signal peptide sequence and protein sequence can be divided into multiple tokens to form the sequence using letter, word, phrase, etc. In some embodiments, the tokens can be divided according to amino acid residues.

[0109] In some embodiments, the pre-trained language model includes a pre-trained bidirectional masked language model. The bidirectional masked language model can be a Bert model or other variant models based on the Bert model. Through the masked language model with bidirectional encoding capability, the information below can be directly used to generate the previous text while keeping the input text structure unchanged.

[0110] In some embodiments, the pre-trained language model may be a pre-trained protein language model esm, such as esm2 or a variant model based on esm2, such as esm2_t33_650M_UR50D and other protein language models, or esm1 or a variant model based on esm1, such as esm-1b, esm-msa-1b, esm-1v and other protein language models. In some embodiments, the protein language model may be a language model that is unsupervisedly trained using a large number of protein sequences without labeled signal peptides.

[0111] In some embodiments, the second input sequence in the training set is derived from a protein sequence containing a signal peptide sequence annotation. The protein sequence containing a signal peptide sequence annotation can be obtained through a protein sequence database such as NCBI and Uniprot.

[0112] In some embodiments, the second input sequence in the training set is defined to include a preceding context and a following context according to sequence order, wherein, for the obtained protein sequence containing a signal peptide sequence annotation, the preceding context is generated based on the signal peptide sequence, and the following context is generated based on the protein sequence.

[0113] In some embodiments, the length of the preceding word segment and the length of the following word segment can be fixed lengths. Setting them to fixed lengths can make it easier to distinguish the preceding and following words, and limit the amount of data for each context, thereby limiting the amount of computation required for model training, and improving the training efficiency of the pre-trained language model.

[0114] In some embodiments, when the context in the second input sequence is of fixed length, the context can be generated by the signal peptide sequence based on padding or truncation. For example, based on left alignment, the right side of the signal peptide sequence is padded or truncated; based on right alignment, the left side of the signal peptide sequence is padded or truncated. Preferably, the right alignment is used, and the words used for padding can be special words, such as <bos> 、 <eos>or <pad>wait.

[0115] In some embodiments, the fixed length of the context in the second input sequence is set according to the word length of the signal peptide sequence. For example, if more than 96% of the signal peptide word lengths are less than 32, the word length of the context may be 32.

[0116] In some embodiments, when the context in the second input sequence is of fixed length, the context can be generated by using the protein sequence based on padding or truncation. For example, based on left alignment, the right side of the protein sequence is padded or truncated; based on right alignment, the left side of the protein sequence is padded or truncated. Preferably, the left alignment is used, and the padded words can be special words, such as <pad> 、 <eos>wait.

[0117] In some embodiments, the fixed length of the following in the second input sequence can be set according to the word length of the protein sequence, for example, the word length is 128, 256, 512, 1024, etc.

[0118] Among them, when the above is generated in a right-aligned manner based on the signal peptide sequence, and the below is generated in a left-aligned manner based on the protein sequence, the part containing valid data in the second input sequence of the input prediction model can be concentrated, which is beneficial to improving the efficiency of prediction model training.

[0119] In some embodiments, when the context in the first input sequence is of fixed length, the target protein sequence can be used to generate the context based on padding or truncation, for example, based on left alignment, the right side of the target protein sequence is padded or truncated; based on right alignment, the left side of the target protein sequence is padded or truncated. Generally, the alignment method is consistent with the alignment method when generating the context in the second input sequence.

[0120] In some embodiments, the length of the preceding word segment and the length of the following word segment can be indefinite. For example, the length of the context can be adjusted with the signal peptide sequence and the protein sequence to represent the complete signal peptide sequence and the protein sequence. Special words can be used between the contexts, such as <sep>In some embodiments, only the upper or lower text may be fixed in length. The fixed-length part may be processed with reference to the above alignment method, which will not be described in detail. Among them, the variable-length method can improve the accuracy of prediction because it can represent the complete signal peptide sequence or protein sequence, but it will also lead to an increase in the amount of calculation for model training.

[0121] In some embodiments, in order to improve the prediction accuracy of the prediction model, a mask operation may be performed on the partial sequence in the second input sequence. In some embodiments, the mask operation includes replacing the partial sequence with a special byte or with other words.

[0122] In some embodiments, the masking operation of the partial sequence may include performing one of the following operations, or any combination of operations, on the partial sequence, wherein the following probability values ​​are examples and may also be other values:

[0123] (1) Replace with a special word with a probability of 80%, such as <mask>;

[0124] (2) Replace with other normal words with a probability of 10%, such as replacing with words at other positions in the above sequence;

[0125] (3) Remain unchanged with a probability of 10%.

[0126] In some embodiments, in order to improve the accuracy of prediction, the loss function used to train the pre-trained language model includes:

[0127]

[0128] Among them, x i represents the ith sequence in the second input sequence, s represents the actual word length of the signal peptide sequence corresponding to the above sequence, and k represents the set word length of the above sequence.

[0129] In this loss function, only the special delimited word section closest to the first normal word section in the above text is used to calculate the loss function, and other special delimited words sections are not used to calculate the loss function. This is because most of the above text needs to be completed with special delimited words sections. If all special delimited words sections are used to calculate the loss function, the model will tend to predict too many special delimited words sections. The above loss function of this application can solve this problem.

[0130] In some embodiments, during the inference process, i.e., during the process of inputting the first input sequence into the prediction model to obtain the predicted signal peptide sequence, in order to make the structure of the first input sequence conform to the input format required by the prediction model (the required input format is the same format as the structure of the second input sequence during training), the first input sequence is formed by adding a context represented by a mask before the context generated by the target protein sequence. For example, the context can use special words, such as <mask>Equal mask representation.

[0131] In some embodiments, during the training phase of the prediction model, it is necessary to pre-set hyperparameters of the prediction model, such as the number of neural network layers, the number of training rounds, the format of the neural network input, the activation function, etc., and also to set the model architecture.

[0132] In some embodiments, the prediction model architecture of the present application may include an input layer, a language model, and a linear model coupled in sequence, for example, Figure 2 An example of a prediction model of such an architecture provided by the second embodiment of the present application is shown. The prediction model provided by this embodiment is introduced below:

[0133] In the training phase, the input layer can be used to obtain the signal peptide and protein sequence to generate a second input sequence, which includes the context representing the signal peptide and the context representing the protein sequence. Specifically, the context includes a sequence composed of each word segment. The input layer can also perform mask processing on part of the sequence of the context. The generated second input sequence is input into the language model (such as Figure 2 The bidirectional masked language model in is a pre-trained language model), the intermediate vector of the word segment output by the language model is provided to the linear model, and the linear model serves as the output layer to output each word segment in turn. The output words as a whole constitute the signal peptide sequence predicted during the training process. The prediction model is trained according to the predicted signal peptide sequence and the signal peptide obtained by the input layer in combination with the loss function, wherein the parameters of the language model and the parameters of the linear model are trained simultaneously.

[0134] In the inference stage, the input layer can be used to obtain the target protein sequence and generate a first input sequence. The first input sequence includes the context represented by the mask and the context representing the target protein sequence. Specifically, the context includes a sequence composed of each word. The generated first input sequence is input into the language model (such as Figure 2 The bidirectional masked language model in is the model trained in the above training stage), the intermediate vector of the word segment output by the language model is provided to the linear model, and the linear model outputs each word segment in turn as the output layer, and the output words constitute the predicted signal peptide sequence as a whole.

[0135] When the input layer inputs its output (such as the second input sequence or the first input sequence) into the language model, it can also input the position encoding of each word segment into the language model. In some embodiments, the position encoding can also exist as part of the language model. The language model may include multiple attention encoding layers (such as self-attention encoding layers), linear layers, etc. For example, the language model can be a bidirectional masked language model based on BERT.

[0136] Among them, the linear layer is used as the output layer, and each word with the largest predicted probability is output in turn to obtain the predicted signal peptide sequence. In some embodiments, when the linear layer outputs each word, it can use non-autoregression decoding method (Non-Autoregression) to decode and obtain the output of the word at the current position. In some embodiments, the linear layer can use autoregression decoding method (Autoregression) to decode and obtain the output of the word at the current position based on the previously output word. When the predicted above sequence is obtained After that, search for the first occurrence of the special delimiter word from the end to the front. <bos>or <eos>The final predicted sequence is

[0137] For the specific process of the training phase and the inference phase of the prediction model, please refer to the aforementioned method for predicting signal peptides, or the method for constructing a prediction model described later, and the corresponding devices and descriptions in each specific implementation manner will not be repeated here.

[0138] In addition, for Figure 2 In the architecture of the prediction model shown in FIG. 1 , in some other embodiments, the linear model may also appear as part of the language model, for example, as the last layer in the language model, instead of being represented separately. In some other embodiments, other layers may be included between the language model and the linear layer, such as a fully connected network layer, an RNN layer, a BERT layer, a transformer layer, a self-attention layer, etc. These other layers may also be regarded as part of the language model layer or the linear layer.

[0139] Correspondingly, the present application also provides a method for constructing a prediction model, by training the above-mentioned pre-trained language model to obtain the prediction model of the present application. Figure 3 An example of a prediction model construction method provided in the third embodiment of the present application is shown, comprising the following steps:

[0140] S31: Obtain a sequence pair consisting of a signal peptide sequence and a protein sequence.

[0141] In some embodiments, protein sequences containing signal peptide annotations can be obtained by screening protein sequence databases such as NCBI and Uniprot, and the signal peptide sequence and the subsequent protein sequence constitute a sequence pair.

[0142] S32: Generate a second input sequence according to the sequence pair consisting of the signal peptide sequence and the protein sequence; the second input sequence includes an upper context representing the signal peptide sequence and a lower context representing the protein sequence.

[0143] S33: Using a training set to train a pre-trained language model to obtain a prediction model, wherein the training set includes the second input sequence.

[0144] The prediction model construction method provided in the present application and the content of the prediction model can refer to the corresponding description in the method for predicting signal peptides, or refer to the description in the content of the invention, which will not be repeated here.

[0145] In order to better understand the present application, the present application is further introduced in conjunction with specific embodiments. Among them, an example of a scheme for predicting signal peptides provided in the fourth embodiment of the present application includes a training phase and an inference phase, which are introduced below:

[0146] I. Training Phase: This phase is used to train the prediction model.

[0147] Among them, the prediction model can refer to Figure 2 the example shown, including an input layer, a language model, and a linear model. The input layer is used to generate a second input sequence. The language model is represented by h = B( m ), where B is a bidirectional masked language model, including token embeddings, position embeddings, and multiple layers of Transformer blocks, which maps each token of the input into an embedding representation vector of the language model. In this example, the language model is a pre-trained bidirectional masked language model. The linear model is represented by y = W(h), where W is a linear model that maps the embedding representation vector of each token into the probabilities of all tokens in the vocabulary. Specifically as follows:

[0148] S1. The input layer obtains the signal peptide sequence and the protein sequence pair and generates a second input sequence. This process includes tokenization, truncation and alignment, masking operations, etc., which can be specifically referred to in the following (a)-(c).

[0149] (a) The input context is divided into multiple tokens according to word segmentation methods such as letters, words, phrases, etc., where the length of the tokens in the above context is s, and the length of the tokens in the following context is m.

[0150] (b) Let x = (x1,…,x k ,x k+1 ,…x n ) be the input text of a fixed length. If s < k, that is, the length of the above context is less than k, then x k-s+1 ,…,x k are the respective tokens of the above context, and x1,…,x k-s are special tokens <bos>or <eos>If s >= k, that is, the length of the above text is greater than k, then truncate the above text, x1, …, x k Align it to the right with the above text, that is, x k is the last word segment of the above text. If k + m >= n, that is, the length of the text below k + m exceeds n, then truncate the part of the text below that exceeds n, x k+1 , …, x n Align it to the left with the below text, that is, x k+1 is the first word segment of the below text; if k + m < n, then use the special word segment <pad>Fill the text to length n, that is, x k+1 ,…,x k+m For each word corresponding to the following text, x k+m+1 ,…,x n For special words <pad>.

[0151] (c) Based on m = M (x) performs a mask operation, where M is the mask method, and converts x = (x1, ..., x k ,x k+1 ,…x n ) in (x1,…,x k ) is replaced with a special word <mask>Specifically, in the training set, the mask method M adopts the mask method of the BERT article in 2018, that is, x1,…,x k 80% are replaced by special words <mask>, 10% are replaced by other normal words, and 10% remain unchanged.

[0152] S2. The output of the input layer is processed by the language model and the linear model and then output, and the loss function is selected. The prediction model is trained according to the output and the loss function, including training the language model and the linear model. The details are as follows:

[0153] The loss function is x1,…,x1 in each sequence x in the sequence library X k Each natural word in the sequence and the special word closest to the first natural word in the previous sequence, that is, x k-s ,x k-s+1 ,…,x k The average of the negative log-likelihood probability. If s>=k, the loss function is x1,…,x k The negative log-likelihood probability p(x i ). i ) is the masked word output by the model and is correctly predicted as the original word x i Probability:

[0154]

[0155] During training, a suitable optimizer and learning rate training model are used to minimize the loss function.

[0156] 2. Inference stage: This stage uses the prediction model to infer the signal peptide sequence of the target protein.

[0157] The reasoning phase is as follows:

[0158] S1. Prepare the target protein sequence for the context that needs to be predicted.

[0159] S2, the input layer obtains the target protein sequence and generates the first input sequence. This process includes word segmentation, truncation and alignment, mask operation, etc. Specifically:

[0160] Segment the target protein sequence below and replace it with words, and add k words in front of the following <mask>Special word segment, if the number of words after adding the special word segment is less than n, add <pad>Special word completion, if it exceeds n, it will be truncated after the sequence.

[0161] S3, the output of the input layer is processed by the language model and the linear model and then output to obtain the predicted signal peptide sequence. Specifically, the probability of the word segment in the above text predicted by the linear layer output y = (y1, .., y k ), the words with the highest probability are the words that constitute the signal peptide sequence.

[0162] Among them, the output layer can obtain the predicted signal peptide sequence in one of the following two ways:

[0163] 1) Use non-autoregression decoding method to get the predicted signal peptide sequence:

[0164] The word segment at each position is predicted to be That is, the word with the highest probability at each position.

[0165] Get the predicted sequence above After that, search for the first occurrence of the special delimiter word from the end to the front. <bos>or <eos>The final sequence of the above text is

[0166] 2) Using the autoregression decoding method (Autoregression) to obtain the predicted signal peptide sequence:

[0167] Autoregressive decoding uses the beam search method to find the maximum probability P(x1,…,x k |x k+1 ,…,x n )=P(x1|x2,…,x k , x k+1 ,…,x n )*..*P(x k-1 |x k , x k+1 ,…,x n )*P(x k |x k+1 ,…,x n ). Beam search includes a hyperparameter, the beam size B. If B is 1, the beam search is a greedy search. If B is greater than 1, the search space is expanded, making it easier to obtain the global optimal solution, and the computational complexity will increase accordingly.

[0168] Find the probability value P(x) of the output closest to the following text k |x k+1 ,…,x n ), take the largest first B words x k save.

[0169] The x obtained in the first B cases are k As input, find P(x k-1 |x k , x k+1 ,…,x n ) probability value, taking the maximum probability P(x k-1 ,x k |x k+1 ,…,x n )=P(x k-1 |x k , x k+1 ,…,x n )*P(x k |x k+1 ,…,x n )'s first B results are saved.

[0170] And so on, each step saves the top B results with the highest probability until a special delimiting word section appears.

[0171] <bos>or <eos>, select the result with the highest probability as the decoding result.

[0172] Its objective function is:

[0173]

[0174] After taking the logarithm of the probability and adding the sequence length penalty, the objective function of the speed search becomes:

[0175]

[0176] Where T is the length of the output sequence, α∈[0,1] is a hyperparameter. If α is 0, it means that there is no penalty for the sequence length.

[0177] The present application is further described below in combination with the fifth to eighth embodiments. In the following examples, the performance of the signal peptide prediction method of the present application is also evaluated. Specifically, the generated sequence is aligned with the true sequence, and the sequence recovery rate SRR (Sequence Recovery Rate, the percentage of words with consistent amino acids at the same position after alignment to the total number of words) is calculated as an evaluation indicator of the model.

[0178] The fifth embodiment of the present application:

[0179] S1. Extract the signal peptide and mature protein paired sequence dataset: Extract protein sequences with signal peptide annotations and evidence type "ECO:0000269" in Uniprot, obtain the start and end positions of the signal peptide according to the annotations, define the sequence from the start position to the end position of the signal peptide as the signal peptide sequence, and define the sequence from the end position of the signal peptide +1 to the end as the mature protein sequence. Obtain the one-to-one correspondence between the Uniprot ID and the signal peptide-mature protein sequence.

[0180] S2. Divide the training set and validation set: Map the Uniprot ID in step 1 to the Uniref50 ID. Sequences with sequence similarity higher than 50% will be mapped to the same Uniref50 ID. According to the Uniref50 ID, the data set is randomly divided into the training set and the validation set in a ratio of 9:1, ensuring that the sequence similarity between the validation set and the training set is no higher than 50%.

[0181] S3. Prepare a suitable pre-trained bidirectional masked language model: Select the bidirectional masked model esm2_t33_650M_UR50D pre-trained on more than 250 million protein sequences as the pre-trained model.

[0182] S4. Taking each amino acid residue as a word segment, assuming the length of the signal peptide sequence is s, the length of the mature protein sequence is m, and x = (x1, …, x k , x k+1 , … x n ) is the prepared input text of a fixed length n. If s < k, then x k-s+1 , …, x k are the amino acid residues of the signal peptide sequence, and x1, …, x k-s are special word segments <bos>or <eos>; if s >= k, then x1, …, x k is aligned to the right with the signal peptide sequence, i.e., x k is aligned with the last amino acid residue of the signal peptide sequence, and the extra signal peptide amino acid residues will be truncated. If k + m < n, then use the special word segment <pad>Fill the input text to length n, that is, x k+1 ,…,x k+m is the amino acid sequence of the mature protein, x k+m+1 ,…,x n For special words <pad>; If k+m>=n, then cut off the part longer than n, x k+1 ,…,x n Aligned to the left with the mature protein amino acid sequence, i.e. x k+1 It is the first amino acid residue of the mature protein sequence.

[0183] S5. Mask: Mask x1,…,x k Use masking method M to mask it, that is, x1,…,x k 80% are replaced by special words <mask>, 10% are replaced by other normal words, and 10% remain unchanged. k 100% replaced with special words <mask>.

[0184] S6. The loss function is x1,…,x2 in each signal peptide-mature protein pairing sequence x in the signal peptide-mature protein pairing library X. k Each amino acid residue in the signal peptide sequence and the special word x k-s ,x k-s+1 ,…,x k The negative log-likelihood probability p(x i ). If s>=k, the loss function is x1,…,x k The negative log-likelihood probability p(x i ). i ) is the masked word output by the model and is correctly predicted as the original word x i Probability.

[0185]

[0186] S7. Select n=128, 256, 512, 1024, k=32 (more than 96% of signal peptide lengths s are less than 32), EPOCH=5, learning_rate=1e-4, batch_size=16, optimizer is Adam optimizer, and start training.

[0187] S8. When the Loss in the validation set no longer decreases or reaches EPOCH=5, stop training.

[0188] S9. Use non-autoregressive methods to test the validation set x1,…,x k For decoding, the word segment at each position is predicted as = argmaxp(x i ), that is, the word with the highest probability at each position. Get the predicted sequence above After that, search for the first occurrence of the special delimiter word from the end to the front. <bos>or <eos>Position j, the final signal peptide sequence is

[0189] S10. Calculation The predicted signal peptide sequence Compared with the original signal peptide sequence (x k-s+1 ,…,x k ) to perform sequence alignment, counting the number of identical amino acid residues i and the total length of the aligned sequences n, the sequence recovery rate (SRR) is:

[0190] S11. The final sequence recovery rates obtained by the models trained with n=128, 256, 512, and 1024 are shown in the following table.

[0191] N＝ 128 256 512 1024 SRR 43.5％ 45.0％ 44.6％ 45.7％

[0192] Sixth embodiment of the present application

[0193] S1. See steps S1-S6 of the fifth embodiment.

[0194] S2. Select n=256, k=32 (more than 96% of signal peptide lengths s are less than 32), EPOCH=5, learning_rate=1e-4, batch_size=16, Adam optimizer, freeze the weights of some attention layers of esm2_t33_650M_UR50D, and only train the last 1, 2, 4, 8, 16 and 33 layers of the model's attention layer (do not freeze the weights of any layer).

[0195] S3. When the Loss in the validation set no longer decreases or reaches EPOCH=5, stop training.

[0196] S4. Use non-autoregressive methods to analyze the validation set x1,…,x k For decoding, the word segment at each position is predicted as That is, the word with the highest probability at each position. Get the predicted sequence above After that, search for the first occurrence of the special delimiter word from the end to the front. <bos>or <eos>Position j, the final signal peptide sequence is

[0197] S5. Calculation The predicted signal peptide sequence Compared with the original signal peptide sequence (x k-s+1 ,…,x k ) to perform sequence alignment, counting the number of identical amino acid residues i and the total length of the aligned sequences n, the sequence recovery rate (SRR) is:

[0198] S6. The final sequence recovery rates obtained by training the model with different numbers of attention layers are shown in the following table.

[0199] 1 2 4 8 16 33 SRR 36.9％ 39.5％ 41.6％ 42.4％ 43.8％ 45.0％

[0200] The seventh embodiment of the present application:

[0201] S1. See steps S1-S6 of the fifth embodiment.

[0202] S2. Select n=256, k=32 (more than 96% of signal peptide lengths s are less than 32), EPOCH=5, learning_rate=1e-4, batch_size=16, and the optimizer is Adam optimizer.

[0203] S3. When the Loss in the validation set no longer decreases or reaches EPOCH=5, stop training.

[0204] S4. Use autoregression method to analyze the validation set x1,…,x k Decode. Autoregressive decoding uses the beam search method to find the objective function. α is 0, that is, no penalty is imposed on the sequence length, and the cluster size B is 1 and 10

[0205] First, the probability value P(x k |x k+1 ,…,x n ), take the first B sequences x with the highest scores k save.

[0206] The B sequences x obtained in the previous step are k As input, find P(x k-1 |x k , x k+1 ,…,x n ) probability value, take the maximum probability value P(x k-1 ,x k |x k+1 ,…,x n )=P(x k-1 |x k , x k+1 ,…,x n )*P(x k |x k+1 ,…,x n )'s first B sequences x k-1 ,x k save.

[0207] Repeat the previous step to get the first B sequences as input, find the probability of the previous word, and maximize it Until you encounter a special delimiter <bos>or <eos>So far, the result with the highest probability is selected as the decoding result to obtain the final signal peptide sequence:

[0208] S5. Calculation The predicted signal peptide sequence Compared with the original signal peptide sequence (x k-s+1 ,…,x k ) to perform sequence alignment, counting the number of identical amino acid residues i and the total length of the aligned sequences n, the sequence recovery rate (SRR) is:

[0209] S6. The final sequence recovery rates obtained by using different autoregressive beam searches with different beam sizes and non-autoregressive methods are shown in the following table.

[0210] Non-autoregressive Autoregressive B = 1 Autoregressive B = 10 SRR 45.0％ 45.3％ 45.6％

[0211] The eighth embodiment of the present application:

[0212] S1. Directly use the pre-trained esm2_t33_650M_UR50D model to use a non-autoregressive method to train the validation set x1,…,x k For decoding, the word segment at each position is predicted as That is, the word with the highest probability at each position. Get the predicted sequence above Then, the position j of the first non-amino acid word is searched from the back to the front, and the final signal peptide sequence is obtained as

[0213] S2. Calculation The predicted signal peptide sequence Compared with the original signal peptide sequence (x k-s+1 ,…,x k ) to perform sequence alignment, counting the number of identical amino acid residues i and the total length of the aligned sequences n, the sequence recovery rate (SRR) is:

[0214] S3. The sequence recovery rates of the model without fine-tuning and the model with fine-tuning are as follows.

[0215] Pre-training Fine-tuning SRR 22.7％ 45.0％

[0216] The present application provides embodiments of corresponding signal peptide prediction devices. For the beneficial effects or technical problems solved by the devices, please refer to the descriptions in the methods corresponding to each device, or refer to the descriptions in the content of the invention, which will not be repeated here.

[0217] like Figure 4 As shown, in the example of the signal peptide prediction device provided in the ninth embodiment of the present application, the device includes:

[0218] The acquisition unit is used to acquire the target protein sequence.

[0219] A processing unit is used to input a first input sequence generated by the target protein sequence into a prediction model to obtain a predicted signal peptide sequence; the first input sequence includes a context representing the target protein sequence; wherein the prediction model is obtained by training a pre-trained language model using a training set, and the training set includes a second input sequence; the second input sequence includes a context representing a signal peptide sequence and a context representing a protein sequence.

[0220] The present application provides corresponding embodiments of a device for constructing a prediction model. For the beneficial effects or technical problems solved by the device, please refer to the description of the methods corresponding to each device, or refer to the description in the content of the invention, which will not be repeated here.

[0221] like Figure 5 As shown, in the example of a device for constructing a prediction model provided in the tenth embodiment of the present application, the device includes:

[0222] The acquisition unit is used to acquire a sequence pair consisting of a signal peptide sequence and a protein sequence.

[0223] The sequence generating unit is used to generate a second input sequence according to the sequence pair consisting of the signal peptide sequence and the protein sequence; the second input sequence includes an upper context representing the signal peptide sequence and a lower context representing the protein sequence.

[0224] A training unit is used to train the pre-trained language model using a training set to obtain a prediction model, wherein the training set includes the second input sequence.

[0225] Embodiments of the computing device of the present application

[0226] Figure 6 1 is a schematic structural diagram of a computing device 900 provided in the eleventh embodiment of the present application. The computing device 900 includes: a processor 910 , a memory 920 , and may also include a communication interface 930 .

[0227] It should be understood that Figure 6 The communication interface 930 in the computing device 900 shown in FIG. 9 may be used to communicate with other devices.

[0228] The processor 910 may be connected to a memory 920. The memory 920 may be used to store the program code and data. Therefore, the memory 920 may be a storage unit inside the processor 910, or an external storage unit independent of the processor 910, or a component including a storage unit inside the processor 910 and an external storage unit independent of the processor 910.

[0229] Optionally, the computing device 900 may further include a bus. The memory 920 and the communication interface 930 may be connected to the processor 910 via a bus. The bus may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc.

[0230] It should be understood that in the embodiment of the present application, the processor 910 may adopt a central processing unit (CPU). The processor may also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. Alternatively, the processor 910 may adopt one or more integrated circuits to execute relevant programs to implement the technical solutions provided in the embodiment of the present application.

[0231] The memory 920 may include a read-only memory and a random access memory, and provides instructions and data to the processor 910. A portion of the processor 910 may also include a nonvolatile random access memory. For example, the processor 910 may also store information on the device type.

[0232] When the computing device 900 is running, the processor 910 executes the computer-executable instructions in the memory 920 to perform the operating steps of the above method.

[0233] It should be understood that the computing device 900 according to the embodiment of the present application can correspond to the corresponding subjects in the methods according to the embodiments of the present application, and the above-mentioned and other operations and / or functions of each module in the computing device 900 are respectively for implementing the corresponding processes of each method of the present embodiment. For the sake of brevity, they will not be repeated here.

[0234] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0235] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0236] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0237] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0238] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0239] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0240] An embodiment of the present application also provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the program is used to execute a method for generating diversified questions, which includes at least one of the solutions described in the above embodiments.

[0241] The computer storage medium of the embodiment of the present application can adopt any combination of one or more computer-readable media. Computer-readable media can be computer-readable signal media or computer-readable storage media. Computer-readable storage media can be, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or devices, or any combination of the above. More specific examples (non-exhaustive lists) of computer-readable storage media include: electrical connections with one or more wires, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In this document, computer-readable storage media can be any tangible medium containing or storing programs, which can be used by instruction execution systems, devices or devices or used in combination with them.

[0242] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0243] The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0244] Computer program code for performing the operation of the present application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer, partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., using an Internet service provider to connect through the Internet).

[0245] In addition, the words "first, second, third, etc." or module A, module B, module C and other similar terms in the specification and claims are only used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that the specific order or sequence can be interchanged where permitted so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0246] In the above description, the numbers representing the steps, such as S110, S120, etc., do not necessarily mean that the steps will be executed accordingly. If permitted, the order of the steps can be interchanged or they can be executed simultaneously.

[0247] The term "comprising" as used in the description and claims should not be interpreted as being limited to what is listed thereafter; it does not exclude other elements or steps. Therefore, it should be interpreted as specifying the presence of the features, integers, steps or components mentioned, but does not exclude the presence or addition of one or more other features, integers, steps or components and groups thereof. Therefore, the expression "a device comprising means A and B" should not be limited to a device consisting of components A and B only.

[0248] References to "one embodiment" or "an embodiment" in this specification mean that a particular feature, structure, or characteristic described in conjunction with the embodiment is included in at least one embodiment of the present application. Therefore, the phrases "in one embodiment" or "in an embodiment" appearing in various places in this specification do not necessarily refer to the same embodiment, but may refer to the same embodiment. In addition, in one or more embodiments, the particular features, structures, or characteristics can be combined in any appropriate manner, as would be apparent to one of ordinary skill in the art from this disclosure.

[0249] Note that the above are only preferred embodiments of the present application and the technical principles used. Those skilled in the art will understand that the present application is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present application. Therefore, although the present application is described in more detail through the above embodiments, the present application is not limited to the above embodiments, and may also include more other equivalent embodiments without departing from the concept of the present application, all of which belong to the scope of protection of the present application.< / eos> < / bos> < / eos> < / bos> < / eos> < / bos> < / mask> < / mask> < / pad> < / pad> < / eos> < / bos> < / eos> < / bos> < / eos> < / bos> < / pad> < / mask> < / mask> < / mask> < / pad> < / pad> < / eos> < / bos> < / eos> < / bos> < / mask> < / mask> < / sep> < / eos> < / pad> < / pad> < / eos> < / bos> < / sep> < / mask> < / pad> < / eos> < / go> < / bos> < / sos>

Claims

1. A method for predicting a signal peptide, characterized in that: include: Obtain target protein sequence; Generate a first input sequence using the target protein sequence and input it into a prediction model to obtain a predicted signal peptide sequence; the first input sequence includes the following content representing the target protein sequence; The prediction model is obtained by training a pre-trained language model using a training set, wherein the training set includes a second input sequence; the second input sequence includes a context representing a signal peptide sequence and a context representing a protein sequence; The word length of the preceding text and the word length of the following text are respectively fixed lengths; The above in the second input sequence is generated by right-aligned padding or truncation using the signal peptide sequence; and / or, The following in the second input sequence is generated by padding or truncating the protein sequence based on left alignment; and / or, The following in the first input sequence is generated by padding or truncating the target protein sequence based on left alignment.

2. The method according to claim 1, characterized in that The pre-trained language model includes a pre-trained bidirectional masked language model.

3. The method according to claim 1 or 2, characterized in that: Also includes: A mask operation is performed on the above partial sequence in the second input sequence.

4. The method according to claim 1, characterized in that: The loss functions used to train the pre-trained language model include: ； in, Indicates the first A sequence, Indicates the actual word length of the signal peptide sequence corresponding to the above sequence, Indicates the word length of the set previous sequence.

5. The method according to claim 1, characterized in that The step of using the target protein sequence to generate a first input sequence and inputting it into a prediction model to obtain a predicted signal peptide sequence comprises: generating the following text representing the target protein sequence according to the target protein sequence; Adding the preceding context represented by the mask before the following context to form the first input sequence; The first input sequence is input into the prediction model to obtain a predicted signal peptide sequence.

6. The method according to claim 1, characterized in that include: Obtaining a sequence pair consisting of a signal peptide sequence and a protein sequence; Generate a second input sequence according to the sequence pair consisting of the signal peptide sequence and the protein sequence; The second input sequence includes an upper part representing a signal peptide sequence and a lower part representing a protein sequence; The pre-trained language model is trained using a training set to obtain a prediction model, wherein the training set includes the second input sequence.

7. A method for predicting a signal peptide, characterized in that: include: Obtain target protein sequence; Generate the following text representing the target protein sequence according to the target protein sequence to obtain a first input sequence; Inputting the first input sequence into a prediction model to obtain a predicted signal peptide sequence; The prediction model is obtained by training a pre-trained language model using a training set, wherein the training set includes a second input sequence; the second input sequence includes a context representing a signal peptide sequence and a context representing a protein sequence, and a partial sequence in the context representing the signal peptide sequence is masked; The loss functions used to train the pre-trained language model include: ； in, Indicates the first A sequence, Indicates the actual word length of the signal peptide sequence corresponding to the above sequence, Indicates the length of the word in the set context sequence; The pre-trained language model includes a pre-trained bidirectional masked language model; The word length of the preceding text and the word length of the following text are respectively fixed lengths; The above in the second input sequence is generated by right-aligned padding or truncation using the signal peptide sequence; and / or, The following in the second input sequence is generated by padding or truncating the protein sequence based on left alignment; and / or, The following in the first input sequence is generated by padding or truncating the target protein sequence based on left alignment.

8. A device for executing the method for predicting a signal peptide according to any one of claims 1 to 6, characterized in that: include: An acquisition unit, used for acquiring a target protein sequence; A processing unit, configured to generate a first input sequence from the target protein sequence and input it into a prediction model to obtain a predicted signal peptide sequence; the first input sequence includes a subtext representing the target protein sequence; The prediction model is obtained by training a pre-trained language model using a training set, and the training set includes a second input sequence; the second input sequence includes a context representing a signal peptide sequence and a context representing a protein sequence.

9. A device for constructing a prediction model, characterized in that: The prediction model executes the method according to any one of claims 1 to 6, and the device comprises: An acquisition unit, used for acquiring a sequence pair consisting of a signal peptide sequence and a protein sequence; A sequence generating unit, used to generate a second input sequence according to the sequence pair consisting of the signal peptide sequence and the protein sequence; the second input sequence includes an upper context representing the signal peptide sequence and a lower context representing the protein sequence; A training unit is used to train the pre-trained language model using a training set to obtain a prediction model, wherein the training set includes the second input sequence.

10. A computing device, characterized in that include: processor, and A memory having program instructions stored thereon, wherein when the program instructions are executed by the processor, the processor executes the method according to any one of claims 1 to 6, or when the program instructions are executed by the processor, the processor executes the method according to claim 7.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is suitable for being loaded and executed by a processor, so that a computer device having the processor executes the method according to any one of claims 1 to 6 or the method according to claim 7.

Images