Protein sequence prediction method, device and equipment based on large language model and storage medium

Abstract

The invention provides a protein sequence prediction method and device based on a large language model, equipment and a storage medium, and the method comprises the steps: training an initial large language model based on a first sample data set, obtaining an intermediate large language model, generating a second sample data set according to original base sequence data, and obtaining a second sample data set; and training the intermediate large language model according to the second sample data set to obtain a target large language model, and inputting the to-be-predicted germline and the specified protein framework into the target large language model to obtain a protein sequence corresponding to the specified protein framework under the to-be-predicted germline. According to the method, the germline and the protein framework of the protein sequence are taken as prompt information, so that the finally obtained target large language model can generate the protein sequence by taking the germline and the protein framework as conditions, and the requirement of a user on generating the protein sequence by taking the germline as the condition is met.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a protein sequence prediction method, apparatus, device and storage medium based on a large language model. Background Art

[0002] Nanobodies (Nb) are of great significance for diagnosis, treatment and molecular research, and the construction of diverse and high-quality nanobody libraries has become increasingly important. The traditional method of constructing a synthetic library of nanobodies usually uses a certain protein framework as a template and manually formulates rules to design the sequence of the CDR3 region. Although this method is effective, it is usually time-consuming and has many constraints, which limits the speed of discovery and development.

[0003] In order to increase the speed of building a nano antibody library, a deep learning method can also be introduced. In the prior art, a protein large language model can be used to construct antibody sequences, thereby generating a nano antibody library.

[0004] However, when applying the current large protein language model, users cannot select the required germline, so the generated results cannot meet the diverse needs of users. Summary of the invention

[0005] The purpose of the present application is to provide a protein sequence prediction method, apparatus, device and storage medium based on a large language model to address the deficiencies in the above-mentioned prior art, so as to solve the problem in the prior art that users cannot select the required species and thus the generated results cannot meet the diverse needs of users.

[0006] To achieve the above objectives, the technical solutions adopted in this application are as follows:

[0007] In a first aspect, the present application provides a protein sequence prediction method based on a large language model, the method comprising:

[0008] Training an initial large language model based on a first sample data set to obtain an intermediate large language model, wherein the first sample data set includes a plurality of first protein sequences;

[0009] Generate a second sample data set according to the original base sequence data, wherein the second sample data set includes second protein sequences of multiple germ lines, and the prompt information of the second protein sequence includes: the germ line to which the second protein sequence belongs and the protein framework of the second protein sequence;

[0010] Training the intermediate large language model according to the second sample data set to obtain a target large language model;

[0011] The germline to be predicted and the designated protein framework are input into the target large language model to obtain the protein sequence corresponding to the designated protein framework under the germline to be predicted.

[0012] Optionally, generating a second sample data set according to the original base sequence data includes:

[0013] The original base sequence is translated to obtain candidate protein sequences of multiple germ lines;

[0014] Data preprocessing is performed on the candidate protein sequences of the plurality of germ lines to obtain the second protein sequence.

[0015] Optionally, the original base sequence is translated to obtain multiple candidate protein sequences of germ lines, including:

[0016] Translating each of the original base sequences based on a preset coding frame to obtain at least one initial protein sequence corresponding to each of the original base sequences;

[0017] Perform germline identification on each of the initial protein sequences based on an antibody sequence recognition model to obtain the germline and protein framework of each of the initial protein sequences;

[0018] The candidate protein sequence is generated according to the germline of the initial protein sequence, the protein framework of the initial protein sequence and the initial protein sequence.

[0019] Optionally, generating the candidate protein sequence according to the germline of the initial protein sequence, the protein framework of the initial protein sequence and the initial protein sequence comprises:

[0020] Preset identifiers are added to both ends of the initial protein sequence, and the germline of the initial protein sequence and the protein framework of the initial protein sequence are added to the initial protein sequence as prompt information of the initial protein sequence to obtain the candidate protein sequence, wherein the preset identifier is used to characterize that the initial protein sequence is a protein sequence, and the prompt information is a natural language.

[0021] Optionally, performing data preprocessing on the protein sequences of the plurality of germ lines to obtain the second protein sequence comprises:

[0022] Screening the candidate protein sequences according to their germline and length to obtain at least one screened sequence;

[0023] The similarity between each of the screened sequences and the standard protein sequence is determined, and the screened sequences with a similarity greater than a preset threshold are used as the second protein sequence.

[0024] Optionally, the screening of the candidate protein sequences according to the germline and length of each candidate protein sequence to obtain at least one screened sequence comprises:

[0025] Comparing the germline of each candidate protein sequence with the target germline, if the germline of the candidate protein sequence is the same as the target germline, retaining the candidate protein sequence, otherwise, deleting the candidate protein sequence;

[0026] The retained candidate protein sequences are screened based on a preset length interval to determine at least one screened sequence whose sequence length is within the length interval.

[0027] Optionally, the intermediate large language model comprises at least: a decoder, wherein the decoder comprises a plurality of layers;

[0028] The step of training the intermediate large language model according to the second sample data set to obtain a nanobody large language model comprises:

[0029] Adding a low-rank adapter to at least one level of the decoder, wherein the low-rank adapter is used to determine target parameters that need to be trained in the target level;

[0030] The intermediate large language model is trained based on the second sample data set to adjust the target parameters in each of the target levels to obtain the target large language model.

[0031] In a second aspect, the present application provides a protein sequence prediction device based on a large language model, the device comprising:

[0032] A first training module is used to train an initial large language model based on a first sample data set to obtain an intermediate large language model, wherein the first sample data set includes a plurality of first protein sequences;

[0033] A sample generation module is used to generate a second sample data set according to the original base sequence data, wherein the second sample data set includes second protein sequences of multiple germ lines, and the prompt information of the second protein sequence includes: the germ line to which the second protein sequence belongs and the protein framework of the second protein sequence;

[0034] A second training module, used for training the intermediate large language model according to the second sample data set to obtain a target large language model;

[0035] The sequence generation module is used to input the germline to be predicted and the specified protein framework into the target large language model to obtain the protein sequence corresponding to the specified protein framework under the germline to be predicted.

[0036] Optionally, the sample generation module is specifically used to:

[0037] The original base sequence is translated to obtain candidate protein sequences of multiple germ lines;

[0038] Data preprocessing is performed on the candidate protein sequences of the plurality of germ lines to obtain the second protein sequence.

[0039] Optionally, the sample generation module is specifically used to:

[0040] Translating each of the original base sequences based on a preset coding frame to obtain at least one initial protein sequence corresponding to each of the original base sequences;

[0041] Perform germline identification on each of the initial protein sequences based on an antibody sequence recognition model to obtain the germline and protein framework of each of the initial protein sequences;

[0042] The candidate protein sequence is generated according to the germline of the initial protein sequence, the protein framework of the initial protein sequence and the initial protein sequence.

[0043] Optionally, the sample generation module is specifically used to:

[0044] Preset identifiers are added to both ends of the initial protein sequence, and the germline of the initial protein sequence and the protein framework of the initial protein sequence are added to the initial protein sequence as prompt information of the initial protein sequence to obtain the candidate protein sequence, wherein the preset identifier is used to characterize that the initial protein sequence is a protein sequence, and the prompt information is a natural language.

[0045] Optionally, the sample generation module is specifically used to:

[0046] Screening the candidate protein sequences according to their germline and length to obtain at least one screened sequence;

[0047] The similarity between each of the screened sequences and the standard protein sequence is determined, and the screened sequences with a similarity greater than a preset threshold are used as the second protein sequence.

[0048] Optionally, the sample generation module is specifically used to:

[0049] Comparing the germline of each candidate protein sequence with the target germline, if the germline of the candidate protein sequence is the same as the target germline, retaining the candidate protein sequence, otherwise, deleting the candidate protein sequence;

[0050] The retained candidate protein sequences are screened based on a preset length interval to determine at least one screened sequence whose sequence length is within the length interval.

[0051] Optionally, the intermediate large language model comprises at least: a decoder, wherein the decoder comprises a plurality of layers;

[0052] The second training module is specifically used for:

[0053] Adding a low-rank adapter to at least one level of the decoder, wherein the low-rank adapter is used to determine target parameters that need to be trained in the target level;

[0054] The intermediate large language model is trained based on the second sample data set to adjust the target parameters in each of the target levels to obtain the target large language model.

[0055] In a third aspect, an embodiment of the present application further provides an electronic device, comprising: a processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the storage medium via the bus, and the processor executes the machine-readable instructions to perform the steps of the protein sequence prediction method based on a large language model as described in any one of the first aspects.

[0056] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the protein sequence prediction method based on a large language model as described in any one of the first aspects are executed.

[0057] The beneficial effect of the present application is that by using the germline and protein framework of the protein sequence as prompt information, the target large language model finally obtained can generate protein sequences based on the germline and protein framework, thereby meeting the user's demand for generating protein sequences based on the germline. When subsequently constructing an antibody library based on the protein sequence, a diversified and high-quality nano antibody library can also be constructed by generating protein sequences based on the germline.

[0058] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0060] Figure 1 A flowchart of a protein sequence prediction method based on a large language model provided in an embodiment of the present application is shown;

[0061] Figure 2 A flow chart for obtaining a second protein sequence provided in an embodiment of the present application is shown;

[0062] Figure 3 A flowchart for generating a protein sequence to be selected provided in an embodiment of the present application is shown;

[0063] Figure 4 Another flow chart for obtaining a second protein sequence provided in an embodiment of the present application is shown;

[0064] Figure 5 A flow chart of obtaining a screened sequence provided in an embodiment of the present application is shown;

[0065] Figure 6 A schematic diagram of the architecture of a large language model provided in an embodiment of the present application is shown;

[0066] Figure 7 A flowchart of a training target large language model provided in an embodiment of the present application is shown;

[0067] Figure 8 A schematic diagram of the structure of a protein sequence prediction device based on a large language model provided in an embodiment of the present application is shown;

[0068] Fig. 9 A schematic structural diagram of an electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0069] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0070] It should be noted that the term "comprising" will be used in the embodiments of the present application to indicate the existence of the features declared thereafter, but does not exclude the addition of other features.

[0071] In the prior art, when constructing a nanobody library, a protein large language model can be used to generate protein sequences. However, in actual application, the current protein large language model cannot generate protein sequences based on germline and formulation framework. In order to construct a diverse and high-quality nanobody library, how to generate protein sequences of a specified germline has become an urgent problem to be solved.

[0072] Based on this, the present application proposes a protein sequence prediction method based on a large language model. By training the model based on the germline, the final large language model generates a protein sequence that meets the user's requirements based on the specified germline and framework.

[0073] The method of the present application can be applied to the scenario of establishing a nanoantibody library. Nanoantibody is a protein with a special structure. When training the model, the model can be trained based on the protein sequence corpus and the nanoantibody sequence corpus, so that the model has the ability to generate nanoantibodies, thereby realizing the construction of a nanoantibody library based on a large language model.

[0074] The method of the present application can be used to train a protein sequence generation or recognition model based on germline conditions, such as training a large language model to generate specific germline protein sequences, or a large language model for training to recognize the germline of protein sequences. The specific application scenarios are not limited in this application.

[0075] Next, combine Figure 1 The protein sequence prediction method based on the large language model of the present application is described, and the execution subject of the method can be an electronic device. Figure 1 As shown, the method includes:

[0076] S101. Train an initial large language model based on a first sample data set to obtain an intermediate large language model, wherein the first sample data set includes a plurality of first protein sequences.

[0077] The first sample data set may be a non-redundant protein sequence database, such as the UniRef (Universal Protein Resource) database, which includes multiple data subsets, each of which includes multiple first protein sequences, and the similarity of the first protein sequences of each data subset is within a preset similarity interval.

[0078] The initial large language model can be a general large language model, or a large language model that can perform multi-task protein language processing, such as the initial LLaMA2 (Large Language Model Meta AI) framework.

[0079] The initial large language model is trained based on the first sample data set, and the pre-trained LLaMA2 framework is trained on the first sample data set to obtain the intermediate large language model.

[0080] The intermediate large language model can predict the properties of protein sequences and can also generate unconditional or controllable protein sequences.

[0081] S102. Generate a second sample data set based on the original base sequence data, wherein the second sample data set includes second protein sequences of multiple species, and the prompt information of the second protein sequence includes: the species to which the second protein sequence belongs and the protein framework of the second protein sequence.

[0082] The second sample data set includes a plurality of second protein sequences of germ lines, and the second protein sequence may be a nanobody sequence. The original base sequence data may be a base sequence data set of a nanobody, and a protein sequence may be obtained after translating the base sequence. In the present application, the second protein sequence of the nanobody may be obtained by translating the base sequence of the nanobody.

[0083] Each second protein sequence corresponds to a piece of prompt information, and the prompt information is used to characterize the germline to which the protein sequence belongs and the protein framework of the protein sequence.

[0084] The framework sequence of a protein refers to the primary structure, secondary structure, tertiary structure and quaternary structure of the protein. These structural levels together define the complete framework and function of the protein. The primary structure refers to the order of amino acids in the protein polypeptide chain, the secondary structure refers to the regular structure formed by the coiling and folding of the polypeptide chain skeleton, the tertiary structure is the three-dimensional conformation of the entire polypeptide chain, and the quaternary structure refers to the polymer structure formed by polypeptide chains with independent tertiary structures connected to each other by non-covalent bonds.

[0085] S103. Train the intermediate large language model according to the second sample data set to obtain a target large language model.

[0086] The intermediate large language model is trained according to the second sample data set, and model parameters of the intermediate large language model are adjusted based on the second sample data set to obtain a target large language model.

[0087] When adjusting the model parameters, in order to further improve the efficiency of model training, model training can be performed based on a low-rank adapter. Since the pre-trained intermediate large language model has a lower intrinsic dimension, it can still be effectively learned even if it is randomly projected to a smaller subspace. Based on this, the weight matrix of the intermediate large model can be decomposed into a low-rank, and only some parameters in the model can be adjusted, thereby reducing the computing resources and storage space required for model training.

[0088] As a possible implementation method, when a model needs to be trained to generate protein sequences, the germline and protein framework in the prompt information can be used as the input of the intermediate large language model, and the intermediate large language model can predict the protein sequence. The loss value is calculated based on the predicted protein sequence and the second protein sequence, and the intermediate large language model is iteratively optimized to obtain the final target large language model.

[0089] As another possible implementation method, when it is necessary to train a model to identify the germline of a protein sequence, the protein sequence can be used as the input of the intermediate large language model, and the intermediate large language model predicts the germline of the protein sequence. The loss value is calculated based on the predicted germline of the protein sequence and the germline of the second protein sequence, and the intermediate large language model is iteratively optimized to obtain the final target large language model.

[0090] S104, inputting the germline to be predicted and the specified protein framework into the target large language model to obtain the protein sequence corresponding to the specified protein framework under the germline to be predicted.

[0091] The germline to be predicted can be a protein germline specified by the user. For example, the user can specify germline IGHV3S53*01 and germline IGHV3_3*01 as the germline to be predicted, and the target large language model can generate a protein sequence under the germline to be predicted.

[0092] The specified protein framework may be a partial protein framework specified by the user or the entire protein framework specified by the user. For example, the user may specify the primary structure of the protein, so that the target large language model only predicts the primary structure of the protein and predicts the arrangement order of amino acids in the protein polypeptide chain.

[0093] It should be noted that the user can specify only the germline for which the protein sequence needs to be generated, thereby generating the protein sequence of the specified germline. It is also possible to specify the germline and protein framework, thereby generating the protein sequence of the specified protein framework under the specified germline. In order to improve the availability of the final constructed nanobody library, a framework with a high degree of humanization and few risk points can be used as the specified protein framework.

[0094] After obtaining multiple protein sequences corresponding to multiple specified protein frameworks under multiple germlines to be predicted, all protein sequences can be statistically analyzed, the frequency of each amino acid site can be counted, and a nano antibody library can be made based on the germline, framework, and amino acid frequency. Traditional library construction methods rely on multi-step laboratory operations, including single-chain antibody collection, vector preparation, insertion of single-chain antibody sequences, etc. These steps are not only time-consuming but also costly. This application quickly generates a large number of high-quality protein sequences through a target large language model, and constructs a nano antibody library based on the generated protein sequences, which can significantly reduce the time and resource consumption of library construction.

[0095] In an embodiment of the present application, the initial large language model is trained based on the first sample data set to obtain an intermediate large language model, the first sample data set includes multiple first protein sequences, and the second sample data set is generated according to the original base sequence data, and the second sample data set includes multiple second protein sequences of germ lines, and the prompt information of the second protein sequence includes: the germ line to which the second protein sequence belongs and the protein framework of the second protein sequence, and the intermediate large language model is trained according to the second sample data set to obtain a target large language model, and the germ line to be predicted and the specified protein framework are input into the target large language model to obtain the protein sequence corresponding to the specified protein framework under the germ line to be predicted. The present application uses the germ line and protein framework of the protein sequence as prompt information, so that the target large language model finally obtained can generate protein sequences with germ line and protein framework as conditions, thereby meeting the user's demand for generating protein sequences with germ line as conditions. When constructing an antibody library based on protein sequence in the future, a diversified and high-quality nano antibody library can also be constructed by generating protein sequences with germ line as conditions.

[0096] The following is a further description of generating the second sample data set based on the original base sequence data. Figure 2As shown, the above step S102 includes:

[0097] S201. Translate the original base sequence to obtain candidate protein sequences of multiple species.

[0098] Optionally, the original base sequence can be translated based on the code frame to translate the base sequence into a protein sequence. The original base sequence data can be base sequence data of multiple species, and the data is stored in fasta format. Through the code frame, every 3 bases can be translated into 1 amino acid, and a sequence composed of all amino acids obtained by translating a base sequence is used as a candidate protein sequence.

[0099] S202, performing data preprocessing on candidate protein sequences of multiple species to obtain a second protein sequence.

[0100] It should be understood that the candidate protein sequences obtained by translating the original base sequence data may include germlines that are not needed for this training. Therefore, the candidate protein sequences can be preprocessed to screen out the germlines needed for this training and retain the germlines that are needed.

[0101] Furthermore, the above step of translating the original base sequence to obtain multiple candidate protein sequences is as follows: Figure 3 As shown, including:

[0102] S301. Translate each original base sequence based on a preset coding frame to obtain at least one initial protein sequence corresponding to each original base sequence.

[0103] The number of preset code frames can be multiple, including a code frame that can be correctly translated for making positive samples, and other code frames for making negative samples. The code frame can perform forward translation and reverse translation on the original base sequence to obtain multiple initial protein sequences after the original base sequence is translated by each code frame.

[0104] For example, assuming that there are 6 codon frames, after translating the original base sequence based on the codon frames, 6 initial protein sequences corresponding to the original base sequence can be obtained.

[0105] S302, performing germline identification on each initial protein sequence based on an antibody sequence recognition model to obtain the germline and protein framework of each initial protein sequence.

[0106] In one possible implementation, the antibody sequence recognition model can recognize protein sequences of a specified germline and a specified protein framework. After the initial protein sequence is input into the antibody sequence recognition model in sequence, the model can recognize the initial protein sequence that is consistent with the specified germline and the specified protein framework.

[0107] In another possible implementation, the antibody sequence recognition model can identify the germline and protein framework of all initial protein sequences. Finally, based on the germline and protein framework identified by the antibody sequence recognition model, screening is performed to obtain the candidate protein sequences of the specified germline and specified protein framework required for this training.

[0108] S303, generating a protein sequence to be selected according to the germline of the initial protein sequence, the protein framework of the initial protein sequence, and the initial protein sequence.

[0109] Optionally, the germline of the initial protein sequence and the protein framework of the initial protein sequence may be used as hint information of the initial protein sequence, and the hint information is added to the initial protein sequence to obtain a protein sequence to be selected.

[0110] Furthermore, the process of generating a candidate protein sequence according to the germline of the initial protein sequence, the protein framework of the initial protein sequence, and the initial protein sequence includes:

[0111] Preset identifiers are added to both ends of the initial protein sequence, and the germline of the initial protein sequence and the protein framework of the initial protein sequence are added to the initial protein sequence as prompt information of the initial protein sequence to obtain a candidate protein sequence. The preset identifier is used to characterize that the initial protein sequence is a protein sequence, and the prompt information is natural language.

[0112] By adding preset marks to both ends of the initial protein sequence, the large language model can correctly identify the protein sequence. For example, a mark "<" can be added to the front of the initial protein sequence, and a mark ">" can be added to the end of the initial protein sequence.

[0113] The germline of the protein sequence and the protein framework of the initial protein sequence can be added as hint information after the initial protein sequence corresponding to each hint information, or all hint information can be added after the initial protein sequence.

[0114] It should be noted that LLaMA2 is a natural language model, so the prompt information can be a natural language. After adding preset tags at both ends of the initial protein sequence, LLaMA2 can recognize the prompt information and the initial protein sequence at the same time. If the language recognized by the large language model used is another form of language, the prompt information can be a language type that the large language model can recognize.

[0115] The prompt information may also include task description information, which is used to describe the goal of model training. For example, the storage format of the protein sequence to be selected may be as shown in Table 1 below:

[0116] Table 1 Examples of protein sequence formats to be selected

[0117]

[0118] like Figure 4 As shown, it is a further description of performing data preprocessing on the protein sequences of multiple germ lines to obtain the second protein sequence. The above step S202 includes:

[0119] S401, screening the candidate protein sequences according to their germline and length to obtain at least one screened sequence.

[0120] In the present application, the target large language model can be used to generate protein sequences of a specified germline. When training the model, the germlines of the candidate protein sequences can be screened and the candidate protein sequences of the specified germline can be retained.

[0121] During model training, a sequence length that is too short or too long may affect the effect of model training. Based on this, the present application can screen the candidate protein sequences according to their sequence lengths, and use the candidate protein sequences that meet the preset length range under the specified germline as the screened sequences.

[0122] S402, determining the similarity between each screened sequence and the standard protein sequence, and taking the screened sequence with a similarity greater than a preset threshold as the second protein sequence.

[0123] The standard protein sequence can be the nanoantibody protein sequence that the model needs to generate. By comparing the screened sequence with the standard protein sequence and retaining the sequence with higher similarity, the similarity between the protein sequence generated by the model and the standard protein sequence can be improved.

[0124] The similarity between the sequence after screening and the standard protein sequence can be the sequence similarity. For example, assuming that the protein sequence of the V gene of the alpaca antibody germline is used as the standard protein sequence of the germline, the data with a similarity of more than 85% after alignment can be used as the second protein sequence.

[0125] The following is a further description of the screening process for the above-mentioned protein sequence to be selected to obtain at least one screened sequence, such as Figure 5 As shown, the above step S401 includes:

[0126] S501. Compare the germline of each candidate protein sequence with the target germline. If the germline of the candidate protein sequence is the same as the target germline, retain the candidate protein sequence; otherwise, delete the candidate protein sequence.

[0127] As a possible implementation method, the character string of the candidate protein sequence can be compared with the character string of the target germline. If the character strings are the same, the candidate protein sequence is retained; if the character strings are different, the candidate protein sequence is deleted.

[0128] The character strings being the same may mean that the similarity of the character strings is greater than a preset similarity threshold, and the character strings being different may mean that the similarity of the character strings is less than or equal to the preset similarity threshold.

[0129] S502: Screen the retained candidate protein sequences based on a preset length interval to determine at least one screened sequence whose sequence length is within the length interval.

[0130] The length interval may be predetermined, or may be determined based on the statistical result of the sequence length of the candidate protein sequence, for example, the preset interval where the median of the sequence length in the candidate protein sequence is located is used as the length interval.

[0131] For example, assuming that the sequence length of the protein sequence to be selected is distributed between 50aa-140aa, and the preset length interval is 105aa-135aa, then all sequences with a length less than 105aa and a length greater than 135aa can be removed.

[0132] It is worth noting that the integrity of the sequence can also be checked in this step, and incomplete candidate protein sequences can also be removed, ultimately obtaining multiple screened sequences.

[0133] There is no strict order relationship between the above step S501 and the above step S502. The candidate protein sequences may be screened based on a preset length interval first, and then the retained candidate protein sequences are compared with the standard protein sequence, and the candidate protein sequences with similarity greater than a preset threshold are retained as the screened sequences. The specific execution order is not limited in this application.

[0134] Next, the large language model of this application is described first. Figure 6 As shown in FIG. 1 , it is a schematic diagram of the architecture of an intermediate large language model provided in the present application. It should be understood that the models of the initial large language model, the intermediate large language model and the target large language model can all be Figure 6 Model architecture shown, with different model parameters.

[0135] Reference Figure 6The large language model includes a serialization module, a vectorization module, a decoder module and an output module. The serialization module divides the germline of the input second protein sequence into words or characters to form a token sequence. The token sequence is further serialized into a list or array and indexed through the corpus to map each token to a unique integer index. After the serialization module outputs the processing result to the vectorization module, the vectorization module can map the processing result into a real vector and input the real vector into the decoder module.

[0136] The decoder module includes multiple decoders, each of which includes a self-attention mechanism, a feedforward network, layer normalization, and an output layer. The decoder decodes the real vector to obtain a decoding result and sends the decoding result to the output module.

[0137] The output module includes a linear layer and a softmax function. The output module can normalize the decoding result to obtain a prediction result, which can be a protein sequence corresponding to the germline.

[0138] Based on the loss function, the loss value is calculated for the prediction result and the second protein sequence corresponding to the germline, and the loss value is updated through gradient backpropagation to update the model parameters. After multiple iterations, the target large language model can be obtained. The loss function can be a mean square error function.

[0139] Optionally, the intermediate large language model includes at least: a decoder, wherein the decoder includes multiple layers. Figure 6 ,The decoder module includes self-attention mechanism, feed-forward network, and multiple normalization layers.

[0140] The above process of training the intermediate large language model according to the second sample data set to obtain the nanobody large language model is as follows: Figure 7 As shown, including:

[0141] S701. Add a low-rank adapter to at least one level of a decoder module, where the low-rank adapter is used to determine target parameters that need to be trained in a target level.

[0142] The user can specify to add a low-rank adapter to at least one layer of the decoder module. Figure 6 The low-rank adapter LoRA (Low-Rank Adaptation) shown in Figure 1 can decompose the weight matrix of the model into two low-rank matrices, thereby greatly reducing the number of parameters that need to be updated.

[0143] Assumptions is the weight matrix of an intermediate large language model, where d and k represent the dimensions of input and output, respectively. The low-rank adapter can decompose the weight matrix W into two low-rank matrices and Where r<<d,k.

[0144] W=W0+A×B

[0145] Among them, W0 is the fixed weight matrix during pre-training, and A and B are low-rank matrices that need to be learned during fine-tuning. Through such a decomposition, only A and B need to be updated during fine-tuning without changing W0. A and B can be integrated into the original model using the above formula. At the same time, the low-rank adapter can prevent catastrophic forgetting of the original knowledge, because the rank of the newly learned knowledge is lower than the original knowledge, and finally SiLU can be used as the activation function. As an example, the configuration information of the low-rank adapter can be: rank = 64, alpha = 128, dropout = 0.05.

[0146] S702: Train the intermediate large language model based on the second sample data set to adjust target parameters in each target level to obtain a target large language model.

[0147] The germline and protein framework of the second protein sequence in the second sample data set are input into the intermediate large language model in sequence, and the intermediate large language model is used to perform predictions to obtain predicted protein sequences corresponding to the germline and protein framework, and a loss value is calculated based on the predicted protein sequence and the second protein sequence, and the target parameters in the target level in the model are adjusted according to the loss value, and the adjusted target parameters are saved.

[0148] After completing the adjustment of all target parameters, the final intermediate large language model can be used as the target large language model.

[0149] Based on the same inventive concept, the embodiments of the present application also provide a protein sequence prediction device based on a large language model corresponding to the protein sequence prediction method based on a large language model. Since the principle of solving the problem by the device in the embodiments of the present application is similar to the above-mentioned protein sequence prediction method based on a large language model in the embodiments of the present application, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0150] Figure 8 A schematic diagram of the structure of a protein sequence prediction device based on a large language model provided in an embodiment of the present application is shown.

[0151] A first training module 801 is used to train an initial large language model based on a first sample data set to obtain an intermediate large language model, wherein the first sample data set includes a plurality of first protein sequences;

[0152] The sample generation module 802 is used to generate a second sample data set according to the original base sequence data, wherein the second sample data set includes second protein sequences of multiple germ lines, and the prompt information of the second protein sequence includes: the germ line to which the second protein sequence belongs and the protein framework of the second protein sequence;

[0153] A second training module 803 is used to train the intermediate large language model according to the second sample data set to obtain a target large language model;

[0154] The sequence generation module 804 is used to input the germline to be predicted and the specified protein framework into the target large language model to obtain the protein sequence corresponding to the specified protein framework under the germline to be predicted.

[0155] Optionally, the sample generation module 802 is specifically used for:

[0156] The original base sequence is translated to obtain candidate protein sequences of multiple germ lines;

[0157] Data preprocessing is performed on candidate protein sequences of multiple germ lines to obtain a second protein sequence.

[0158] Optionally, the sample generation module 802 is specifically used for:

[0159] Translating each original base sequence based on a preset coding frame to obtain at least one initial protein sequence corresponding to each original base sequence;

[0160] Perform germline identification on each initial protein sequence based on the antibody sequence recognition model to obtain the germline and protein framework of each initial protein sequence;

[0161] A candidate protein sequence is generated according to the germline of the initial protein sequence, the protein framework of the initial protein sequence, and the initial protein sequence.

[0162] Optionally, the sample generation module 802 is specifically used for:

[0163] Preset identifiers are added to both ends of the initial protein sequence, and the germline of the initial protein sequence and the protein framework of the initial protein sequence are added to the initial protein sequence as prompt information of the initial protein sequence to obtain a candidate protein sequence. The preset identifier is used to characterize that the initial protein sequence is a protein sequence, and the prompt information is natural language.

[0164] Optionally, the sample generation module 802 is specifically used for:

[0165] Screening the candidate protein sequences according to the germline and length of each candidate protein sequence to obtain at least one screened sequence;

[0166] The similarity between each screened sequence and the standard protein sequence is determined, and the screened sequence with a similarity greater than a preset threshold is used as the second protein sequence.

[0167] Optionally, the sample generation module 802 is specifically used for:

[0168] Compare the germline of each candidate protein sequence with the target germline. If the germline of the candidate protein sequence is the same as the target germline, retain the candidate protein sequence; otherwise, delete the candidate protein sequence;

[0169] The retained candidate protein sequences are screened based on a preset length interval to determine at least one screened sequence whose sequence length is within the length interval.

[0170] Optionally, the intermediate large language model includes at least: a decoder, wherein the decoder includes a plurality of layers;

[0171] The second training module 803 is specifically used for:

[0172] Adding a low-rank adapter to at least one level of the decoder, the low-rank adapter being used to determine target parameters to be trained in the target level;

[0173] The intermediate large language model is trained based on the second sample data set to adjust target parameters in each target level to obtain a target large language model.

[0174] The embodiment of the present application uses the germline and protein framework of the protein sequence as prompt information, so that the final target large language model can generate protein sequences based on the germline and protein framework, thereby meeting the user's demand for generating protein sequences based on the germline. When subsequently constructing an antibody library based on the protein sequence, a diversified and high-quality nano antibody library can also be constructed by generating protein sequences based on the germline.

[0175] Fig. 9 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application is shown, including: a processor 901, a storage medium 902 and a bus 903, wherein the storage medium 902 stores machine-readable instructions executable by the processor 901. When the electronic device runs the protein sequence prediction method based on a large language model in the embodiment, the processor 901 communicates with the storage medium 902 through the bus 903, and the processor 901 executes the machine-readable instructions and the preamble of the method item of the processor 901 to execute the steps of the above-mentioned protein sequence prediction method based on a large language model.

[0176] An embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. The computer program is executed when a processor is running, and the processor executes the steps of the above-mentioned protein sequence prediction method based on a large language model.

[0177] In the embodiment of the present application, the computer program can also execute other machine-readable instructions when run by the processor to execute other methods described in the embodiment. For the specific execution method steps and principles, please refer to the description of the embodiment, which will not be repeated here.

[0178] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0179] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0180] In addition, each functional unit in the embodiments provided in the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0181] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0182] It should be noted that similar numbers and letters represent similar items in the following figures. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. In addition, the terms "first", "second", "third", etc. are only used to distinguish the description and are not to be understood as indicating or implying relative importance.

[0183] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solutions of the present application, rather than to limit them. The protection scope of the present application is not limited thereto. Although the present application is described in detail with reference to the above-mentioned embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-mentioned embodiments within the technical scope disclosed in the present application, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application. They should all be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A protein sequence prediction method based on a large language model, characterized in that: include: Training an initial large language model based on a first sample data set to obtain an intermediate large language model, wherein the first sample data set includes a plurality of first protein sequences; Generate a second sample data set according to the original base sequence data, wherein the second sample data set includes second protein sequences of multiple germ lines, and the prompt information of the second protein sequence includes: the germ line to which the second protein sequence belongs and the protein framework of the second protein sequence; Training the intermediate large language model according to the second sample data set to obtain a target large language model; The germline to be predicted and the designated protein framework are input into the target large language model to obtain the protein sequence corresponding to the designated protein framework under the germline to be predicted.

2. The method according to claim 1, characterized in that Generating a second sample data set according to the original base sequence data includes: The original base sequence is translated to obtain candidate protein sequences of multiple germ lines; Data preprocessing is performed on the candidate protein sequences of the plurality of germ lines to obtain the second protein sequence.

3. The method according to claim 2, characterized in that The original base sequence is translated to obtain multiple candidate protein sequences of germ lines, including: Translating each of the original base sequences based on a preset coding frame to obtain at least one initial protein sequence corresponding to each of the original base sequences; Perform germline identification on each of the initial protein sequences based on an antibody sequence recognition model to obtain the germline and protein framework of each of the initial protein sequences; The candidate protein sequence is generated according to the germline of the initial protein sequence, the protein framework of the initial protein sequence and the initial protein sequence.

4. The method according to claim 3, characterized in that: The step of generating the protein sequence to be selected according to the germline of the initial protein sequence, the protein framework of the initial protein sequence and the initial protein sequence comprises: Preset identifiers are added to both ends of the initial protein sequence, and the germline of the initial protein sequence and the protein framework of the initial protein sequence are added to the initial protein sequence as prompt information of the initial protein sequence to obtain the candidate protein sequence, wherein the preset identifier is used to characterize that the initial protein sequence is a protein sequence, and the prompt information is a natural language.

5. The method according to claim 2, characterized in that: The step of performing data preprocessing on the protein sequences of the plurality of germ lines to obtain the second protein sequence comprises: Screening the candidate protein sequences according to their germline and length to obtain at least one post-screening sequence; The similarity between each of the screened sequences and the standard protein sequence is determined, and the screened sequences with a similarity greater than a preset threshold are used as the second protein sequence.

6. The method according to claim 5, characterized in that The step of screening the candidate protein sequences according to their germline and length to obtain at least one screened sequence comprises: Comparing the germline of each candidate protein sequence with the target germline, if the germline of the candidate protein sequence is the same as the target germline, retaining the candidate protein sequence, otherwise, deleting the candidate protein sequence; The retained candidate protein sequences are screened based on a preset length interval to determine at least one screened sequence whose sequence length is within the length interval.

7. The method according to claim 1, characterized in that The intermediate large language model at least includes: a decoder, wherein the decoder includes multiple levels; The step of training the intermediate large language model according to the second sample data set to obtain a nanobody large language model comprises: Adding a low-rank adapter to at least one level of the decoder, wherein the low-rank adapter is used to determine target parameters that need to be trained in a target level; The intermediate large language model is trained based on the second sample data set to adjust the target parameters in each of the target levels to obtain the target large language model.

8. A protein sequence prediction device based on a large language model, characterized in that: include: A first training module is used to train an initial large language model based on a first sample data set to obtain an intermediate large language model, wherein the first sample data set includes a plurality of first protein sequences; A sample generation module is used to generate a second sample data set according to the original base sequence data, wherein the second sample data set includes second protein sequences of multiple germ lines, and the prompt information of the second protein sequence includes: the germ line to which the second protein sequence belongs and the protein framework of the second protein sequence; A second training module, used for training the intermediate large language model according to the second sample data set to obtain a target large language model; The sequence generation module is used to input the germline to be predicted and the specified protein framework into the target large language model to obtain the protein sequence corresponding to the specified protein framework under the germline to be predicted.

9. An electronic device, characterized in that: include: A processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor. When the electronic device is running, the processor and the storage medium communicate via the bus, and the processor executes the machine-readable instructions to perform the steps of the protein sequence prediction method based on a large language model as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the protein sequence prediction method based on a large language model as claimed in any one of claims 1 to 7 are executed.

Images