A method and related device for cyclically iterating optimization to generate fitness of sequence mutations

Abstract

The application relates to the field of protein engineering, and discloses a method for cyclically and iteratively optimizing generation of sequence mutation fitness and a related device, which comprises the following steps: obtaining homologous family sequence data of a target protein sequence, pre-processing the data to generate a first training set; constructing a generative adversarial network model, and pre-training the model by using the first training set; obtaining a protein mutation data set, encoding amino acid sequences in the data set by using different encoding modes to obtain a second training set and a test set; training and testing a plurality of regression models by using the second training set and the test set, calculating the test set correlation of the regression models corresponding to each encoding mode, and sorting the regression models; determining a predictor for optimizing protein sequence mutation fitness according to a sorting result; introducing the predictor into the generative adversarial network model, cyclically and iteratively updating model parameters, and maximizing mutation fitness of a generated sequence. In this way, a sequence with high mutation fitness can be directly generated.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of protein engineering, and in particular to a method, device, equipment and storage medium for cyclic iteration optimization of generating sequence mutation fitness. BACKGROUND

[0002] Proteins are the material basis of life and the main undertakers of life activities. Naturally occurring proteins often cannot meet people's needs, and designing protein sequences with clear functional characteristics has always been the long-term vision of protein engineering. The three-dimensional structure, functional characteristics and physicochemical properties of proteins are determined by the amino acid sequence, and amino acids are the basic units of proteins. Traditional biological experimental methods to obtain new protein sequences with enhanced characteristics often require a large number of biological experimental screening, such as random mutation, which has large randomness and low mutation frequency, and usually small changes in protein sequence will probably cause activity to decline or even inactivation. Traditional directed evolution starts from a protein with a certain desired function, and repeatedly undergoes several rounds of unit point saturation mutation and screening, while fixing the best variant, and iterating until the appropriate improvement level. However, traditional directed evolution only focuses on single mutation, ignores the interdependence between mutations, and also only explores the possible proteins in the vicinity of the natural protein, which has certain limitations.

[0003] In recent years, with the development of artificial intelligence technology, artificial intelligence technology has been applied to the field of protein engineering, and the use of generative models to generate new functional proteins to explore the vast protein sequence space has also been in preliminary development. The mutation fitness of a protein is used to measure the effect of mutation on the biological activity of a functional protein, and the improvement of mutation fitness can purposefully optimize the specific function of the protein. However, the existing protein sequence generation model does not consider generating sequences from the perspective of protein sequence mutation effect, so it cannot directly generate new sequences with high mutation fitness. In addition, the existing protein sequence generation model has poor diversity of generated results.

[0004] Therefore, how to ensure the generation of sequences with high mutation fitness and improve the richness of generated sequences is a technical problem to be solved by those skilled in the art. SUMMARY

[0005] Therefore, how to ensure the generation of sequences with high mutation fitness and improve the richness of generated sequences is a technical problem to be solved by those skilled in the art.

[0006] A method for cyclic iteration optimization of generating sequence mutation fitness, comprising:

[0007] obtaining homologous family sequence data of a target protein sequence, preprocessing the homologous family sequence data to generate a first training set;

[0008] constructing a generative adversarial network model, and pre-training the generative adversarial network model by using the first training set;

[0009] obtaining a protein mutation data set, encoding the amino acid sequence of each protein in the protein mutation data set by using different encoding methods to obtain a second training set and a test set under different encodings;

[0010] training and testing a plurality of regression models by using the second training set and the test set, calculating the test set correlation of the regression model corresponding to each encoding method, and sorting the regression model according to the correlation, and determining a predictor for optimizing the mutation fitness of the protein sequence according to the correlation sorting result;

[0011] introducing the predictor into the generative adversarial network model, and iteratively updating the parameters of the generative adversarial network model to maximize the mutation fitness of the generated sequence.

[0012] Preferably, in the above-mentioned method for iteratively optimizing the mutation fitness of the generated sequence provided in the embodiments of the present application, the preprocessing of the homologous family sequence data to generate the first training set comprises:

[0013] filtering the homologous family sequence data to filter out sequences containing atypical amino acids and super-long amino acid sequences;

[0014] clustering the filtered homologous family sequence data according to sequence identity by using the MMseq2 tool;

[0015] performing data balancing and expansion processing on unbalanced classes by using the NTA algorithm after clustering;

[0016] dividing the expanded data to generate the first training set.

[0017] Preferably, in the above-mentioned method for iteratively optimizing the mutation fitness of the generated sequence provided in the embodiments of the present application, the data balancing and expansion processing on unbalanced classes by using the NTA algorithm comprises:

[0018] using the degeneracy of codons to translate each amino acid sequence in the unbalanced class into a plurality of corresponding nucleotide sequences.

[0019] Preferably, in the above-mentioned method for iteratively optimizing the mutation fitness of the generated sequence provided in the embodiments of the present application, the construction of the generative adversarial network model and the pre-training of the generative adversarial network model by using the first training set comprises:

[0020] The generator and the discriminator, both of which are connected by multiple layers of residuals, are used to construct a generative adversarial network model based on a deep residual neural network;

[0021] The generative adversarial network model is pre-trained using the first training set, and the generator and the discriminator are cyclically and alternately updated during the training.

[0022] Preferably, in the above-mentioned method for cyclically and iteratively optimizing the fitness of sequence mutations provided in the embodiments of the present application, the correlation degrees of the test sets of the regression models corresponding to each encoding mode are calculated and sorted, and the prediction model for optimizing the fitness of protein sequence mutations is determined according to the sorting results of the correlation degrees, comprising:

[0023] The correlation degrees of the test sets of the regression models corresponding to each encoding mode are calculated;

[0024] For each encoding mode, the calculated correlation degrees are sorted from high to low, and the mean of the correlation degrees in the top set number is calculated;

[0025] The highest mean among all the means is found, and the optimal encoding mode and the corresponding regression model are determined according to the highest mean;

[0026] The determined encoding mode and regression model are combined as the prediction model for optimizing the fitness of protein sequence mutations.

[0027] Preferably, in the above-mentioned method for cyclically and iteratively optimizing the fitness of sequence mutations provided in the embodiments of the present application, the parameters of the generative adversarial network model are cyclically and iteratively updated, comprising:

[0028] A new protein sequence is randomly sampled by the generator of the generative adversarial network model;

[0029] The new protein sequence is predicted by the predictor, and the fitness loss is calculated;

[0030] The generator loss is updated using the fitness loss, the generative adversarial network model is fine-tuned, and the parameters of the generative adversarial network model are cyclically and iteratively updated.

[0031] Preferably, in the above-mentioned method for cyclically and iteratively optimizing the fitness of sequence mutations provided in the embodiments of the present application, the new protein sequence is predicted by the predictor, and the fitness loss is calculated, comprising:

[0032] The new protein sequence is encoded using the determined encoding mode;

[0033] The encoded protein sequence is predicted by the determined set number of regression models respectively, and the set number of predicted values is obtained.

[0034] The mean of the set number of prediction values is taken as a mutation fitness prediction value, and a fitness loss is calculated according to the mutation fitness prediction value.

[0035] The embodiment of the present application also provides a device for cyclic iteration optimization of sequence mutation fitness, comprising:

[0036] A data processing module is configured to acquire homologous family sequence data of a target protein sequence, pre-process the homologous family sequence data, and generate a first training set.

[0037] A model training module is configured to construct a generative adversarial network model, and pre-train the generative adversarial network model by using the first training set.

[0038] A data encoding module is configured to acquire a protein mutation data set, encode the amino acid sequence of each protein in the protein mutation data set by using different encoding modes, and obtain a second training set and a test set under different encodings.

[0039] A predictor determination module is configured to train and test a plurality of regression models by using the second training set and the test set, calculate the test set correlation of the regression model corresponding to each encoding mode and sort the test set correlation, and determine a predictor for optimizing the mutation fitness of the protein sequence according to the correlation sorting result.

[0040] A cyclic iteration module is configured to introduce the predictor into the generative adversarial network model, cyclically and iteratively update the parameters of the generative adversarial network model, and maximize the mutation fitness of the generated sequence.

[0041] The embodiment of the present application also provides a device for cyclic iteration optimization of sequence mutation fitness, comprising a processor and a memory, wherein the processor realizes the above-mentioned method for cyclic iteration optimization of sequence mutation fitness provided by the embodiment of the present application when executing the computer program stored in the memory.

[0042] The embodiment of the present application also provides a computer readable storage medium for storing a computer program, wherein the computer program is executed by a processor to realize the above-mentioned method for cyclic iteration optimization of sequence mutation fitness provided by the embodiment of the present application.

[0043] From the above technical solution can be seen, the method for generating sequence mutation fitness by cyclic iteration optimization provided by the application, comprising: obtaining homologous family sequence data of target protein sequence, preprocessing the homologous family sequence data to generate a first training set; constructing a generative adversarial network model, and pre-training the generative adversarial network model using the first training set; obtaining a protein mutation dataset, encoding the amino acid sequence of each protein in the protein mutation dataset using different encoding methods to obtain a second training set and a test set under different encodings; training and testing a plurality of regression models using the second training set and the test set, and calculating the test set correlation of the regression model corresponding to each encoding method and sorting, determining the predictor for optimizing the protein sequence mutation fitness according to the correlation sorting result; introducing the predictor into the generative adversarial network model, and cyclically updating the parameters of the generative adversarial network model to maximize the mutation fitness of the generated sequence.

[0044] The method for generating sequence mutation fitness by cyclic iteration optimization provided by the application first obtains homologous family sequence data of a target protein sequence, and pre-processes and divides the data set, then constructs and pre-trains a generative adversarial network model to fit the data distribution of the target sequence, further optimizes the protein sequence generation model from the perspective of mutation fitness, first obtains a protein mutation dataset, encodes the mutation sequence using different encoding methods, trains and tests a plurality of regression models, determines the predictor for optimizing the protein sequence mutation fitness according to the test set correlation sorting result, and finally introduces the predictor into the generative adversarial network model for further optimization training by cyclic iteration, which can effectively improve the fitness value of the generated sequence, thus retaining the original data distribution and adding fitness optimization, the generative adversarial network model optimized by cyclic iteration can directly generate sequences with high mutation fitness, overcoming the limitation that directed evolution can only perform mutation at a few sites around the natural protein, and further revealing the mystery of the protein sequence space and exploring the understanding of the mutation effect of the generation model, thereby opening up a new perspective for sequence generation.

[0045] In addition, the application also provides a corresponding device, equipment and computer readable storage medium for the method for generating sequence mutation fitness by cyclic iteration optimization, further making the above method more practical, and the device, equipment and computer readable storage medium have corresponding advantages. BRIEF DESCRIPTION OF DRAWINGS

[0046] In order to more clearly illustrate the technical solutions in the embodiments of the application or the related art, the following will briefly introduce the drawings needed to be used in the embodiment or related art description. Obviously, the drawings in the following description are only embodiments of the application, and for those skilled in the art, other drawings can be obtained without creative labor based on the provided drawings.

[0047] Figure 1 A flow chart of a method for generating sequence mutation fitness by iterative optimization is provided for an embodiment of the present application;

[0048] Figure 2 A schematic diagram of an architecture corresponding to the method for generating sequence mutation fitness by iterative optimization is provided for an embodiment of the present application;

[0049] Figure 3 A schematic diagram of a target protein sequence data distribution gradually fitted over the course of training is provided for an embodiment of the present application;

[0050] Figure 4 A structural schematic diagram of a device for generating sequence mutation fitness by iterative optimization is provided for an embodiment of the present application. DETAILED DESCRIPTION

[0051] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0052] The present application provides a method for generating sequence mutation fitness by iterative optimization, as shown in Figure 1 The method comprises the following steps:

[0053] S101, obtaining homologous family sequence data of a target protein sequence, pre-processing the homologous family sequence data to generate a first training set;

[0054] In actual application, the UniProt (Universal Protein) protein sequence database can be used to obtain the homologous family sequence data of the target protein sequence.

[0055] S102, constructing a generative adversarial network model, and pre-training the generative adversarial network model using the first training set;

[0056] The generative adversarial network model in step S102 can be constructed based on a deep residual neural network. The adversarial network model can comprise two networks, a generator and a discriminator, both of which use a two-dimensional convolution layer with a convolution kernel size of 3, a Leaky ReLU activation function, and a non-saturated loss function with R1 regularization. Spectral normalization is introduced in the network to stabilize the training process. After a certain step of pre-training, the data distribution of the target protein sequence can be fitted.

[0057] S103, obtain a protein mutation dataset, encode the amino acid sequence of each protein in the protein mutation dataset using different encoding methods, obtain a second training set and a test set under different encodings;

[0058] The protein mutation dataset obtained in step S103 can include the amino acid sequence X = {X1, X2, X3,..., X N} of the mutant protein and its corresponding mutation fitness label y = {y1, y2, y3,..., y N}, wherein X n represents the nth mutant protein sequence composed of 20 amino acids with a length of L n represents a vector space, represents the corresponding amino acid at the kth position of the nth protein, y n represents the mutation fitness label of X n , y n > 0 indicates that X n is a beneficial mutation with a mutation fitness better than the wild type sequence, y n < 0 indicates that X n is a harmful mutation with a mutation fitness lower than the wild type sequence.

[0059] The different encoding methods used in step S103 can include UniRep, UniRep+VAE (i.e., UniRep combined with VAE encoding), Transformer, esm series protein language models, protein pre-training models in ProtTrans, etc. Among them, the esm series protein language models include esm_t6, esm_t12, esm_t34_50S, esm_t34_50D, esm_t34_100, etc.; the protein pre-training models in ProtTrans include ProtAlbert, ProtXLNet, ProtT5_XL_BFD, ProtBert, ProtBert_BFD, etc.

[0060] The mutant protein sequence X is encoded into a feature vector of different dimensions using the above encoding methods The encoding methods are different, and the dimensions d of the feature vectors are also different.

[0061] ​It should be noted that UniRep adopts a 1900-dimensional protein feature representation model developed by the George Church laboratory based on bidirectional LSTM; UniRep+VAE represents the combination of 1900-dimensional UniRep encoding and 1-dimensional mutation effect prediction value generated by the variational autoencoder model trained by multiple sequence alignment information of the corresponding family sequence, so d is 1901; Transform adopts the pre-trained 768-dimensional learning embedding encoding in TAPE; esm is a protein language model, and the encoding dimensions are 768 for esm_t6, 768 for esm_t12, 1280 for esm_t34_50S, 1280 for esm_t34_50D, and 1280 for esm_t34_100; The rest are protein pre-training models in ProtTrans, and the dimensions are 4096 for ProtAlbert, 1024 for ProtXLNet, 1024 for ProtT5_XL_BFD, 1024 for ProtBert, and 1024 for ProtBert_BFD.

[0062] S104, train and test the plurality of regression models respectively by using the second training set and the test set, calculate the test set correlation of the regression model corresponding to each encoding mode and sort, and determine the predictor for optimizing the prediction of the protein sequence mutation fitness according to the correlation sorting result.

[0063] The regression model in step S104 can include: XGBTree (regression tree model in XGBoost), Linear (linear regression model), GradientBoosting (gradient boosting regression model), RandomForest (random forest regression model), ARD (automatic relevance determination regression model), KernelRidge (kernel ridge regression model), BayesianRidge (Bayesian linear regression model), Bagging (ensemble regression model), LassoLarsCV (cross-validation lasso regression), KNeighbors (K-nearest neighbor regression model), etc.

[0064] The present application selects a plurality of different encoding modes and a plurality of different regression models for permutation and combination, aiming to determine the optimal predictor, so as to achieve better results. The encoding mode and the regression model here can be any combination of other arbitrary encoding mode and arbitrary regression model.

[0065] In the step S104, the second training set under different encodings is used to train the regression model for each encoding mode. In each encoding mode, the correlation of the test set of different regression models is sorted, and the best encoding mode (i.e., the optimal encoding mode) can be determined according to the sorting result. According to the combination of the best encoding mode and the regression model corresponding to the encoding mode (e.g., the regression model with the top three correlation in the test set corresponding to the encoding mode), the predictor for optimizing the mutation fitness of the protein sequence can be determined.

[0066] S105, the predictor is introduced into the generative adversarial network model, and the parameters of the generative adversarial network model are updated in a loop iteration to maximize the mutation fitness of the generated sequence.

[0067] In the above-mentioned method for cyclic iteration and optimization of the mutation fitness of the generated sequence provided by the embodiments of the present application, the homologous family sequence data of the target protein sequence is first obtained, and preprocessed and divided into data sets, and then the generative adversarial network model is constructed and pre-trained to fit the data distribution of the target sequence. From the perspective of mutation fitness, the protein mutation data set is first obtained, and after different encoding of the mutation sequence, multiple regression models are trained and tested, and the predictor for optimizing the mutation fitness of the protein sequence is determined according to the sorting result of the test set correlation. Finally, after the predictor is introduced into the generative adversarial network model and further optimized in a loop iteration, the fitness value of the generated sequence can be effectively improved. In this way, the original data distribution is preserved, and the fitness optimization is added. The generative adversarial network model optimized in a loop iteration can directly generate sequences with high mutation fitness, overcoming the limitation of directed evolution that can only perform mutation at a few sites around the natural protein. Thus, the mystery of the protein sequence space is revealed, and the understanding of the mutation effect of the generative model is explored, opening up a new perspective for sequence generation.

[0068] Further, in the above-mentioned method for cyclic iteration and optimization of the mutation fitness of the generated sequence provided by the embodiments of the present application, the homologous family sequence data is preprocessed in step S101 to generate the first training set, which can specifically include: first, filtering the homologous family sequence data to filter out sequences containing atypical amino acids and super-long amino acid sequences; then, using the MMseq2 (many-to-many sequence search) tool to cluster the filtered homologous family sequence data according to sequence identity; after clustering, using the NTA (nucleotide augmentation, biological sequence data augmentation technology) algorithm to balance and expand the data of the unbalanced categories; finally, dividing the expanded data into the first training set and the validation set. Such preprocessing method can enhance the richness of the data of the protein sequence generation model.

[0069] In practical applications, the atypical amino acids include 'B', 'O', 'U', 'X', 'Z', and 'J'; the super-long amino acid sequence is a protein sequence with an amino acid number greater than 512. The unbalanced category contains a large proportion of a small number of sequence categories. When clustering, the MMseq2 tool can be used to cluster the protein family sequence data at 70% sequence identity. In addition, for homologous protein family sequence data, after clustering based on sequence identity, the unbalanced category problem will occur. In order to enhance the data set richness and alleviate the data imbalance problem, the NTA algorithm is used to balance and expand the data set in a biologically meaningful way to enhance the data and improve the richness of the training set.

[0070] In specific implementation, the NTA algorithm is used to balance and expand the data of the unbalanced category in the above steps, which can specifically include: using the degeneracy of the amino acid code, the amino acid sequence in the unbalanced category is translated into a plurality of corresponding codon nucleotide sequences. Specifically, by translating the protein sequence X={a1, a2, a3,..., a... a n},a i into a plurality of different nucleotide sequences , which represents the i-th amino acid translated into the j-th nucleotide in the codon. Each amino acid can be translated into 3 corresponding codon nucleotides, and the correspondence between amino acid and codon is one-to-many, so that the training data can be balanced in a biologically meaningful way, and the amount of information in the model training process is increased, further improving the richness of the generated sequence.

[0071] In the process of dividing the data into the first training set and the validation set, 20% of the data can be selected from each category as the validation set, and the remaining sequences are used as the first training set of the generative adversarial network model.

[0072] Further, in the above-mentioned cyclic iteration optimization method for generating sequence mutation fitness provided by the embodiments of the present application, step S102 constructs a generative adversarial network model and pre-trains the generative adversarial network model using the first training set, which can specifically include: first, a generator and a discriminator both connected by multiple layers of residual connections are used to construct a generative adversarial network model based on a deep residual neural network; then, the generative adversarial network model is pre-trained using the first training set, and the generator and the discriminator are updated alternately in the training. In addition, the pre-trained generative adversarial network model can be evaluated and verified using the validation set.

[0073] Preferably, the generator can adopt 6 layers of residual connection, the discriminator can adopt 5 layers of residual connection, and the step length of the generator and the discriminator in pre-training can be 1:1. The number of residual layers of the generator and the discriminator and the step length can be determined according to actual conditions, and are not limited herein.

[0074] Further, in the above-mentioned method for generating sequence mutation fitness by cyclic iteration optimization provided in the embodiments of the present application, the amino acid sequence of each protein in the protein mutation data set is encoded by using different encoding methods in step S103 to obtain the second training set and the test set under different encodings, which can specifically include: the protein mutation data set is divided according to a set proportion (such as 8:2) to obtain the second training set and the test set; the amino acid sequence of each protein in the second training set is encoded by using different encoding methods to obtain the second training set under different encodings; and the amino acid sequence of each protein in the test set is encoded by using different encoding methods to obtain the test set under different encodings.

[0075] Further, in the above-mentioned method for generating sequence mutation fitness by cyclic iteration optimization provided in the embodiments of the present application, the test set correlation of the regression model corresponding to each encoding method is calculated and sorted in step S104, and the prediction model for optimizing the protein sequence mutation fitness is determined according to the correlation sorting result, which can specifically include: first, the correlation of the test set of the regression model corresponding to each encoding method is calculated; then, for each encoding method, the calculated correlation is sorted from high to low, and the mean of the correlation of the top set number (such as the top three) is calculated; the highest mean value among all the mean values is found, and the optimal encoding method and the corresponding regression model are determined according to the highest mean value; finally, the determined encoding method and the regression model are combined as the prediction model for optimizing the protein sequence mutation fitness.

[0076] It can be understood that while the calculated correlation is sorted from high to low, the regression model corresponding to different encoding methods is also sorted. The mean of the test set correlation of the prediction model of the top set number (such as the top three) of each encoding is selected to measure the effect of the prediction model, and the best encoding method can be determined. The determined best encoding method and the corresponding regression model (such as the regression model with the top three test set correlations corresponding to the encoding method) are combined as the predictor for optimizing the protein sequence mutation fitness.

[0077] Furthermore, in a specific implementation, in the method for iteratively optimizing the fitness of generated sequence mutations provided in the embodiments of the present invention, step S105 iteratively updates the parameters of the generative adversarial network model, which may specifically include: randomly sampling and generating new protein sequences through the generator of the generative adversarial network model; predicting the new protein sequences using a predictor and calculating the fitness loss; updating the generator loss using the fitness loss, fine-tuning the generative adversarial network model, and iteratively updating the parameters of the generative adversarial network model.

[0078] In practical implementation, the above steps involve using a predictor to predict new protein sequences and calculate fitness loss. Specifically, this may include: encoding the new protein sequence using a determined encoding method; predicting the encoded protein sequence using a predetermined number of regression models to obtain a predetermined number of predicted values; using the mean of these predicted values ​​as the mutation fitness prediction value, and calculating the fitness loss based on the mutation fitness prediction value. In this way, the predictor provides the mutation fitness prediction value for the generated sequence, thereby adding an additional fitness loss to the generator. This, combined with the iterative update of the model parameters by the discriminator, optimizes the protein sequence generation model from the perspective of mutation fitness. Through iterative optimization training, the fitness value of the newly generated sequence is effectively improved.

[0079] Specifically, a batch of new protein sequences X is generated by randomly sampling through the generator of the generative adversarial network model. f ={X f1 X f2 X f3 , ..., X fb}, X fn Let X represent the nth new sequence generated, obtained using the predictor described above. f The predicted mutation fitness value y for each sequence in the sequence f ={y f1 y f2 y f3 , ..., y fb}, to calculate the fitness loss C of the generator. fitness_loss Add fitness loss to the original generator loss C generator_loss Above, that is:

[0080] C generator_loss =C generator_loss +C fitness_loss .

[0081] By optimizing the generator loss through an updated generator, the generator can be moved toward higher mutation fitness, thereby directly obtaining new generated sequences with high mutation fitness.

[0082] Wherein, the calculation of the mutation fitness loss is divided into two parts due to the possibility of illegal sequences (sequences not starting with methionine or protein sequences with a length less than 50) generated by the newly generated sequences, the logarithmic probability of legal sequences higher than the target threshold (i.e. the loss of legal sequences) and the penalty term for illegal sequences (i.e. the loss of illegal sequences), that is:

[0083] C fitness_loss legal_seqs_loss illegal_seqs_loss

[0084] Wherein, C legalseqs_loss is the loss of legal sequences, and C illegal_seqs_loss is the loss of illegal sequences.

[0085] For the loss of legal sequences, the calculation is the logarithmic probability of the predicted value y f higher than the target threshold t. The predicted distribution is simulated as a Gaussian distribution, and the mean and variance of the predicted value y f are calculated respectively. The fitness loss of legal sequences is:

[0086]

[0087] Wherein, γ fitness is the fitness penalty weight of legal sequences, and t is the target threshold, i.e. the lower bound of the survival function.

[0088] For the penalty loss of illegal sequences, the number of illegal sequences generated in a batch is used to calculate the penalty term:

[0089] C illegal_seqs_loss = (size batch_seqs -size legal_seqs )·γ illegal_seqs

[0090] Wherein, γ illegalseqs is the penalty weight of illegal sequences, size batch_seqs is the total number of new sequences generated by the generator sampling a batch, and size legal_seqs is the number of legal sequences in the generated new sequences in a batch.

[0091] Finally, based on the pre-trained generative adversarial network model, the fitness loss is used to update the generator loss, the model is fine-tuned, and the model parameters are updated iteratively to maximize the fitness of the generated sequences.

[0092] ​​It is emphasized that the present application selects the optimal combination as a predictor for optimizing the generation of sequence mutation fitness by comparing various mutation fitness encodings and regression models. Based on the optimal predictor, the pre-trained generative adversarial network model is further optimized and trained, and the additional fitness loss is calculated and incorporated into the generator loss, so that the generator can be optimized towards the direction of high mutation fitness, thereby directly generating new sequences with high mutation fitness through the trained generator.

[0093] The above-mentioned method for cyclic iteration and optimization of sequence mutation fitness provided by the embodiments of the present application is described below with a specific example, as shown in the following table: Figure 2 The specific steps are as follows:

[0094] Step 1: Obtain homologous family sequence data of the protein sequence and perform preprocessing and data set division;

[0095] In this example, BLAT_ECOLX (P62593) is selected as the target protein sequence, and homologous protein family sequence data is obtained from the UniProt database, totaling 38128. After filtering out non-typical amino acids (B, O, U, X, Z, J) and sequences longer than 512, there are 37139 remaining.

[0096] Using MMseqs tool, clustering is performed according to 70% sequence identity, resulting in a total of 7910 clusters. A large portion of these clusters contain only a small number of sequences, with 2396 clusters containing only one sequence. The quality and quantity of training data often limit the model, so the NTA algorithm is used to expand and balance the clustered sequences, ultimately obtaining about two million training data.

[0097] Each of the expanded clusters is sampled, with 20% selected as the validation set (791), and the remaining sequences as the first training set.

[0098] Step 2: Construct a generative adversarial network model based on a deep residual neural network and pre-train for 250,000 steps to fit the data distribution of the target sequence. As shown in the following table: Figure 3 As the training process progresses, the generated sequences gradually fit the data distribution of the target sequence.

[0099] Step 3: Obtain the protein mutation data set of BLAT_ECOLX, which contains 4997 mutant sequences and their corresponding mutation fitness labels. Divide the training set and test set samples according to an 8:2 ratio to obtain 3998 training samples.

[0100] The amino acid sequences of all mutant proteins in the data set are encoded, and 13 encoding methods are used respectively: UniRep, UniRep+VAE, Transformer, esm_t6, esm_t12, esm_t34_50S, esm_t34_50D, esm_t34_100, ProtAlbert, ProtXLNet, ProtT5_XL_BFD, ProtBert, ProtBert_BFD.

[0101] After encoding the mutant sequences with experimental label mutation fitness values, 10 regression models are trained respectively: XGBTree, Linear, GradientBoosting, RandomForest, ARD, KernelRidge, BayesianRidge, Bagging, LassoLarsCV, KNeighbor.

[0102] The correlation of the test set of the regression model fitted by each encoding model is calculated respectively, ranked from high to low according to the correlation, and the mean of the test set correlation of the top three regression models is taken to determine the optimal combination of encoding and regression model. The correlation values of the top three models of each encoding test set are as follows in Table 1:

[0103] Table 1

[0104] test_top1 test_top2 test_top3 test_mean UniRep 0.798 0.754 0.734 0.762 UniRep+VAE 0.865 0.843 0.842 0.850 Transformer 0.775 0.748 0.746 0.756 esm_t6 0.767 0.745 0.724 0.745 esm_t12 0.799 0.799 0.797 0.798 esm_t34_50S 0.847 0.842 0.837 0.842 esm_t34_50D 0.837 0.831 0.822 0.830 esm_t34_100 0.820 0.812 0.802 0.811 ProtAlbert 0.846 0.840 0.809 0.832 ProtXLNet 0.806 0.806 0.804 0.806 ProtT5_XL_BFD 0.843 0.834 0.828 0.835 ProtBert 0.842 0.837 0.825 0.834 ProtBert_BFD 0.841 0.836 0.830 0.836

[0105] From Table 1, the regression model test set mean of UniRep+VAE 1901-dimensional encoding is the highest, so it is used as the predictor for generating sequence mutation fitness. The better the performance of the predictor, the more reliable the result.

[0106] Step 4: Introduce the mutation fitness predictor obtained in step 3 into the generative adversarial network model. Through a batch of new sequences randomly sampled, first encode them into 1901-dimensional feature vectors through the UniRep+VAE model, then use the top three regression models trained to make predictions respectively, take the mean of the three as the mutation fitness prediction value of the generated sequence, and then calculate the fitness loss to update the generator loss. On the basis of the pre-trained generative adversarial network model, continue to train and optimize the generative adversarial network model towards high fitness. With the iteration of the training steps, the mutation fitness values of the generated sequences in the training process are determined. Each time the generator directly generates sequences of the same order of magnitude, and the predictor gives the mutation fitness value of the generated sequence. Table 2 shows the number of sequences with fitness values greater than a given threshold t in each batch of generated sequences as the training steps increase, and the results are as follows:

[0107] Table 2

[0108] steps 300,000 340,000 350,000 360,000 Number of sequences (Fitness>0) 2319 3462 6183 7289

[0109] From Table II, it can be concluded that the application improves the fitness of the generated sequence mutation, and continuously improves with the increase of training steps.

[0110] Based on the same inventive concept, the application also provides a device for cyclically iterating to optimize the fitness of the generated sequence mutation. Since the principle of the device for solving the problem is similar to the aforementioned method for cyclically iterating to optimize the fitness of the generated sequence mutation, the implementation of the device can be referred to the implementation of the method for cyclically iterating to optimize the fitness of the generated sequence mutation, and the repeated parts will not be described herein.

[0111] In a specific implementation, the device for cyclically iterating to optimize the fitness of the generated sequence mutation provided by the application includes, as shown in Figure 4 , specifically comprises:

[0112] The data processing module 11 is configured to obtain homologous family sequence data of a target protein sequence, pre-process the homologous family sequence data, and generate a first training set.

[0113] The model training module 12 is configured to construct a generative adversarial network model, and pre-train the generative adversarial network model by using the first training set.

[0114] The data encoding module 13 is configured to obtain a protein mutation data set, encode the amino acid sequence of each protein in the protein mutation data set by using different encoding methods, and obtain a second training set and a test set under different encodings.

[0115] The predictor determining module 14 is configured to train and test a plurality of regression models by using the second training set and the test set, calculate the test set correlation of the regression model corresponding to each encoding method, and sort the test set correlation, and determine a predictor for optimizing the fitness of the protein sequence mutation according to the correlation sorting result.

[0116] The cyclic iteration module 15 is configured to introduce the predictor into the generative adversarial network model, cyclically update the parameters of the generative adversarial network model, and maximize the fitness of the generated sequence mutation.

[0117] In the device for cyclically iterating to optimize the fitness of the generated sequence mutation provided by the application, the five modules can interact with each other to directly generate a sequence with high mutation fitness, thereby realizing the optimization of the specific function of the protein sequence.

[0118] The more specific working processes of the above-mentioned modules can be referred to the corresponding content disclosed in the foregoing embodiments, and will not be described herein.

[0119] Correspondingly, the application also discloses a device for cyclically and iteratively optimizing generation of sequence mutation fitness, comprising a processor and a memory; wherein the processor realizes the method for cyclically and iteratively optimizing generation of sequence mutation fitness disclosed in the foregoing embodiments when executing the computer program stored in the memory. More specific processes of the foregoing method can refer to the corresponding contents disclosed in the foregoing embodiments, which will not be repeated here.

[0120] Further, the application also discloses a computer readable storage medium for storing a computer program; the computer program is executed by the processor to realize the method for cyclically and iteratively optimizing generation of sequence mutation fitness disclosed in the foregoing embodiments. More specific processes of the foregoing method can refer to the corresponding contents disclosed in the foregoing embodiments, which will not be repeated here.

[0121] The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the difference from other embodiments. The same or similar parts between various embodiments can refer to each other. For the devices, apparatuses and storage media disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the related parts can refer to the method part.

[0122] The skilled person can further realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be realized by electronic hardware, computer software or a combination of both. In order to clearly show the interchangeability of hardware and software, the components and steps of the examples have been described in the above description. Whether the functions are realized by hardware or software depends on the specific application and design constraints of the technical solution. The skilled person can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0123] The steps of the method or algorithm described in combination with the embodiments disclosed herein can be directly implemented by hardware, a software module executed by a processor, or a combination of both. The software module can be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0124] In summary, the embodiment of the present application provides a method, device, equipment and storage medium for cyclically and iteratively optimizing generation of sequence mutation fitness, comprising: obtaining homologous family sequence data of a target protein sequence, pre-processing the homologous family sequence data to generate a first training set; constructing a generative adversarial network model, and pre-training the generative adversarial network model by using the first training set; obtaining a protein mutation dataset, encoding the amino acid sequence of each protein in the protein mutation dataset by using different encoding methods to obtain a second training set and a test set under different encodings; training and testing a plurality of regression models by using the second training set and the test set, calculating the test set correlation of the regression model corresponding to each encoding method and sorting, determining a predictor for optimizing the fitness of the protein sequence mutation according to the correlation sorting result; introducing the predictor into the generative adversarial network model, and cyclically and iteratively updating the parameters of the generative adversarial network model to maximize the mutation fitness of the generated sequence. In this way, the homologous family sequence data of the target protein sequence is first obtained, pre-processed and divided into data sets, and then the generative adversarial network model is constructed and pre-trained to fit the data distribution of the target sequence. Then, the protein sequence generation model is optimized from the perspective of mutation fitness. After the mutation sequence is encoded in different ways, a plurality of regression models are trained and tested, and the predictor for optimizing the fitness of the protein sequence mutation is determined according to the test set correlation sorting result. Finally, the predictor is introduced into the generative adversarial network model for further optimization training by cyclic iteration, which can effectively improve the fitness value of the generated sequence. In this way, the original data distribution is retained and the fitness optimization is added. The cyclically and iteratively optimized generative adversarial network model can directly generate a sequence with high mutation fitness, overcoming the limitation of directed evolution that can only mutate a few sites around the natural protein. Thus, the mystery of the protein sequence space is revealed, and the understanding of the mutation effect of the generation model is explored, opening up a new perspective for sequence generation. In addition, the present application also provides a corresponding device, equipment and computer readable storage medium for the method of cyclically and iteratively optimizing generation of sequence mutation fitness, further making the above method more practical, and the device, equipment and computer readable storage medium have corresponding advantages.

[0125] Finally, it is to be understood that the phraseology or terminology such as "first" and "second" etc. used herein is merely intended to differentiate one entity or operation from another entity or operation, without necessarily requiring or implying any actual such relationship or order between such entities or operations. Moreover, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises... a" does not, without more constraints, exclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0126] The method, device, equipment and storage medium for generating sequence mutation fitness by cyclic iteration optimization provided by the present application are described in detail above, the principle and implementation mode of the present application are described by applying specific examples in the present article, the above example description is only used for helping to understand the method of the present application and its core idea; meanwhile, for the general technical personnel in the art, according to the idea of the present application, the specific implementation mode and application range will have changes, and the above description should not be understood as the limitation of the present application.

Claims

1. A method of cyclically iterating optimization to generate a sequence of fitness of mutations, characterized in that, The method comprises the following steps: obtaining homologous family sequence data of a target protein sequence, preprocessing the homologous family sequence data to generate a first training set; constructing a generative adversarial network model and pre-training the generative adversarial network model using the first training set; obtaining a protein mutation data set, encoding the amino acid sequence of each protein in the protein mutation data set using different encoding methods to obtain a second training set and a test set under different encodings; training and testing a plurality of regression models using the second training set and the test set, calculating the correlation of the test set of the regression model corresponding to each encoding method, and sorting the correlation, determining a predictor for optimizing the mutation fitness of the protein sequence according to the correlation sorting result; introducing the predictor into the generative adversarial network model, and iteratively updating the parameters of the generative adversarial network model to maximize the mutation fitness of the generated sequence.

2. The method of claim 1, wherein the sequence of mutations is generated by cyclic iteration of the optimization. The preprocessing of the homologous family sequence data to generate a first training set comprises the following steps: filtering the homologous family sequence data to filter out sequences containing non-canonical amino acids and super-long amino acid sequences; clustering the filtered homologous family sequence data according to sequence identity using the MMseq2 tool; using the NTA algorithm to balance and expand the data in the unbalanced class after clustering; dividing the expanded data to generate a first training set.

3. The method of claim 2, wherein the sequence of mutations is generated by cyclic iteration of the optimization. The data balancing and expansion processing of the unbalanced class using the NTA algorithm comprises the following steps: using the degeneracy of codons, reverse translating each amino acid sequence in the unbalanced class into multiple corresponding nucleotide sequences.

4. The method of claim 3, wherein the sequence of mutations is generated by cyclic iteration of the optimization. The construction of the generative adversarial network model and the pre-training of the generative adversarial network model using the first training set comprises the following steps: constructing a generative adversarial network model based on a deep residual neural network using a generator and a discriminator both connected by multiple layers of residual connections; pre-training the generative adversarial network model using the first training set, and cyclically and alternately updating the generator and the discriminator during training.

5. The method of claim 4, wherein the sequence of fitnesses of sequence mutations is generated by cyclic iteration optimization, and The calculation of the test set correlation of the regression model corresponding to each encoding method and the sorting of the correlation, and the determination of a prediction model for optimizing the mutation fitness of the protein sequence according to the correlation sorting result, comprises the following steps: calculating the correlation of the test set of the regression model corresponding to each encoding method; for each encoding method, sorting the calculated correlation from high to low, and calculating the mean of the correlation of the top set number; finding the highest mean value among all the mean values, and determining the optimal encoding method and the corresponding regression model according to the highest mean value; combining the determined encoding method and regression model as a prediction model for optimizing the mutation fitness of the protein sequence.

6. The method of claim 5, wherein the sequence of fitnesses of sequence mutations is generated by cyclic iteration optimization. The iteratively updating of the parameters of the generative adversarial network model comprises the following steps: randomly sampling a new protein sequence by the generator of the generative adversarial network model; predicting the new protein sequence using the predictor and calculating the fitness loss; The generator loss is updated by using the fitness loss, the generative adversarial network model is fine-tuned, and the parameters of the generative adversarial network model are updated in a loop iteration.

7. The method of claim 6, wherein the sequence of fitnesses of sequence mutations is generated by cyclic iteration optimization. The new protein sequence is predicted by using the predictor, and the fitness loss is calculated, including: The new protein sequence is encoded by using the determined encoding mode; The encoded protein sequence is predicted by using the determined regression model of the set number, and the set number of predicted values is obtained. The mean of the set number of predicted values is taken as the mutation fitness predicted value, and the fitness loss is calculated according to the mutation fitness predicted value.

8. An apparatus for cyclically iterating optimization to generate a sequence of fitness of mutations, comprising: It includes: The data processing module is used for obtaining the homologous family sequence data of the target protein sequence, preprocessing the homologous family sequence data, and generating a first training set; The model training module is used for constructing a generative adversarial network model, and pre-training the generative adversarial network model by using the first training set; The data encoding module is used for obtaining a protein mutation data set, encoding the amino acid sequence of each protein in the protein mutation data set by using different encoding modes, and obtaining a second training set and a test set under different encodings; The predictor determination module is used for training and testing a plurality of regression models by using the second training set and the test set, calculating the test set correlation of the regression model corresponding to each encoding mode and sorting, and determining the predictor for optimizing the mutation fitness of the protein sequence according to the correlation sorting result; The loop iteration module is used for introducing the predictor into the generative adversarial network model, updating the parameters of the generative adversarial network model in a loop iteration, and maximizing the mutation fitness of the generated sequence.

9. A device for iteratively optimizing the generation of sequence mutation fitness, characterized in that, It includes a processor and a memory, wherein the processor implements the method of claim 1-7 when executing the computer program stored in the memory.

10. A computer-readable storage medium, characterized in that, A computer program for storing a computer program, wherein the computer program is executed by a processor to implement the method of claim 1-7 for optimizing the mutation fitness of the generated sequence in a loop iteration. A computer program for storing a computer program, wherein the computer program is executed by a processor to implement the method of claim 1-7 for optimizing the mutation fitness of the generated sequence in a loop iteration.

Images