AAV2 capsid protein variant design method and system and storage medium

Abstract

The invention belongs to the field of gene therapy, and particularly relates to an AAV2 capsid protein variant design method and system and a storage medium. The system comprises a preprocessing module which is configured to preprocess collected sequence data of AAV2 capsid protein; the input module is configured to input AAV2 capsid protein sequences with different sequence lengths; the prediction module is configured to classify the AAV2 capsid protein sequences through a fine-tuned AntiBERTy model, and obtain AAV2 capsid protein variant sequences according to functional scores of the sequences; and an output module configured to output the sequence of the prediction modules. According to the method, high-quality protein sequences with different lengths and functionality can be generated, the homology and diversity of the generated sequences are remarkably improved, and the functional characteristics of the sequences are accurately predicted, so that the effective screening of the AAV2 capsid protein variants is realized.

Description

Technical Field

[0001] The present invention belongs to the field of gene therapy, and in particular relates to a method, system and storage medium for designing AAV2 capsid protein variants. Background Art

[0002] As a revolutionary treatment method, gene therapy has shown great potential in the treatment of many diseases such as malignant tumors and genetic diseases in recent years. The key step of gene therapy is to design and construct gene vectors to safely and effectively deliver target genes to host cells, ensure that genes are correctly expressed in cells, and exert therapeutic effects. Adeno-associated virus (AAV) is the simplest non-enveloped single-stranded DNA virus discovered so far. The viral genome is about 4.7Kb long and belongs to the parvovirus family. Because AAV is non-pathogenic and has a wide host range and does not cause strong immune responses in the human body, it has become one of the most important gene vectors in the field of gene therapy. The AAV2 capsid is the first component of gene therapy approved by the U.S. Food and Drug Administration for use in humans.

[0003] The key requirement for gene vectors is to be able to resist human immune protection mechanisms. However, natural AAV has some shortcomings. Many human sera have neutralizing ability against the virus, which can prevent the virus from further infecting host cells, resulting in the inability of the vector to successfully deliver the drug or target gene to the designated location, resulting in treatment failure, which poses a challenge to gene therapy. In order to overcome this problem, it is necessary to design new and diverse AAV variants that can alleviate the problem of natural immunity to a certain extent, which is of great significance for gene therapy. At present, most variant designs focus on the amino acid sites 561-588 in the AAV2 capsid protein VP3. The specific natural wild-type capsid protein sequence is: DEEEIRTTNPVATEQYGSVSTNLQRGNR (SEQ ID NO.1), which is located near the three-fold symmetry axis of the AAV2 VP1 protein, including buried regions, surface regions and interface regions, and overlaps with known heparin and antibody binding sites. However, traditional protein design methods often rely on artificial experience to conduct directed evolution and a large number of experimental screening of wild-type (WT) protein sequences, including strategies such as random mutations, gene shuffling, and targeted recombination of protein fragments. Directed evolution is a powerful approach, and repeated application of random mutagenesis and artificial selection is often the default experimental strategy when mechanistic understanding is limited. However, although experimental-based techniques can test a large number of biological sequences, the engineered protein library rarely exceeds the sequence diversity of the natural protein family and most mutations are non-functional proteins. Because the generated sequences are still very similar to natural isolates, these variants have limited ability to overcome antibody neutralization and cannot have the ability to evade immunity.

[0004] With the continuous development of artificial intelligence technology, data-driven machine learning protein engineering design methods have begun to rise. The application of data-driven machine learning in protein engineering design not only improves the efficiency of design, but also opens up new possibilities for biomedical research. In the article "Bryant, DH, Bashir, A., Sinai, S. et al. Deepdiversification of an AAV capsid protein by machine learning. Nat Biotechnol 39, 691–696 (2021)", a large amount of AAV2 capsid protein data was used to train a classification model through supervised learning machine learning methods to predict whether the capsid protein sequence variants are active, including logistic regression models (LR), convolutional neural networks (CNN), and recurrent neural networks (LSTM). By sampling different sequences, these machine learning algorithms are used to classify and discriminate the functionality of the sequences to guide the diversified generation of AAV2 capsid proteins. Compared with traditional methods, the advantage is that feasible AAV2 variant sequences can be quickly generated through the program, but the classification prediction accuracy of the model still has room for improvement. In the article "Sinai S, Jain N, Church GM, et al. Generative AAV capsid diversification by latent interpolation [J]. bioRxiv, 2021: 2021.04.16.440236.", the authors used an unsupervised method to directly generate new AAV2 variant sequences using the evolutionary and mutational data of AAV capsid proteins using a generative model of a variational autoencoder (VAE). Although this method can generate many AAV2 variant sequences, the length of the generated sequences is fixed, and the generated sequences are not homologous enough, and there are still a large number of non-functional proteins.

[0005] It can be seen that the existing data-driven methods still have problems in terms of both the quality of generated sequences and the ability to classify and discriminate sequences, and have failed to provide a high-quality and effective solution. Therefore, a new method that can efficiently design diverse AAV2 capsid protein variants is urgently needed. Summary of the invention

[0006] In view of the problems of the prior art, the present invention provides an AAV2 capsid protein variant design system.

[0007] An AAV2 capsid protein variant design system, comprising:

[0008] The preprocessing module is configured to: preprocess the collected sequence data of AAV2 capsid protein;

[0009] The input module is configured to: input AAV2 capsid protein sequences of different sequence lengths;

[0010] The prediction module is configured to: classify AAV2 capsid protein sequences of different sequence lengths through the fine-tuned AntiBERTy model, and obtain AAV2 capsid protein variant sequences according to the functional scores of the sequences;

[0011] The output module is configured to: output the sequence of the prediction module.

[0012] Preferably, the fine-tuned AntiBERTy model includes an AntiBERTy network layer and an MLP network layer.

[0013] Preferably, the AntiBERTy network layer includes a characterization sequence hidden layer and an average pooling layer; the characterization sequence hidden layer includes a first feature extraction layer, a second feature extraction layer and a last feature extraction layer, and the average pooling layer concatenates the features obtained by the first feature extraction layer, the second feature extraction layer and the last feature extraction layer.

[0014] Preferably, the number of the MLP network layers is two, the first MLP network layer is used to project the vector obtained by concatenating the protein sequence features to be evaluated and the mutation features to a low dimension, and the second MLP network layer is used to classify and evaluate the vector projection.

[0015] Preferably, the mutation feature is a difference feature obtained by subtracting the protein sequence feature to be evaluated from the wild-type AAV2 capsid protein sequence feature; the wild-type AAV2 capsid protein sequence is shown in SEQ ID NO.1.

[0016] Preferably, the preprocessing operations include sequence cleaning, redundancy removal, and alignment.

[0017] Preferably, the AAV2 capsid protein sequences of different sequence lengths are generated by a fine-tuned Progen2 model.

[0018] Preferably, the fine-tuned Progen2 model adjusts model parameters through a loss function to generate AAV2 capsid protein sequences of different sequence lengths; the loss function calculation formula is as follows:

[0019]

[0020] In the above formula, L represents the calculated loss, y tk Indicates the actual token value. Represents the token output value predicted by the model.

[0021] The present invention also provides a method for designing AAV2 capsid protein variants using the above system, comprising the following steps:

[0022] Step 1, collecting sequence data of AAV2 capsid protein and performing preprocessing;

[0023] Step 2, input AAV2 capsid protein sequences of different sequence lengths;

[0024] Step 3: Classify AAV2 capsid protein sequences of different sequence lengths using the fine-tuned AntiBERTy model, and obtain AAV2 capsid protein variant sequences based on the functional scores of the sequences.

[0025] A computer-readable storage medium stores thereon: a computer program for implementing the above system, and a program for implementing the method.

[0026] The AAV2 capsid protein variant design system provided by the present invention mainly includes a pre-trained fine-tuned Progen2 model and a fine-tuned AntiBERTy model; the fine-tuned Progen2 model generates AAV2 capsid protein variant sequences of different sequence lengths by adjusting model parameters, and then inputs the sequences into the fine-tuned AntiBERTy model, and obtains the functional score of the sequences through feature splicing processing, thereby screening out high-quality AAV2 capsid protein sequences. Therefore, the beneficial effects of the present invention are: high-quality AAV2 capsid protein sequences can be obtained, the homology and diversity of the generated sequences can be improved, and the system of the present invention has good prediction performance and classification performance.

[0027] Obviously, according to the above contents of the present invention, in accordance with common technical knowledge and customary means in the art, without departing from the above basic technical ideas of the present invention, other various forms of modification, replacement or change may be made.

[0028] The above contents of the present invention are further described in detail below through specific implementation methods in the form of embodiments. However, this should not be understood as the scope of the above subject matter of the present invention being limited to the following examples. All technologies realized based on the above contents of the present invention belong to the scope of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 Site-wise Shannon entropy of AAV2 capsid protein sequences generated for different models.

[0030] Figure 2 Cumulative density function distribution of functionality scores for sequences generated by the AAV2 capsid protein variant design system. DETAILED DESCRIPTION

[0031] It should be noted that the algorithms of data collection, transmission, storage and processing steps not specifically described in the embodiments, as well as the hardware structure, circuit connection, etc. not specifically described can all be implemented through the contents disclosed in the prior art.

[0032] Example 1 AAV2 capsid protein variant design system and method

[0033] The AAV2 capsid protein variant design system of this embodiment includes:

[0034] The preprocessing module is configured to: preprocess the collected sequence data of AAV2 capsid protein;

[0035] The input module is configured to: input AAV2 capsid protein sequences of different sequence lengths;

[0036] The prediction module is configured to: classify the AAV2 capsid protein sequence by the fine-tuned AntiBERTy model, and obtain the AAV2 capsid protein variant sequence according to the functional score of the sequence;

[0037] The output module is configured to: output the sequence of the prediction module.

[0038] This embodiment also provides a method for designing AAV2 capsid protein variants using the above system, the method comprising the following steps:

[0039] Step 1, collecting sequence data of AAV2 capsid protein and performing preprocessing;

[0040] Step 2, input AAV2 capsid protein sequences of different sequence lengths;

[0041] Step 3: Classify AAV2 capsid protein sequences of different sequence lengths using the fine-tuned AntiBERTy model, and obtain AAV2 capsid protein variant sequences based on the functional scores of the sequences.

[0042] In the above preprocessing module, the preprocessing operations include sequence cleaning, redundancy removal, alignment, etc.

[0043] In the input module, the AAV2 capsid protein sequences of different sequence lengths are generated by a fine-tuned Progen2 model. The network structure and training method of the fine-tuned Progen2 model are as follows:

[0044] (1) The AAV capsid protein variant data from the literature (Bryant, DH, Bashir, A., Sinai, S. et al. Deep diversification of an AAV capsid protein by machine learning. Nat Biotechnol 39, 691-696 (2021).) was used to screen out functional protein sequences, using 70% as the training set and 30% as the test set. The pre-trained Progen2 tokenizer was used to process the sequence data to obtain the corresponding tokens;

[0045] (2) When fine-tuning the Progen2 model, the optimizer uses the AdamW method in pytorch, the learning rate is set to 0.00001, and the number of training rounds is 5. In the training phase, the token at the next position is continuously predicted from the first position of the input sequence, and the loss is calculated using the actual token and all predicted token values. The loss function uses the cross entropy loss function as follows:

[0046]

[0047] In the above formula, L represents the calculated loss, y tk Indicates the actual token value. Represents the token output value predicted by the model;

[0048] (3) After completing the model training, set the sampling parameters of the model. The key parameters include: max-length is the longest sequence length, num-samples is the number of sequence samples, and t is the sampling temperature (the higher the temperature, the better the diversity of sequence generation); specifically, the parameters are set to: max-length = 45, t = 1.3, num-samples = 10000;

[0049] (4) The generated tokens are translated into protein sequences using the corresponding tokenizer dictionary to obtain AAV2 capsid protein sequences of different sequence lengths.

[0050] In the prediction module, the constructed fine-tuned AntiBERTy model was used to classify, evaluate and score the AAV2 capsid protein sequences of different sequence lengths generated by the fine-tuned Progen2 model to screen out high-quality AAV2 capsid protein variant sequences.

[0051] Among them, the network structure and training method of the fine-tuned AntiBERTy model are as follows:

[0052] (1) The AAV capsid protein variant data in the literature (Bryant, DH, Bashir, A., Sinai, S. et al. Deep diversification of an AAV capsid protein by machine learning. Nat Biotechnol 39, 691-696 (2021)) was used, using 70% of the data as the training set, 15% of the data as the validation set, and 15% of the data as the test set;

[0053] (2) Extracting features from protein sequences by using a fine-tuned AntiBERTy model, wherein the fine-tuned AntiBERTy model comprises an AntiBERTy network layer and an MLP network layer; the AntiBERTy network layer comprises a representation sequence hidden layer and an average pooling layer, wherein the representation sequence hidden layer comprises a first feature extraction layer, a second feature extraction layer and a last feature extraction layer, and the average pooling layer concatenates features obtained from the first feature extraction layer, the second feature extraction layer and the last feature extraction layer.

[0054] (3) The number of MLP network layers is two, and the number of neurons is 1024 and 512 respectively. The first MLP network layer is used to project the vector after the protein sequence feature to be evaluated and the mutation feature are concatenated to a low dimension, and the second MLP network layer is used to classify and evaluate the vector projection, and predict whether the given sequence has a functional binary classification result. The mutation feature is the difference between the protein sequence feature to be evaluated and the wild-type AAV2 capsid protein sequence feature; the wild-type AAV2 capsid protein sequence described is the sequence of the 561-588 sites of the wild-type AAV2 capsid protein VP3: DEEEIRTTNPVATEQYGSVSTNLQRGNR (SEQ ID NO.1), and the protein sequence to be evaluated is a capsid protein variant sequence designed or generated by mutation (in this embodiment: DEEEIRTTNPVATEQYGSVSTNLQRGEF (SEQ ID NO.2));

[0055] The loss function used by the model is the improved weighted cross entropy loss function L weights , the formula is:

[0056]

[0057] Among them L cross is the cross entropy loss function, Here y represents the true label. Represents the label predicted by the model, N represents the number of samples calculated in a batch, the learning rate during training is 0.0001, the number of training rounds is 5, the training set is used for training, the validation set is used for validation, the model parameters with the best performance on the validation set are retained, and finally the test set is used for testing to obtain the fine-tuned AntiBERTy model.

[0058] Comparative Example 1: Progen2 model without fine-tuning

[0059] The method for constructing the Progen2 model is the same as that for fine-tuning in Example 1, except that no loss function is used, no model training is performed, and no model pre-training parameters are updated.

[0060] Comparative Example 2 Variational Autoencoder Model (VAE)

[0061] The same data as that used for fine-tuning the Progen2 model in Example 1 was used for training to directly generate AAV2 variant sequences of different lengths.

[0062] Comparative Example 3 Convolutional Neural Network Model (CNN)

[0063] The same data as the fine-tuned AntiBERTy model in Example 1 was used for training, and the same loss function was used. The difference was that the model method was different, and a convolutional neural network (CNN) was used to extract features from the variant protein sequence and output the functional score of the sequence.

[0064] Comparative Example 4 Recurrent Neural Network Model (LSTM)

[0065] The same data as the fine-tuned AntiBERTy model in Example 1 was used for training, and the same loss function was used. The difference was that the model method was different, and a recurrent neural network (LSTM) was used to extract features from the variant protein sequence and output the functional score of the sequence.

[0066] Comparative Example 5 Logistic Regression Model (LR)

[0067] The same data as the fine-tuned AntiBERTy model in Example 1 was used for training, and the same loss function was used. The difference was that the model method was different. A logistic regression model (LR) was used to extract features from the variant protein sequence and output the functional score of the sequence.

[0068] Comparative Example 6 [CLS] Classification Model

[0069] The method for building the fine-tuned AntiBERTy model is the same as that in Example 1, except that the special classification head [CLS] feature vector of the last hidden layer in the AntiBERTy model is directly used, all pre-trained parameters are frozen without fine-tuning parameter updates, and only a single-layer MLP nonlinear classifier is used for classification evaluation.

[0070] The technical solution of the present invention is further illustrated by experiments below.

[0071] Experimental Example 1: Sequence features generated using a fine-tuned Progen2 model

[0072] In this experimental example, the AAV2 group is the AAV2 capsid protein dataset trained by the fine-tuned Progen2 model in Example 1, the VAE group is the sequence generated by Comparative Example 2, the Progen2-SFT-AAV group is the sequence generated by the fine-tuned Progen2 model in Example 1, and the Progen2-pretrain group is the sequence generated by Comparative Example 1.

[0073] 1. Experimental Methods

[0074] 1. Shannon entropy

[0075] By comparing the representative statistics of the amino acid variation of the generated sequence, the retention of the evolutionary characteristics of the natural data set is characterized. The calculation formula of Shannon entropy H(i) is as follows:

[0076] H(i)=-∑ a p(a i )log(p(a i ))

[0077] Where p(a i ) is the probability of amino acid a appearing at position i, and a represents all possible amino acid types.

[0078] 2. Experimental Results

[0079] The Progen2-SFT-AAV group and the AAV2 group were compared for homology by calculating the Shannon entropy, and the sequence generation results showed significant homology.

[0080] like Figure 1As shown, the Shannon entropy value of each site in the generated results of the Progen2-pretrain group is almost the same, which is obviously inconsistent with the capsid protein sequence distribution of the AAV2 group; compared with the AAV2 group, the VAE group is partially similar in curve trend, but the Shannon entropy value is significantly higher than the data entropy distribution of the AAV2 group; the Progen2-SFT-AAV group and the AAV2 group are basically consistent in the trend of the curve, indicating that the distribution of Shannon entropy of the two is very similar. The experimental results show that the AAV2 capsid protein sequence generated by the fine-tuned Progen2 model in the present invention has good homology.

[0081] Experimental Example 2: Prediction Performance of the Fine-tuned AntiBERTy Model

[0082] The fine-tuned AntiBERTy model used in this experimental example is constructed according to Example 1, and the CNN, LSTM, LR and [CLS] classification models are constructed according to Comparative Examples 3, 4, 5 and 6.

[0083] 1. Experimental Methods

[0084] A total of 153,682 complete data of AAV capsid proteins in the literature "Bryant, DH, Bashir, A., Sinai, S. et al. Deep diversification of an AAV capsid protein by machine learning. Nat Biotechnol 39, 691-696 (2021)" were used, including functional sequences and non-functional sequences. 70% of the data were used as training sets, 15% of the data were used as validation sets, and 15% of the data were used as test sets. The training input is different sequences to be evaluated, and the prediction result is the functional score of the sequence. A sequence greater than 0.5 is considered an available sequence, and a sequence less than 0.5 is considered an unavailable sequence. The evaluation standard uses classification accuracy (Accuracy), which can be used to indicate the proportion of samples correctly classified by the model to the total samples. The accuracy calculation formula is as follows:

[0085]

[0086] TP (True Positives): True positive examples, that is, the number of positive examples that are actually positive and predicted as positive. TN (True Negatives): True negative examples, that is, the number of negative examples that are actually negative and predicted as negative. FP (False Positives): False positive examples, that is, the number of negative examples that are actually negative but predicted as positive. FN (False Negatives): False negative examples, that is, the number of positive examples that are actually positive but predicted as negative. The higher the accuracy, the better the prediction result.

[0087] In addition, the area under the AUC (Area Under the Curve) curve is used for evaluation. AUC is the area under the ROC (Receiver Operating Characteristic) curve, which can measure the model's ability to distinguish between positive and negative samples. The ROC curve is drawn by calculating the true positive rate (TPR) and false positive rate (FPR) at different thresholds.

[0088]

[0089] The definitions of TP, FN, FP, and TN are consistent with the accuracy rate. The higher the AUC value, the better the model prediction effect.

[0090] 2. Experimental Results

[0091] The results are shown in Table 1. The algorithm of the fine-tuned AntiBERTy model of the present invention has an accuracy of 0.93 and an AUC value of 0.98, which are better than other algorithms involved in the comparison. When no fine-tuning parameter update is performed, the [CLS] classification model that only uses the pre-trained parameters of the AntiBERTy model for feature extraction has a prediction performance similar to that of general logistic regression classification, which is significantly lower than the method proposed in the present invention. The experimental results show that the fine-tuned AntiBERTy model in the present invention has good prediction performance and classification performance.

[0092] Table 1 Accuracy and AUC values ​​of different model algorithms

[0093]

[0094] Experimental Example 3: Sequence Screening Using the AAV2 Capsid Protein Variant Design System

[0095] The AAV2 capsid protein variant design system used in this experimental example was constructed according to Example 1.

[0096] 1. Experimental Methods

[0097] The protein sequence generated by the fine-tuned Progen2 model is input into the fine-tuned AntiBERTy model, and the feature processing method of the prediction module is used to process the generated sequence and finally output the prediction score. The cumulative density function distribution of the prediction score is calculated and visualized to show the further evaluation results of the generated sequence, so as to achieve effective screening of AAV2 capsid protein variants.

[0098] 2. Experimental Results

[0099] like Figure 2As shown in the figure, after the AAV2 capsid protein sequence generated by the fine-tuned Progen2 model was functionally evaluated using the fine-tuned AntiBERTy model, most of the generated sequences received high evaluation scores, indicating that the quality of the generated sequences was good, but there were still some sequences with low scores. Finally, by arranging the generated sequences in descending order of their functional scores, variant sequences with high scores can be effectively screened out.

[0100] In summary, the AAV2 capsid protein variant design system of the present invention generates an AAV2 capsid protein sequence with good homology, and the system is superior to the existing machine learning algorithm and has better prediction performance and classification performance, thereby achieving effective screening of AAV2 capsid protein variants.

Claims

1. A system for designing AAV2 capsid protein variants, characterized in that: include: The preprocessing module is configured to: preprocess the collected sequence data of AAV2 capsid protein; The input module is configured to: input AAV2 capsid protein sequences of different sequence lengths; The prediction module is configured to: classify AAV2 capsid protein sequences of different sequence lengths through the fine-tuned AntiBERTy model, and obtain AAV2 capsid protein variant sequences according to the functional scores of the sequences; The output module is configured to: output the sequence of the prediction module.

2. The AAV2 capsid protein variant design system according to claim 1, characterized in that: The fine-tuned AntiBERTy model includes an AntiBERTy network layer and an MLP network layer.

3. The AAV2 capsid protein variant design system according to claim 2, characterized in that: The AntiBERTy network layer includes a representation sequence hidden layer and an average pooling layer; the representation sequence hidden layer includes a first feature extraction layer, a second feature extraction layer and a last feature extraction layer, and the average pooling layer concatenates the features obtained by the first feature extraction layer, the second feature extraction layer and the last feature extraction layer.

4. The AAV2 capsid protein variant design system according to claim 2, characterized in that: The number of the MLP network layers is two. The first MLP network layer is used to project the vector obtained by concatenating the protein sequence features to be evaluated and the mutation features to a low dimension, and the second MLP network layer is used to classify and evaluate the vector projection.

5. The AAV2 capsid protein variant design system according to claim 4, characterized in that: The mutation feature is a difference feature obtained by subtracting the protein sequence feature to be evaluated from the wild-type AAV2 capsid protein sequence feature; the wild-type AAV2 capsid protein sequence is shown in SEQ ID NO.

1.

6. The AAV2 capsid protein variant design system according to claim 1, characterized in that: The pre-processing operations include sequence cleaning, redundancy removal, and alignment.

7. The AAV2 capsid protein variant design system according to claim 1, characterized in that: The AAV2 capsid protein sequences of different sequence lengths were generated by a fine-tuned Progen2 model.

8. The AAV2 capsid protein variant design system according to claim 7, characterized in that: The fine-tuned Progen2 model adjusts model parameters through a loss function to generate AAV2 capsid protein sequences of different sequence lengths; the loss function calculation formula is as follows: In the above formula, L represents the calculated loss, y tk Indicates the actual token value. Represents the token output value predicted by the model.

9. A method for designing AAV2 capsid protein variants using the AAV2 capsid protein variant design system according to any one of claims 1 to 8, characterized in that: The steps include: Step 1, collecting sequence data of AAV2 capsid protein and performing preprocessing; Step 2, input AAV2 capsid protein sequences of different sequence lengths; Step 3: Classify AAV2 capsid protein sequences of different sequence lengths using the fine-tuned AntiBERTy model, and obtain AAV2 capsid protein variant sequences based on the functional scores of the sequences.

10. A computer-readable storage medium, characterized in that: Stored thereon are: a computer program for implementing the system described in any one of claims 1 to 8, and a program for implementing the method.

Images