A protein mutation optimization method based on a protein language model and a ranking loss

Abstract

The application discloses a protein mutation optimization method based on a protein language model and a ranking loss, and the method comprises the following steps: constructing a saturation mutation dataset; initializing parameters; generating a high-dimensional embedding representation of a protein sequence by using a protein embedding model; in the first round of screening, selecting saturation mutations for screening according to a mutation selection strategy; in each round of screening thereafter, introducing a Bradley-Terry ranking loss function into a limit gradient boosting algorithm, and selecting the mutation with the highest predicted activity value for screening; checking the number of screening rounds to determine whether to end, if the maximum number of screening rounds has not been reached, returning to the next round of screening, otherwise ending the screening process; and obtaining a high-activity mutation set; the application overcomes the problem of insufficient samples caused by high experimental costs and time-consuming sample generation in traditional methods, and supports multi-objective optimization, such as simultaneously improving activity, stability and expression level.

Description

TECHNICAL FIELD

[0001] The present application relates to a protein mutation optimization method, and in particular to a protein mutation optimization method based on a protein language model and a ranking loss. BACKGROUND

[0002] In the field of protein engineering, protein mutation screening is an important step to achieve the optimization of protein function. Protein language models are pre-trained on a large number of protein sequences, which encode sequences into high-dimensional embedding vectors containing functional characteristics, providing support for mutation screening. Existing machine learning algorithms use these embeddings to train function prediction models, establishing a mapping relationship between sequence embeddings and protein functions, such as predicting the affinity of mutants in antibody optimization or identifying high catalytic efficiency mutations in enzyme engineering, showing strong application potential.

[0003] However, these algorithms also face some limitations in practical applications. First, the scarcity of sample data is a major challenge. Current machine learning algorithms highly rely on large amounts of experimental data for training. However, obtaining these data requires costly manual experiments, and the sample generation process is time-consuming, making it difficult to have sufficient sample data. The lack of sample data directly affects the efficiency of the screening algorithm, making it difficult to quickly screen high-activity mutants. Especially in complex protein optimization tasks, the lack of sample data often leads to the need for multiple iterations of the algorithm to obtain effective information, thereby prolonging the development cycle.

[0004] Secondly, in terms of screening accuracy and efficiency, existing machine learning algorithms generally use mean square error and cross-entropy loss functions. These loss functions mainly focus on the absolute error between the predicted value and the true value, ignoring the importance of the relative order between mutants. When the functional differences between mutants are subtle or the functional landscape is complex, these loss functions may result in underestimation or omission of high-activity mutants. This means that the algorithm may not accurately capture the relative order between mutants, thereby making incorrect assessments of some mutants with high potential activity, thereby reducing the accuracy of screening. SUMMARY

[0005] The purpose of the present application is to provide a protein mutation optimization method based on a protein language model and a ranking loss to optimize the problem of sample scarcity, improve screening efficiency, and reduce the number of screening rounds.

[0006] Technical solution: The protein mutation optimization method provided by the present application comprises the following steps:

[0007] S1, constructing a saturated mutation dataset;

[0008] S2, initializing parameters, including setting the experimental dataset, the number of screening rounds, the number of mutations evaluated per round, and the mutation selection strategy;

[0009] S3, generating a high-dimensional embedding representation of the protein sequence of the saturated mutation dataset by using a protein language model;

[0010] S4, in the first round of screening, selecting saturated mutations for screening according to an initial strategy;

[0011] S5, in each subsequent round of screening, introducing a Bradley-Terry ranking loss function into the extreme gradient boosting algorithm to select the mutation with the highest predicted activity value for screening;

[0012] S6, checking the number of screening rounds to determine whether to end, if the maximum number of screening rounds is not reached, returning to the next round of screening, otherwise ending the screening process;

[0013] S7, obtaining a set of high-activity mutations.

[0014] Preferably, the saturated mutation dataset in S1 is obtained by processing part of the positions of the protein sequence by continuous and discrete saturated mutation methods, obtaining corresponding saturated mutation samples, and measuring their activity value labels by experiments, and all saturated mutation samples constitute the saturated mutation dataset.

[0015] Preferably, the mutation selection strategy in S2 includes an initial strategy and a selection strategy; the initial strategy is used as the mutation selection strategy type for the first round of screening, and the selection strategy is used as the mutation selection strategy type for subsequent rounds of screening.

[0016] Preferably, the initial strategy is manual selection, and the selection strategy is a top-n strategy.

[0017] Preferably, the high-dimensional embedding representation in S3 is realized by the following formula:

[0018] ;

[0019] wherein, is an encoding function of the protein language model, represents that each element contains a mutation sequence, represents a mutation sequence corresponding embedding.

[0020] Preferably, S4 includes: randomly selecting M saturated mutations from the saturated mutation dataset by using the initial strategy, for the i-th selected saturated mutation, directly using the measured activity value label as the true activity value, updating the experimental dataset, and using the updated experimental dataset for the next round of screening.

[0021] Preferably, the Bradley-Terry ranking loss function of S5 aims to minimize the ranking error, by increasing the difference between the predicted values of the mutants, making the correct ranking probability close to 1, and then guiding the limit gradient boosting algorithm to build decision trees and calculate leaf node weights through first-order and second-order derivatives, to determine the node splitting method and optimize the relative order.

[0022] Preferably, let , the first-order and second-order derivative formulas are as follows:

[0023] ;

[0024] ;

[0025] wherein, , denotes the first-order derivative, reflecting the correctness of the pair-wise ranking, , denotes the second-order derivative, and changes with , reflecting the dynamic curvature of the ranking task, denotes the predicted value of the mutant , denotes the predicted value of the mutant .

[0026] Preferably, the limit gradient boosting algorithm uses the first-order and second-order derivatives to calculate the gain Gain, to evaluate the contribution of node splitting to the reduction of negative log-likelihood.

[0027] Through the multiple rounds of screening iterations of step S6, the final high-activity mutant set is obtained, that is, the top M mutants with the largest activity value selected in the last round.

[0028] Advantages: Compared with the prior art, the present application has the following significant advantages: 1. By generating high-dimensional sequence embeddings through the protein language model, the functional characteristics of the sequence are captured, providing rich feature representation for functional prediction, and then combined with the multi-round screening mode, the high-activity mutant can be quickly locked with a small amount of experimental data, thereby alleviating the sample scarcity problem; 2. By introducing the Bradley-Terry ranking loss function to optimize the protein language model, the optimized model can more accurately rank the mutants, thereby improving the screening accuracy of high-activity mutants, and the efficiency of the screening task is also significantly accelerated; 3. By introducing the protein saturation mutant sample, the model can learn more diverse data, thereby improving its performance in actual screening tasks and reducing the number of screening rounds. BRIEF DESCRIPTION OF DRAWINGS

[0029] Figure 1 is a flowchart of the present application;

[0030] Figure 2 Performance comparison of the present application. DETAILED DESCRIPTION

[0031] The technical solutions of the present application will be further described below in combination with the drawings.

[0032] As Figure 1 shown, the protein mutation optimization method based on protein language model and ranking loss includes the following steps:

[0033] S1, construct a saturation mutation dataset, the specific steps are as follows:

[0034] Given a wild-type protein sequence and a mutation dataset , the mutation dataset , wherein each mutation is a single-point mutation of . For example, if there is a protein sequence with a sequence length of 100, each point can be mutated into 20 amino acids, then its mutation dataset has , two methods of continuous saturation mutation and discrete saturation mutation are used to process part of the point positions, to obtain the corresponding saturation mutation samples, and the activity value label is measured by experiment, and the dataset composed of these saturation mutation samples is called saturation mutation dataset , denoted as .

[0035] Continuous saturation mutation selects a continuous segment of sites (for example, 301-400 sites of AsCas12f, 161-200 sites of beta-lactamase), processes it, and obtains the corresponding saturation mutation sample. Considering that the current multi-round iteration method has strong dependence on random sampling, random distribution of mutation sites is selected by discrete mutation samples (for example, 10% of the saturation mutation sites of the amino acid sequence are randomly selected on the AsCas12f dataset and the beta-lactamase dataset, which are verified by experiments).

[0036] By constructing a saturated mutation dataset, the model's ability to explore the protein function landscape is improved, and a more reliable data foundation is provided for subsequent multiple iterations; by systematically integrating saturated mutation samples, the invention can more efficiently lock high-activity mutations in the early screening stage, thereby reducing experimental costs and time expenditure. Based on the current multiple iteration model framework, the optimization effect of saturated mutation samples is verified on two protein datasets, AsCas12f and beta-lactamase. Among them, the AsCas12f dataset has 7941 mutation samples, with a high-activity proportion of 18% (1436); the beta-lactamase dataset has 4978 mutation samples, with a high-activity proportion of 7.9% (393). For example, on the beta-lactamase dataset, compared with the extreme gradient boosting algorithm + ranking loss + protein saturated mutation samples and the extreme gradient boosting algorithm + ranking loss, the application of protein saturated mutation samples reduces the screening rounds from 4.4 to 3.8, fully embodying its important value in data diversity optimization. More importantly, this strategy has wide applicability and can seamlessly adapt to different types of protein families and function optimization tasks, providing a new technical path for the intelligent development of protein engineering.

[0037] The quantification of high activity is usually based on the activity value of the wild-type protein as the standard. If the activity value of the mutant is greater than that of the wild-type protein, it is defined as high activity. The protein datasets used in this invention are all based on the activity value of the wild-type protein as the standard for measuring high activity.

[0038] S2, initialize parameters, including setting the screening rounds, the number of mutations evaluated per round, the experimental dataset, and the mutation selection strategy, the specific steps are as follows:

[0039] Set the experimental dataset , set the screening rounds R=10, and the number of mutations evaluated per round M=16, where the experimental dataset D is the dataset constructed for each experiment, each element in D contains a mutation sequence , a true activity value , and a mutation sequence embedding .

[0040] Where R refers to the number of iterations in the experiment, M is usually determined according to the actual scene, for example, given a protein, the number of mutations to be screened. Here, M=16 is a value obtained after statistical analysis of a large number of experimental results and a trade-off between experimental costs in real scenarios. The reference range is {8, 16, 20}.

[0041] The protein language model can be freely selected according to specific circumstances. In this embodiment, the protein language model selected is the ESM-2 model (150 billion parameters). As the first round of mutation selection strategy type, the selection strategy S is selected as the subsequent round of mutation selection strategy type; the initial strategy The random selection method is selected, that is, the first round of screening is selected from M saturated mutations are randomly selected for subsequent screening, and the top n strategy is selected as the selection strategy, that is, the top n mutations with the highest activity value in the prediction result are selected, and n=M=16 is set. If the initial strategy is manual selection, the experimental data set D can be set as a non-empty set, and the selected mutation is added.

[0042] S3, the protein language model is used to generate a high-dimensional embedding representation of the saturated mutation data set protein sequence, and the specific steps are as follows:

[0043] For the saturated mutation set , the embedding set is calculated, wherein the embedding of each mutation is generated by the following formula:

[0044] ;

[0045] Wherein, is the encoding function of the protein language model, indicates that each element contains a mutation sequence, indicates that the mutation sequence corresponds to the embedding, the protein language model selected in the experiment is the ESM-2 model (150 billion parameters), and a 5120-dimensional protein sequence embedding is generated, and finally the embedding matrix is obtained, which provides feature input for subsequent model training.

[0046] In the first round of screening (the first round of screening), the initial strategy is used to randomly select M saturated mutations from , and for the i-th selected saturated mutation, the activity value label measured in advance is directly used as the true activity value , the experimental data set D is updated, and is added, and the updated experimental data set D is used for the next round of screening.

[0047] In each subsequent round of screening (1-R round of screening), the Bradley-Terry sorting loss function is introduced into the extreme gradient boosting algorithm (XGBoost), and the mutation with the highest predicted activity value is selected for screening, and the specific steps are as follows:

[0048] Bradley-Terry ranking loss is a commonly used ranking loss function that can effectively capture the ranking relationship in pairwise comparison, especially suitable for multi-candidate ranking tasks. The default loss function used in random forest algorithm and extreme gradient boosting algorithm is mean square error loss function MSE, which is a pointwise loss widely used in regression tasks. It measures the prediction error of the model by calculating the average of the squared difference between the predicted value and the true value. Below is an example of introducing Bradley-Terry ranking loss function into the extreme gradient boosting algorithm.

[0049] The mean square error loss function MSE is defined as:

[0050] ;

[0051] Where n is the number of samples, is the true target value of sample i, is the predicted value of sample i.

[0052] The extreme gradient boosting algorithm minimizes the loss function through gradient optimization, which requires the first-order gradient and the second-order gradient to indicate the steepest descent direction and approximate curvature respectively to accelerate convergence. For , the first and second gradients are as follows:

[0053] First-order gradient (Gradient):

[0054] ;

[0055] The gradient reflects the difference between the predicted value and the true value. If the gradient is positive ( ), it means the predicted value is too high, and the model should reduce . If the gradient is negative ( ), it means the predicted value is too low, and the model should increase .

[0056] Second-order gradient (Hessian):

[0057] ;

[0058] Hessian is a constant, indicating that MSE is a quadratic function with fixed curvature. Constant Hessian simplifies the optimization process because it does not depend on the predicted value or the true value.

[0059] After introducing the Bradley-Terry ranking loss function:

[0060] Given the wild-type protein sequence and the mutation dataset. The mutation set of Each mutation yes A single-point mutation. The set of embeddings computed over the mutation set. The prediction obtained using the limiting gradient boosting algorithm .

[0061] Based on the Bradley-Terry ranking loss function, assuming mutants Should be placed in the mutant Predicted value The probability of them being correctly sorted is:

[0062] ;

[0063] Defined as the negative log-likelihood for all positive pairs ( Should be ranked forward):

[0064] ;

[0065] in, and Indicates mutant and mutants The tag value.

[0066] The goal of the Bradley-Terry sorting loss function is to minimize the sorting error by increasing... Probability of correct sorting It is close to 1. Unlike the absolute error of regression tasks, the Bradley-Terry ranking loss focuses on relative order, making it suitable for scenarios that require priority ranking.

[0067] The limiting gradient boosting algorithm constructs a decision tree by using the first derivative (gradient) and the second derivative (Hessian) to determine how nodes split. , The formulas for the first and second derivatives are as follows:

[0068] ;

[0069] ;

[0070] in, , This represents the first derivative, reflecting the correctness of pairwise sorting. That is, sorting error. A negative value indicates an increase. ; A positive value indicates a decrease. ; , denotes the second derivative, and varies with , reflecting the dynamic curvature of the ranking task, the smaller the value, reflecting the dynamic curvature of the ranking task, denotes the predicted value of the mutant , denotes the predicted value of the mutant .

[0071] The limit gradient boosting algorithm uses these derivatives to calculate the gain Gain, to evaluate the contribution of node splitting to reduction, its formula is as follows:

[0072] .

[0073] The split with large Gain value makes the samples in the left and right child nodes more consistent in ranking, for example, concentrates high priority samples on one side, shaping the tree structure through derivatives, optimizing the relative order.

[0074] After the tree is constructed, the derivative of is used to calculate the leaf node weight, update the prediction score of the mutant , the weight of leaf node j is:

[0075] ;

[0076] The weight reflects the ranking adjustment needs of the samples in the leaf node, and the dynamic nature of Hessian makes the calculation adapt to the pairwise relationship. The prediction value of sample i is updated as:

[0077] ;

[0078] where, is the learning rate, which controls the update amplitude, and the updated makes increase, improving ; Optimizing the score difference through weight ensures that high priority samples get higher scores.

[0079] The improved algorithm f is trained on the experimental data set D, that is, the limit gradient boosting algorithm with the introduction of Bradley-Terry ranking loss function is used to train the mutant data set, so that f can predict the activity value of all mutants : ; where, .

[0080] For The first M mutations with the largest activity values are selected using the selection strategy S. As in the first round of screening, for the selected ith mutation, if the mutation belongs to the saturated mutation dataset, the pre-determined activity value label is directly used as the true activity value ; if the mutation belongs to the non-saturated mutation dataset, the true activity value is determined by experiment , and finally the experimental dataset D is updated to include Only the predicted activity values of all saturated mutations are used .

[0081] As shown in Figure 2 , the average number of rounds required to reach 100% high activity for four methods, extreme gradient boosting algorithm + ranking loss + protein saturated mutation sample, random forest algorithm + protein saturated mutation sample, extreme gradient boosting algorithm + ranking loss, and random forest algorithm, on different datasets and sites; including AsCas12f dataset 351-400 sites, beta-lactamase dataset 247-287 sites, AsCas12f dataset random 10% sites, and beta-lactamase dataset random 10% sites.

[0082] It can be seen that by introducing the Bradley-Terry ranking loss function, the sensitivity of the model to subtle activity differences is significantly improved, enabling it to more accurately identify high-activity mutations, especially in tasks with complex activity landscapes. For example, in the beta-lactamase dataset 247-287 site task, compared with the extreme gradient boosting algorithm + ranking loss + protein saturated mutation sample and the random forest algorithm + protein saturated mutation sample (combining the mean square error loss MSE and the random forest algorithm with protein saturated mutation samples). The results show that the improved extreme gradient boosting algorithm shortens the screening rounds from 7.3 rounds to 4.1 rounds, fully demonstrating its advantages in ranking accuracy and efficiency. In addition, this method does not rely on complex protein structure information, but relies on the embedding generated by the protein language model to achieve efficient ranking, with high universality and easy scalability, providing new technical possibilities for the field of protein engineering and laying a solid foundation for future multi-objective optimization tasks.

[0083] S6, check the number of screening rounds to determine whether to end, if the maximum number of screening rounds is not reached, return to the next round of screening, otherwise end the screening process.

[0084] Set the maximum number of screening rounds R, determine whether to end by judging whether the number of screening rounds is reached, that is, judge the size relationship between r and R: when r is less than R, r is increased by one, and return to the previous step; otherwise, end the screening.

[0085] S7、through the multiple rounds of screening iterations of the preceding steps, a final set of high-activity mutations is obtained, i.e., the top M mutations with the largest activity values selected in the last round.

Claims

1. A protein mutation optimization method based on a protein language model and a ranking loss, characterized in that, The method comprises the following steps: S1, constructing a saturation mutation dataset; S2, initializing parameters, including setting an experimental dataset, a screening round, a number of mutations evaluated per round, and a mutation selection strategy; S3, generating a high-dimensional embedding representation of a protein sequence using a protein language model; S4, in the first round of screening, selecting saturation mutations for screening according to an initial strategy; S5, in each subsequent round of screening, introducing a Bradley-Terry ranking loss function into an extreme gradient boosting algorithm XGBoost to select mutations with the highest predicted activity values for screening; wherein the Bradley-Terry ranking loss function aims to minimize ranking errors by increasing the difference between the predicted values of mutants to make the correct ranking probability close to 1, and then using first-order and second-order derivatives to guide the extreme gradient boosting algorithm XGBoost to build decision trees and calculate leaf node weights, determine node splitting methods, and optimize relative order; Let The first and second derivative formulas are as follows: ； ； where g i represents the first derivative, h i represents the second derivative, h i and h j vary with δ, reflecting the dynamic curvature of the ranking task, g i and g j reflect the correctness of pairwise ranking, represents the predicted value of the mutant v i , represents the predicted value of the mutant v j . S6, checking the number of screening rounds to determine whether to end, if the maximum number of screening rounds has not been reached, returning to the next round of screening, otherwise ending the screening process; S7, obtaining a high-activity mutation set.

2. The method of protein mutation optimization of claim 1, wherein, The saturation mutation dataset in S1 is obtained by processing part of the positions of the protein sequence using continuous and discrete saturation mutation methods, obtaining corresponding saturation mutation samples, and measuring their activity value labels through experiments, and all saturation mutation samples constitute the saturation mutation dataset.

3. The method of protein mutation optimization of claim 1, wherein, The mutation selection strategy in S2 includes an initial strategy and a selection strategy; the initial strategy is used as the mutation selection strategy type for the first round of screening, and the selection strategy is used as the mutation selection strategy type for subsequent rounds of screening.

4. The method of protein mutation optimization of claim 3, wherein, The initial strategy is manual selection, and the selection strategy is a top-n strategy.

5. The method of protein mutation optimization of claim 1, wherein, The high-dimensional embedding representation in S3 is realized by the following formula: ； where PLM.encode() is an encoding function of the protein language model, represents that each element contains a mutant sequence, represents that the mutant sequence corresponds to an embedding.

6. The method of protein mutation optimization of claim 1, wherein, S4 includes: using the initial strategy to randomly select M saturation mutations from the saturation mutation dataset, using the measured activity value label as the true activity value for the i-th selected saturation mutation, updating the experimental dataset, and using the updated experimental dataset for the next round of screening.

7. The method of protein mutation optimization of claim 1, wherein, The extreme gradient boosting algorithm XGBoost uses first-order and second-order derivatives to calculate the gain, and evaluates the contribution of node splitting to the reduction of negative log-likelihood.

8. The method of protein mutation optimization of claim 1, wherein, Through the multiple rounds of screening iterations in step S6, the final high-activity mutation set is obtained, which is the top M mutations with the highest activity values selected in the last round.

Images