A protein salt sensitivity prediction method based on feature engineering and ensemble learning

Abstract

The application belongs to the field of bioinformatics, and relates to a protein salt sensitivity prediction method based on feature engineering and ensemble learning. First, by inputting known low, medium and high salt concentration protein sequence data and performing data cleaning and length standardization, the system automatically extracts ten key physical and chemical characteristics such as net charge density and charge asymmetry. Second, a multi-model learning framework is built using random forest, gradient boosting tree and support vector machine, the integrated weight is dynamically allocated based on the accuracy of each model on the validation set, and the final classification result is generated by using the weighted voting mechanism. Finally, the integrated model is quantitatively evaluated and saved, and then new protein sequences are predicted, and the prediction category, confidence and feature importance analysis are output. The application realizes high-precision and interpretable prediction of protein salt sensitivity, and provides condition optimization support for liquid-liquid phase separation research and drug delivery system.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of bioinformatics, and particularly relates to a protein salt sensitivity prediction method based on feature engineering and ensemble learning. The method can automatically extract the physicochemical features of protein sequences, realize high-precision salt concentration sensitivity prediction through multi-model ensemble learning, and provide condition optimization support for liquid-liquid phase separation research and drug delivery systems. BACKGROUND

[0002] Liquid-liquid phase separation (LLPS) plays an important role in cell function regulation and drug delivery carrier design, and salt concentration is one of the key environmental factors affecting its behavior. Traditional experimental methods such as turbidity determination and microscopic observation are costly and time-consuming, making them difficult to apply on a large scale. Existing computational prediction methods mainly face three major bottlenecks. ① Single feature representation: Most methods rely on simple sequence composition, such as amino acid frequency or a few global physicochemical properties, such as average hydrophobicity and isoelectric point, which fail to fully capture the complex physicochemical patterns of local regions in the sequence that are directly related to salt sensitivity, such as charge distribution asymmetry and hydrophobic fragment aggregation tendency. ② Model bias and instability: Single machine learning models such as support vector machines or decision trees are sensitive to data distribution and have significantly reduced generalization ability when facing class imbalance, such as uneven sample numbers under different salt concentrations, and the model performance fluctuates greatly. ③ Lack of interpretability: The prediction model cannot provide the physicochemical basis for its decision-making, making it difficult to understand the biological mechanisms behind the prediction results. Therefore, there is an urgent need for a protein salt sensitivity prediction method that can deeply integrate multi-dimensional interpretable features, improve model robustness through ensemble learning, and output feature contribution. SUMMARY

[0003] The present application proposes a protein salt sensitivity prediction method based on feature engineering and ensemble learning, aiming to solve the problems of insufficient feature representation, large model bias and lack of interpretability in the prior art. The specific technical solution includes the following 5 steps:

[0004] Step 1, Physicochemical feature extraction. Input protein sequence data of known salt concentration low, medium and high categories for data cleaning and length standardization. Then, 10 key physicochemical features are extracted from the protein sequence, including net charge density, charge asymmetry, average hydrophobicity, hydrophobic moment, polar residue proportion, proline content, glycine content, aromatic amino acid, sequence complexity and low complexity region proportion.

[0005] Step 2, Multi-model training and prediction. A multi-model learning framework is built using random forest, gradient boosting tree and support vector machine, each model is optimized by grid search for hyperparameters, and stratified sampling is used to divide the training set and validation set.

[0006] Step 3, Weighted voting decision. Based on the accuracy of each model on the validation set, the integration weight is dynamically allocated, and the weighted voting mechanism is used to fuse the prediction probability of each model to obtain the final salt concentration sensitivity classification result.

[0007] Step 4, Quality assessment and model output. The performance of the integrated model is quantitatively evaluated, and the accuracy, F1 score and other indicators are output, and the model with the best performance and feature standardizer are saved.

[0008] Step 5, Prediction and interpretation. Load the model and standardizer saved in step 4, perform feature extraction and classification prediction on the new protein sequence, and output the predicted category, confidence and feature importance analysis.

[0009] A protein salt sensitivity prediction method based on feature engineering and ensemble learning, step 1 implementation process is as follows: the system reads the FASTA format file classified by salt concentration from the directory specified by the user, the file contains protein sequences under low, medium and high salt concentration conditions. Perform data cleaning, first remove invalid sequences with sequence length less than 10 amino acids; second, convert the sequence to uppercase letters to ensure consistency of amino acid symbols; finally, truncate the sequence longer than 1000 to take the first 1000 amino acids to control the computational complexity, forming a standardized sequence list. Then, automatically extract the 10-dimensional feature vector of the input data, including: net charge density f1 , calculate the sequence static charge based on the pre-defined amino acid charge table ({' D ':-1, ' E ': -1, ' K ': 1, ' R ': 1, ' H ': 0.5}) and divide by the sequence length for normalization; charge asymmetry f2 : divide the sequence into two halves, calculate the net charge difference of the two parts, the formula is: charge_ asymmetry = | left_charge - right_charg e| / (| left_charge | + | right_charge | + ); Left_charge represents the net charge value of the first half of the protein sequence; right_charge represents the net charge value of the second half of the protein sequence; is a very small constant used to prevent division by zero and ensure numerical stability of the formula.

[0010] Average hydrophobicity f3 , the arithmetic mean of the hydrophobicity of all amino acids in the sequence is calculated using the Eisenberg hydrophobicity scale; Hydrophobic moment f4 : The hydrophobicity vector sum of each window is calculated using a sliding window with a window length of 11 amino acids, and the formula is moment += hp ×cos(2 ×π× j / window ), and the maximum moment value in all windows is taken as the feature. Polar residue ratio f5： The number of polar amino acids N, Q, S, T, Y, C, D, E, K, R, H in the sequence is counted, and the proportion of the sequence length is calculated; Proline content f6 : Calculate the frequency of proline P in the sequence; Glycine content f7 : Calculate the frequency of glycine G in the sequence; Aromatic amino acid content f8: Count the total frequency of phenylalanine F, tryptophan W, and tyrosine Y; Sequence complexity f9 : Calculate the ratio of the number of unique amino acids in the sequence to the total length of the sequence; Low complexity region ratio f10 : Use a sliding window with a window size of 10 to scan, and when the proportion of unique amino acids in the window is less than the threshold value 0.7, it is judged as a low complexity region, and the proportion of all such windows to the total window is counted.

[0011] A protein salt sensitivity prediction method based on feature engineering and ensemble learning, the implementation process of step 2 is as follows:

[0012] First, the feature data set obtained in step 1 is standardized using StandardScaler to eliminate the influence of dimension; Then, use the train_test_split function to divide the training set and validation set in a stratified manner with a ratio of 8:2, to ensure consistent proportions of each category; Then, three base learners are trained in parallel:

[0013] ① Random Forest Model: Using sklearn.ensemble.RandomForestClassifier with the following parameter settings: n_estimators=500: specifies the number of decision trees in the forest is 500, the more the number, the more stable the model, not easy to overfit, but the higher the computational cost; max_depth=15: limits the maximum depth of each decision tree to 15 layers, preventing the tree from growing too deep and becoming too complex, which is a key parameter to control overfitting; min_samples_split=5: the minimum number of samples required for internal node splitting is 5, only when the number of samples in a node is greater than or equal to 5, this node will be further divided; min_samples_leaf=2, the minimum number of samples required for leaf nodes is 2, which, in combination with min_samples_split, further prevents the tree from overfitting; max_features='sqrt': the number of random features considered when finding the best split is the square root of the total number of features, the purpose is to introduce randomness, enhance the diversity and generalization ability of the model; bootstrap=True: enable bootstrap sampling method, randomly sample from the training set with replacement when training each tree, used to increase the diversity of the tree; class_weight='balanced': automatically adjust the weights of the classes, proportional to the inverse of the number of samples in each class, used to handle the problem of unbalanced class sample size in the training data; random_state=42: set the seed of the random number generator to 42, ensure that the results of the random process are consistent every time the code is run, making the experiment reproducible; n_jobs=-1: use all available CPU cores of the computer for parallel computing to speed up the training process.

[0014] ② Gradient Boosting Tree Model: Using sklearn.ensemble.GradientBoostingClassifier with the following parameters: n_estimators=300: specifies the number of weak learners as 300, gradient boosting corrects the errors of the previous trees by adding trees one by one; learning_rate=0.05: sets the learning rate to 0.05, which controls the contribution of each tree to the final model, a smaller learning rate usually means more trees are needed to achieve good performance, but the model may be more robust; max_depth=8: limits the maximum depth of each weak learner decision tree to 8 layers, in gradient boosting, trees are usually shallow and are called "stumps"; min_samples_split=10: the minimum number of samples required for internal node splitting is 10; min_samples_leaf=4: the minimum number of samples required for leaf nodes is 4; subsample=0.8: sets the subsample ratio to 0.8, when training each tree, only 80% of the training data is randomly used, which can increase the diversity of the model and its anti-overfitting ability; random_state=42: sets the random seed to 42 to ensure reproducibility of results.

[0015] ③ Support Vector Machine Model: Using sklearn.svm.SVC with the following parameters: C=1.0: This is the regularization parameter that balances model complexity and training error, the smaller the C value, the more errors are tolerated, and the smoother the decision boundary (strong regularization), the larger the C value, the more inclined to fit all training data, which may lead to overfitting (weak regularization); kernel='rbf': specifies the kernel function as "Radial Basis Function", which is a nonlinear kernel that can map data to high-dimensional space to find a nonlinear decision boundary; gamma='scale': sets the coefficient of the RBF kernel,'scale' is the default option, whose value is 1 / ( n_ features × X.var ()) where n_features represents the number of features in the dataset, i.e. the dimension, X-varThe total variance of all feature values in the entire training set X, the purpose of taking the inverse of the product is to automatically calculate a suitable value according to the number of features and feature variance, the Gamma value affects the shape of the decision boundary, the larger the value, the more the model tends to fit each training sample; class_weight='balanced': automatically adjust the weight of the class to deal with the problem of data imbalance, the weight is inversely proportional to the frequency of the class; probability=True: enable probability estimation, after training, the support vector machine can output the probability that the sample belongs to each class, not just the predicted class, which provides a basis for implementing weighted voting and other operations; random_state=42: set the random seed to 42 to ensure that the results are reproducible in operations involving randomness.

[0016] Each model calls the fit method on the training set for fitting and the predict method on the validation set for prediction, and uses accuracy_score to calculate the accuracy of each validation set.

[0017] A protein salt sensitivity prediction method based on feature engineering and ensemble learning, step 3 implementation process as follows:

[0018] Perform weighted voting consensus decision. Let the accuracy of random forest, gradient boosting tree and support vector machine on the validation set be . First, calculate the weight of each model: , , , , where is the sum of the accuracy of random forest, gradient boosting tree and support vector machine on the validation set. For samples in the validation set, get the prediction probability vector , of the three models. The final ensemble probability is: . The final ensemble prediction label is determined by the class corresponding to the maximum probability value in . At the same time, select the single model with the best performance on the validation set as the final deployment model.

[0019] A protein salt sensitivity prediction method based on feature engineering and ensemble learning, step 4 implementation process as follows:

[0020] Use the prediction results of the ensemble model on the validation set to calculate the overall accuracy and macro average F1 score. Extract the feature_importances_ attribute from the trained random forest model to get the contribution ranking of the 10 physical and chemical features to the prediction result. Save the best performance model final_model and feature standardizer scaler to the specified directory through the joblib.dump method.

[0021] A protein salt sensitivity prediction method based on feature engineering and ensemble learning, the step 5 implementation process is as follows:

[0022] For a new protein sequence, the system first loads the saved model and standardizer. Then, using the same AdvancedSaltFeatureExtractor class as step 1 to extract the 10-dimensional physicochemical features of the sequence. After the feature vector is converted by the standardizer, it is input into the loaded model to call the predict method to get the predicted category, and the predict_proba method to get the probability belonging to each category. At the same time, the system outputs the main physicochemical features and their importance on which the prediction is based. BRIEF DESCRIPTION OF DRAWINGS

[0023] Figure 1 It is a general architecture diagram of a protein salt sensitivity prediction method based on feature engineering and ensemble learning.

[0024] Figure 2 It is a flowchart of the physicochemical feature extraction module.

[0025] Figure 3 It is a flowchart of the multi-model ensemble training module.

[0026] Figure 4 It is a flowchart of the weighted voting decision and model saving module. DETAILED DESCRIPTION

[0027] The present application will be described in detail below in conjunction with the drawings and examples.

[0028] Physicochemical feature extraction. As Figure 1 and Figure 2As shown, this step includes two stages: data input and feature calculation. In the data input stage, first, the system reads the classified FASTA files from the specified directory; second, it uses the custom load_fasta_files function to read low_concentration_sequences.fasta, mid_concentration_sequences.fasta, and high_concentration_sequences.fasta, respectively, to obtain the sequence lists low_salt_seqs, mid_salt_seqs, and high_salt_seqs, and the corresponding label list all_labels. Then, the system performs data cleaning operations: using list comprehension to filter out sequences with a length less than 10; then, using the str.upper() method to unify the amino acid symbols; and finally, truncating long sequences by slicing operation seq[:1000]. In the feature calculation stage, first, the system instantiates the AdvancedSaltFeatureExtractor class. This class defines three core dictionaries: the Eisenberg scale hydrophobicity, the charge value charge, and the polarity amino acid set polarity when initialized. Then, it calls the extract_all_features method for each sequence in a loop. In this example, the sequence "MKTVRQERLKSIVRILERSKEPVSGAQ":

[0029] ① Calculate the net charge density: the sequence contains K (2, +1), R (3, +1), E (3, -1), D (0), and H (0). The net charge = 2x1 + 3x1 + 3x(-1) = 2. The sequence length is 26, so the net charge density = 2 / 26 ≈ 0.0769.

[0030] ② Calculate the hydrophobic moment: set the window length to 11. Using sliding calculation, for example, starting from the first letter 'M', take the window "MKTVRQERLKS", calculate the sum of the hydrophobic moment vector of the 11 amino acids in the window. After traversing all 16 windows, assume that the maximum moment value obtained is 1.2, then the normalized hydrophobic moment feature is 1.2 / 11 ≈ 0.109.

[0031] Finally, through the above processing, the system generates a feature vector array physics_features for all sequences, with a dimension of (n_samples, 10).

[0032] Multi-model ensemble training. As shown in the following code: Figure 3As shown, this step includes data preprocessing, model training and validation. First, use StandardScaler() to standardize physics_features, get physics_features_scaled. Then, use train_test_split(physics_features_scaled, all_labels, test_size=0.2, stratify=all_labels, random_state=42) to divide the data, get X_train, X_val, y_train, y_val. Then, initialize and train three models in turn:

[0033] ① Random Forest: Instantiate RandomForestClassifier and set the parameters, call rf_model.fit(X_train, y_train) for training. After training, call rf_val_pred = rf_model.predict(X_val) and rf_val_acc = accuracy_score(y_val, rf_val_pred) for evaluation.

[0034] ② Gradient Boosting Tree: Instantiate GradientBoostingClassifier and set the parameters, call gb_model.fit(X_train, y_train) for training. After training, call gb_val_pred = gb_model.predict(X_val) and gb_val_acc = accuracy_score(y_val, gb_val_pred) for evaluation.

[0035] ③ Support Vector Machine: Instantiate SVC and set the parameters, call svm_model.fit(X_train, y_train) for training. After training, call svm_val_pred = svm_model.predict(X_val) and svm_val_acc = accuracy_score(y_val, svm_val_pred) for evaluation.

[0036] Weighted voting decision and model saving. As shown, Figure 4As shown, this step first calculates the ensemble weights, which are computed from the accuracy of the previous step. For the validation set samples, the predict_proba method of each model is called to obtain the probabilities, which are then weighted and fused ensemble_probs = weights[0] x rf_probs + weights[1] x gb_probs + weights[2] x svm_probs. The final ensemble prediction label is ensemble_pred = np.argmax(ensemble_probs, axis=1). The random forest model with the best performance on the validation set is chosen as the final model. The data is merged using X_full_train = np.vstack([X_train, X_val]) and y_full_train = y_train + y_val, and the final_model.fit(X_full_train, y_full_train) is called to retrain the model on the full data. Finally, the model and the scaler are saved using joblib.dump(final_model, "ensemble_final_model.pkl") and joblib.dump(scaler, "feature_scaler.pkl").

[0037] Prediction and interpretation. For a new sequence, the system first loads the saved model and scaler. Then, the same AdvancedSaltFeatureExtractor instance is used to extract its 10-dimensional feature vector. This vector is input into the loaded random forest model after being converted by the scaler. The predict method is called to obtain the predicted class, and the predict_proba method is called to obtain the probability. At the same time, the feature_importances_ are extracted from the model, and it is shown that "charge asymmetry" and "hydrophobic moment" are the two most important features for making this prediction.

[0038] The model parameters described in the present application are determined in advance through grid search and cross-validation. To verify the effect of the present application, the benchmark test set is tested, and compared with the MambaPhase method based on deep learning, the accuracy of the present application is improved from 77% to 89.4%, the interpretability is significantly improved, the demand for large-scale computing resources is reduced, and the effectiveness and superiority of the present application are proved.

[0039] The above is further detailed description of the present application in combination with specific preferred embodiments, and cannot be deemed as limitation of the specific implementation of the present application to these descriptions. For those skilled in the art to which the present application belongs, without departing from the concept of the present application, a number of simple deductions or substitutions can be made, and all of them shall be deemed as falling within the protection scope of the present application.

Claims

1. A protein salt sensitivity prediction method based on feature engineering and ensemble learning, characterized in that, Includes the following steps: Step 1, Physicochemical Feature Extraction: Input protein sequence data of known low, medium and high salt concentration categories, perform data cleaning and length standardization. The data comes from FASTA format files that have been classified according to salt concentration. Subsequently, 10 key physicochemical features were extracted from the protein sequence, including: net charge density, charge asymmetry, average hydrophobicity, hydrophobic moment, proportion of polar residues, proline content, glycine content, aromatic amino acid content, sequence complexity, and proportion of low-complexity regions. Step 2, Multi-model training and prediction: A multi-model learning framework is built using random forest, gradient boosting tree and support vector machine. Each model is optimized for hyperparameters through grid search. The training set and validation set are divided by stratified sampling. Step 3, Weighted Voting Decision: Based on the accuracy of each model on the validation set, the ensemble weights are dynamically allocated, and the predicted probabilities of each model are fused using a weighted voting mechanism to obtain the final salt concentration sensitivity classification result. Step 4, Quality Assessment and Model Output: Quantitatively evaluate the performance of the ensemble model, output the accuracy and F1 score, and save the best-performing model and feature normalizer. Step 5, Prediction and Interpretation: Load the model and normalizer saved in Step 4, extract features and predict the classification of the new protein sequence, and output the predicted category, confidence level and feature importance analysis.

2. The method according to claim 1, characterized in that, In step 1, the data cleaning operation includes: first, removing invalid sequences with a length of less than 10 amino acids; second, converting the sequences to uppercase letters to ensure consistency of amino acid symbols; and finally, truncating sequences with a length of more than 1000 amino acids and taking the first 1000 amino acids to control computational complexity and form a standardized sequence list.

3. The method according to claim 1, characterized in that, In step 2, the random forest model parameters are set as follows: n_estimators=500: This specifies that the number of decision trees in the forest is 500. The more trees there are, the more stable the model is and the less prone it is to overfitting, but the computational cost is also higher; max_depth=15: This limits the maximum depth of each decision tree to 15 layers to prevent the trees from growing too deep and becoming too complex. This is a key parameter for controlling overfitting. `min_samples_split=5`: The minimum number of samples required for an internal node to be split is 5. A node will only be split if its sample count is greater than or equal to 5. `min_samples_leaf=2`: Leaf nodes require at least 2 samples. Combined with `min_samples_split`, this further prevents the tree from overfitting. `max_features='sqrt'` : Each time the best split is sought, the number of random features considered is the square root of the total number of features. The purpose is to introduce randomness and enhance the model's diversity and generalization ability; bootstrap=True: Enables bootstrap sampling, which randomly draws samples with replacement from the training set during the training of each tree to increase the diversity of the trees; class_weight='balanced': Automatically adjusts the class weights to be proportional to the inverse of the number of class samples to handle the problem of imbalanced class samples in the training data; random_state=42: Sets the seed of the random number generator to 42 to ensure that the results of the random process are consistent each time the code is run, making the experiment reproducible; n_jobs=-1: Uses all available CPU cores of the computer for parallel computing to accelerate the training process.

4. The method according to claim 1, characterized in that, In step 3, the final adaptive weights The accuracy of Random Forest, Gradient Boosting Tree, and Support Vector Machine on the validation set is calculated using the following formulas: ; , , ,in This refers to the sum of the accuracies of the three models—random forest, gradient boosting tree, and support vector machine—on the validation set; the final ensemble probability is: The final integrated prediction label is generated by The category corresponding to the highest probability value is determined.

Images