Methods and online tools for predicting peptide function based on general large language models

Abstract

The application discloses a method and an online tool for predicting peptide functions based on a general large language model, and the method comprises the following steps: S1), data collection, cleaning and arrangement; S2), data preprocessing; S3), PepUniBERT model construction; S4), pre-training; S5), PepUniBERT model architecture optimization; S6), fine-tuning; S7), polypeptide attribute prediction; and S8), model evaluation. The tool integrates the trained model into a platform PepMol, supports input of sequences in fasta, csv and xlsx formats, and automatically calculates and feeds back the results through an email after a user inputs a polypeptide sequence and selects a prediction task. The tool overcomes the limitation of single task of existing tools, the prediction result is accurate and reliable, the application threshold is low, the analysis efficiency of researchers on peptide molecular functions is significantly improved, and the application and transformation of peptides in the fields of drug research and development, functional food and synthetic biology are promoted.

Description

Technical Field

[0001] This invention relates to the fields of bioinformatics, computational biology, and drug screening, specifically to a method and online tool for predicting peptide function based on a general large language model. Background Technology

[0002] Peptides are an important class of bioactive molecules that lie between small molecules and proteins. Due to their moderate molecular weight, flexible structure, and ease of chemical modification, they have broad application potential in fields such as medicine, food, and synthetic biology. For example, antimicrobial peptides are highly effective in combating drug-resistant bacteria, antioxidant peptides can scavenge free radicals and protect the cardiovascular system, and immunomodulatory peptides can be used for inflammation suppression and vaccine adjuvant research.

[0003] However, the development of functional peptides still heavily relies on experimental screening, including sequence design, in vitro synthesis, and functional detection. This process is not only time-consuming and labor-intensive but also costly, making efficient exploration difficult within the vast amino acid sequence space. In recent years, deep learning technology has been applied to peptide function prediction, with methods such as convolutional neural networks, recurrent neural networks, and graph neural networks achieving some success in certain specific tasks. However, existing methods still have significant limitations. On the one hand, most models focus only on single-function prediction, lacking cross-task versatility and failing to meet researchers' needs for multi-dimensional functional analysis. On the other hand, peptide-related data is scattered across different databases with inconsistent formats, resulting in high integration costs and reduced data utilization efficiency. Furthermore, most existing tools are geared towards users with specialized backgrounds, making them difficult for ordinary researchers to use directly, and a unified platform is lacking to achieve integrated operations from prediction to data retrieval.

[0004] Deep learning methods have demonstrated remarkable generalization ability and versatility in multiple fields, including protein generation, drug-target affinity prediction, and protein structure prediction. Many studies combine protein sequences with large-scale pre-trained language models, such as ESM and ProtTrans. These models, pre-trained on hundreds of millions of sequences, perform exceptionally well in downstream tasks, such as subcellular localization prediction, structure prediction, and function prediction based solely on sequence data. Since peptides share the same amino acid composition and exhibit similar biological activities as proteins, large-scale language models used for proteins are expected to be equally applicable to peptide research. The UniDL4BioPep model focuses on predicting peptide biological activity; the PeptideBERT model is used to predict properties such as hemolysis and solubility of peptides; DeepLoc2.0 utilizes protein language models for subcellular localization prediction; and the work of Jacob Witten and Zack Witten employs deep learning regression models to predict the minimum inhibitory concentration (MIC) values ​​of antimicrobial peptides.

[0005] Currently, various techniques are available for the study and prediction of peptide function, but these methods have limitations in accuracy, versatility, and applicability. Traditional experimental methods can directly obtain reliable functional verification results, but they are time-consuming, costly, and difficult to handle large-scale sequence screening tasks. With the improvement of computing power, machine learning and deep learning methods have been gradually introduced into peptide function prediction. These methods can reduce experimental workload and improve prediction efficiency to some extent. However, existing models are mostly designed for specific tasks, such as targeting single functions like antimicrobial peptides, hemolytic activity, or subcellular localization, lacking cross-task versatility. Furthermore, these models mostly rely on limited labeled datasets, making it difficult to fully utilize large-scale unlabeled sequence information, resulting in insufficient generalization ability and stability. While some deep learning models have made significant progress at the protein level, their performance often degrades or applicability is limited when directly transferred to peptide molecular research. In addition, most existing prediction tools are provided in command-line or local deployment form, requiring high computational resources and having a high barrier to entry, hindering direct application by a wide range of biological researchers. Summary of the Invention

[0006] To address the aforementioned technical problems, this invention provides a method and online tool for predicting peptide function based on a general large language model, solving the problems of insufficient universality, low prediction accuracy, low data utilization, and high application threshold of existing functional peptide prediction methods.

[0007] The specific plan is as follows: The method for predicting peptide function based on a general large language model includes the following steps: S1): Data collection, cleaning and organization, collecting pre-training data and fine-tuning data, removing sequences containing illegal characters or non-natural amino acids, and ensuring the diversity and representativeness of data by removing redundancy through similarity clustering; S2): Data preprocessing, using byte-to-byte encoding (BPE) for word segmentation, and using a greedy algorithm to generate vocabularies of different sizes; S3): Model building, based on the BERT architecture, design the PepUniBERT model, treat peptide sequences as natural language, and model the contextual dependencies between amino acids; S4): Pre-training, using the masked language model MLM to model the task and learn a general sequence representation; S5): Optimize the PepUniBERT model architecture by finding the optimal configuration from three dimensions: vocabulary size, model depth, and normalization method, to balance computational cost and performance. S6): Fine-tuning. During fine-tuning, the PepUniBERT model inherits the weights from pre-training, follows the principle of transfer learning, and uses [token] tags to aggregate global information of the sequence for classification / regression. To address overfitting on small samples, amino acid random masks are used for data augmentation, and stratified random sampling is implemented. To address class imbalance, improved FocalLoss and data augmentation strategies are used to improve the model's performance on small and imbalanced datasets. S7): Peptide attribute prediction uses a pre-trained and fine-tuned PepUniBERT peptide big language model to encode the input peptide sequence and connect binary, multi-class, or regression task heads based on the special [token] vector at the beginning of the sequence, to predict the functions of various peptides, including blood-brain barrier permeability peptides (BBBP / BBBPV). S8): Model Evaluation: The model is evaluated on independent test sets and external validation sets using multiple metrics including accuracy (ACC), area under the curve (AUC), precision (Precision), sensitivity (Sen), specificity (Spe), F1 score, and Matthews correlation coefficient (MCC).

[0008] The pre-training data collection involves obtaining large-scale non-redundant protein sequences from the UniRef50 database, screening entries with a length ≤128 amino acids, and constructing a pre-training dataset. The pre-training dataset contains no fewer than 20 million sequences and is divided into a training set and a test set in a 9:1 ratio. The pre-training dataset consists of unlabeled peptides.

[0009] The fine-tuning data consists of experimentally validated functional peptide sequences collected and organized from the PubChem database. This includes retrieving literature from the PubChem database based on functional keywords, filtering the literature using Tongyi Qianwen AI, extracting sequences, parsing them with Python scripts, and manually verifying them to remove redundancy, non-natural modifications, and D-type amino acids to ensure the data's authenticity and validity. The fine-tuning data comprises no fewer than 20 categories and consists of tagged peptides.

[0010] The PepUniBERT model consists of an input layer, a multi-layer Transformer encoder, and a fully connected layer. The input layer includes an input embedding layer and a positional encoding layer. The input embedding layer transforms the amino acid sequence into a vector representation; Positional Encoding injects sequence information into each position in the sequence through learnable absolute position embeddings; The Transformer encoder includes a multi-head attention layer, a feedforward network layer, and residual connections and layer normalization (Add & Norm) to extract and fuse sequence features layer by layer. The fully connected layer is used to predict the occluded token during the pre-training phase. During the fine-tuning phase, the output layer is adjusted so that only the special token at the beginning of the sequence is input into the fully connected layer, and the results are generated according to the specific task. The result generation includes outputting 0 or 1 for binary classification tasks, outputting class probabilities for multi-class tasks, and predicting numerical values ​​for regression tasks.

[0011] The masked language model MLM described in S4) uses an improved dynamic mask, in which 15% of the [token] in each sequence is randomly selected as the prediction target, of which 80% is replaced with the special symbol [MASK], 10% is replaced with a random [token], and the rest remain unchanged.

[0012] In S5), the optimization of the PepUniBERT model architecture includes the following steps: Q1): Vocabulary optimization: Fix an 8-layer Transformer as the baseline model and test the impact of different vocabulary sizes on model performance in order to find an optimal vocabulary size that can fully cover the semantic units of the peptide sequence, while avoiding sparse word vectors or redundant model parameters due to an excessively large vocabulary. Q2): Depth optimization: After determining the optimal vocabulary, compare and test the performance of models with depths of 8, 12, and 16 layers to find the best balance between computational cost and model performance, and select the optimal number of layers as the standard configuration. Q3): Normalization optimization. Drawing on the experience of the advanced model Llama2.0, we compared the root mean square normalization (RMSNorm) and the traditional layer normalization (LayerNorm). Through experiments, we compared the differences between the two in terms of convergence speed, final accuracy, and generalization ability, and selected the most suitable normalization scheme for the peptide language model. In S6), the method for data augmentation using the amino acid random mask is to randomly replace one residue in each sequence with a placeholder X, generate a new sample each time, and the number of replacements does not exceed the total length of the sequence. The augmentation factor is set to 2, 4, 6, 8 or 10 times. In S6), the improved FocalLoss function is: in, p t It is the model's predicted probability of the true class. a t These are class weight parameters used to balance the proportion of positive and negative samples. γ It is a moderating factor used to reduce the loss contribution of easily classified samples. γ The larger the value, the more attention the model pays to difficult-to-classify samples; An online tool for predicting peptides based on a general large language model integrates the trained model into a web platform and supports input sequences in FASTA, CSV, or XLSX formats. Users input peptide sequences and select prediction tasks, and the platform automatically calculates and sends the results back via email. This invention discloses a method and online tool for predicting peptide function based on a general large language model. It uses a large language model to replace traditional sequence alignment or shallow learning methods, resulting in faster prediction speed and lower computational cost after pre-training. It achieves cross-task prediction through a unified PepUniBERT model, overcoming the single-task limitation of existing tools. It develops a technology that can efficiently integrate large-scale sequence information, has cross-task universality, provides accurate and reliable prediction results, and has a low application threshold. In particular, experimental results on the blood-brain barrier permeability peptide (BBBP / BBBPV) dataset show that the PepUniBERT peptide large language model of this invention outperforms existing protein language models and traditional machine learning methods in terms of indicators such as area under the curve (AUC) and Matthews correlation coefficient (MCC), significantly improving the accuracy and robustness of blood-brain barrier permeability prediction.

[0013] In addition, a novel peptide function prediction tool based on a large language model was developed and implemented using an open online platform, which significantly improves the efficiency of researchers in analyzing peptide molecular functions and promotes the application and transformation of peptides in drug development, functional foods and synthetic biology. Attached Figure Description

[0014] Figure 1 This is a diagram illustrating the data collection and cleaning process.

[0015] Figure 2 This describes the structure and training process of the PepUniBERT model.

[0016] Figure 3 This is a detailed data table of the strategies used during the training of the PepUniBERT model.

[0017] Figure 4 This is the Focal Loss parameter settings table.

[0018] Figure 5 This is a comparison of the MCC results between the PepUniBERT model and the reported model in the literature on the binary classification task of bioactive peptide dataset.

[0019] Figure 6 shows the process of using and automating web servers. Detailed Implementation

[0020] The technical solutions in the embodiments of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the implementation of the present invention, and not all of it. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0021] To address the shortcomings of insufficient prediction efficiency, limited functional coverage, and low data utilization in existing peptide molecule research, this paper proposes PepUniBERT, a large-scale language model based on Transformer, and constructs a corresponding online platform. This approach uses the contextual dependence of amino acid sequences as a starting point, forming a complete system from pre-training and task transfer to online deployment and prediction submission. This invention belongs to the interdisciplinary field of bioinformatics and artificial intelligence, proposing a large-scale language model for predicting the function and physicochemical properties of peptide molecules, and achieving automated prediction through an online platform. It is applicable to drug development, functional food development, and synthetic biology research. The overall technical solution includes the following steps: The method for predicting peptide function based on a general large language model includes the following steps: S1): Data collection, cleaning and organization, collecting pre-training data and fine-tuning data, and removing sequences containing illegal characters or natural amino acids, and using similarity clustering to remove redundancy to ensure the diversity and representativeness of the data; like Figure 1 As shown, the training data is divided into two parts: pre-training and fine-tuning. The pre-training data collection is to obtain a large number of non-redundant protein sequences from the UniRef50 database, and to screen entries with a length of ≤128 amino acids to cover natural peptides and short-chain proteins, which are used to construct the pre-training dataset. The number of sequences in the pre-training dataset is no less than 20 million, and it is divided into training set and test set in a 9:1 ratio. The pre-training dataset is unlabeled peptides.

[0022] The fine-tuning data consists of experimentally validated functional peptide sequences collected and organized from the PubChem database. This includes retrieving literature from the PubChem database based on functional keywords, filtering the literature using Tongyi Qianwen AI, extracting sequences, parsing them with Python scripts, and manually verifying them to remove redundancy, non-natural modifications, and D-type amino acids to ensure the data's authenticity and validity. The fine-tuning data comprises no fewer than 20 categories and consists of tagged peptides.

[0023] S2): Data preprocessing, using byte-to-byte encoding (BPE) for word segmentation, and using a greedy algorithm to generate vocabularies of different sizes; To address the short-chain characteristics and local dependence of functional motifs in peptide sequences, Byte Pair Encoding (BPE) was employed as the core word segmentation method in the data preprocessing stage. Unlike traditional single-amino acid encoding, BPE effectively identifies and captures conserved structural patterns and functional motifs in peptide sequences by iteratively merging high-frequency subword units.

[0024] A training set was constructed based on 560,000 high-quality sequences from the Swiss-Prot database, and a greedy algorithm was used to generate three different vocabularies of varying sizes: 24, 1024, and 4096. The 24-vocabulary covers 20 standard amino acids and common modification markers; the 1024-vocabulary can characterize common dipeptide or tripeptide combinations; and the 4096-vocabulary retains over 85% of the features of functional short peptide sequences, providing the model with multi-level sub-word granular representations.

[0025] S3): Model building: Based on the BERT architecture, the PepUniBERT model is designed, which treats peptide sequences as natural language and models the contextual dependencies between amino acids; it achieves more refined semantic feature extraction and provides high-quality input for downstream prediction tasks.

[0026] like Figure 2 As shown, the main body of the PepUniBERT model includes an input layer, a multi-layer Transformer encoder, and a fully connected layer. To ensure that positional relationships are effectively captured, position and label encoding is added before input to convert the amino acid sequence into a vector representation. The input layer includes an embedding layer and a position and label encoding layer. The input embedding layer converts the amino acid sequence into a vector representation; Positional Encoding injects sequence information into each position in the sequence through learnable absolute position embeddings; The core mechanism of the Transformer encoder is bidirectional self-attention, which can simultaneously integrate information from the left and right texts, thereby improving the overall understanding and modeling accuracy of peptide sequences. The Transformer encoder includes a multi-head attention layer, a feedforward network layer, and residual connections and layer normalization (Add & Norm) to extract and fuse sequence features layer by layer. The vectors output by the multi-layer encoder are ultimately input into a fully connected layer. This fully connected layer is used during pre-training to predict the occluded tokens. During fine-tuning, the output layer is adjusted so that only the specific token at the beginning of the sequence is input into the fully connected layer, and results are generated according to the specific task. These results include outputting 0 or 1 for binary classification, outputting class probabilities for multi-class classification, and predicting numerical values ​​for regression tasks. This design maintains the knowledge transfer from the pre-training stage while improving the model's adaptability and generalization performance in specific tasks.

[0027] S4): Pre-training, using the masked language model MLM to model the task and learn a general sequence representation; The masked language model MLM employs an improved dynamic mask to enhance the model's context-dependent modeling capabilities. In each sequence, 15% of the [tokens] are randomly selected as prediction targets, of which 80% are replaced with the special symbol [MASK], 10% are replaced with random [tokens], and the remainder remain unchanged.

[0028] This approach forces the model to rely on contextual information while introducing moderate noise to enhance robustness. The core task of pre-training is masked language modeling (MLM), which requires the model to predict the masked [token] based on the context, thereby gradually learning the deep semantics and local dependencies of the peptide sequence, laying the foundation for subsequent functional prediction tasks.

[0029] S5): Optimize the PepUniBERT model architecture by finding the optimal configuration from three dimensions: vocabulary size, model depth, and normalization method, to balance computational cost and performance. To construct a PepUniBERT model with optimal performance, system optimizations were performed in three aspects: vocabulary size, number of model layers, and normalization method. Detailed data used for model training, such as... Figure 3 As shown; the specific optimization steps are as follows: Q1): Vocabulary optimization: Fix an 8-layer Transformer as the baseline model and test the impact of different vocabulary sizes on model performance in order to find an optimal vocabulary size that can fully cover the semantic units of the peptide sequence, while avoiding sparse word vectors or redundant model parameters due to an excessively large vocabulary. Q2): Depth optimization: After determining the optimal vocabulary, compare and test the performance of models with depths of 8, 12, and 16 layers to find the best balance between computational cost and model performance, and select the optimal number of layers as the standard configuration. The relationship between depth and performance was examined. Shallow models are computationally efficient but may struggle to capture long-range dependencies; deep models, while computationally expensive, can extract richer semantic features, thus improving the accuracy of downstream tasks. Through comparative experiments, the structure that achieves the best trade-off between computational cost and performance was ultimately selected as the standard configuration.

[0030] Q3): Normalization optimization. Drawing on the experience of the advanced model Llama2.0, we compared the root mean square normalization (RMSNorm) and the traditional layer normalization (LayerNorm). Through experiments, we compared the differences between the two in terms of convergence speed, final accuracy, and generalization ability, and selected the most suitable normalization scheme for the peptide language model.

[0031] Root mean square normalization (RMSNorm) avoids mean shift by scaling with the root mean square, making it more computationally efficient and potentially improving training speed and generalization ability. In contrast, traditional layer normalization (LayerNorm) is stable in BERT training but may introduce additional bias in deep networks. Experimental results show differences in convergence speed, accuracy, and generalization performance between the two methods, providing a basis for choosing a more suitable normalization scheme for peptide language modeling.

[0032] S6): Fine-tuning. During fine-tuning, the PepUniBERT model inherits the weights from pre-training and optimizes the parameters for the specific prediction task. Following the principle of transfer learning, the input layer maintains the same [token] partitioning rules and adds a learnable [token] label at the beginning of the sequence to aggregate global information. The output layer is transformed into a classification or regression head based on [token] representation to adapt to binary classification, multi-class classification, or regression tasks, ensuring that the pre-trained knowledge is fully transferred and plays a role in the target task.

[0033] To address overfitting on small samples, amino acid random masking is used for data augmentation, along with stratified random sampling. This data augmentation strategy is introduced into the fine-tuning of the binary classification task. Specifically, one residue is randomly replaced as a placeholder X in each sequence, generating a new sample with each replacement. The number of replacements does not exceed the total sequence length, and the augmentation factor is set to 2, 4, 6, 8, and 10 times. This effectively improves the model's robustness while ensuring sequence diversity, further preventing overfitting to local data. During training, stratified random sampling is used to ensure that each batch contains sequence samples from different sources, while the test set remains unchanged, guaranteeing the reliability of the evaluation results.

[0034] To address class imbalance, where the majority class has more than twice the number of minority classes, we employ improved FocalLoss and data augmentation strategies to enhance the model's performance on small and imbalanced datasets. The improved Focal Loss function is as follows: in, p t It is the model's predicted probability of the true class. a t These are class weight parameters used to balance the proportion of positive and negative samples. γ It is a moderating factor used to reduce the loss contribution of easily classified samples. γ The larger the value, the more attention the model pays to hard-to-classify samples; the focus loss has achieved significant results in object detection and some classification tasks, especially in scenarios with severe class imbalance.

[0035] Focal Loss is an improved cross-entropy loss function proposed to address the class imbalance problem. It assigns lower weights to easily classified samples and higher weights to difficult-to-classify samples, thus making the model focus more on minority classes or hard-to-distinguish samples during training. Focal Loss is used instead of BCE Loss. By adjusting the loss distribution, Focal Loss makes the model focus more on difficult-to-classify samples during training, thereby improving the prediction accuracy of minority classes and avoiding excessive bias towards the majority class. Focal Loss has two key parameters: class weights (alpha, α, β). α ) and regulating factors (gamma, γ These two parameters were arranged and combined, and the specific parameter settings are as follows: Figure 4 As shown.

[0036] S7): Peptide attribute prediction uses a pre-trained and fine-tuned PepUniBERT peptide big language model. The [token] representation of the peptide sequence is input into the corresponding binary, multi-class, or regression output layer to predict the functions of various peptides, including blood-brain barrier permeability peptides. Peptide attribute prediction is performed by PepUniBERT.

[0037] S8): Model Evaluation: The model is evaluated on independent test sets and external validation sets using multiple metrics including accuracy (ACC), area under the curve (AUC), precision (Precision), sensitivity (Sen), specificity (Spe), F1 score, and Matthews correlation coefficient (MCC).

[0038] To evaluate model performance, various evaluation metrics were used and compared with other models. Evaluation metrics included accuracy (ACC), area under curve (AUC), precision, sensitivity (Sen), specificity (Spe), F1 score (F1), and Matthews correlation coefficient (MCC).

[0039] Specifically: For the true value, For predicted values, The mean of the true values. N This refers to the number of samples. A True Positive (TP) is the number of samples that the model correctly predicts as positive; that is, the number of samples that are actually positive and predicted as positive. A False Positive (FP) is the number of samples that the model incorrectly predicts as positive when they are actually negative. A False Negative (FN) is the number of samples that are actually positive but incorrectly predicted as negative by the model. A True Negative (TN) represents the number of samples that the model correctly predicts as negative; that is, the number of samples that are actually negative and predicted as negative.

[0040] ACC directly reflects the overall classification accuracy of the model, but in cases of imbalanced samples, accuracy may be misleading, so it needs to be analyzed in conjunction with other metrics. F1 comprehensively evaluates model performance, especially suitable for cases with imbalanced positive and negative samples. AUC is calculated using the receiver operating characteristic curve (ROC curve) and measures the model's ability to distinguish between positive and negative samples at different thresholds; the closer the AUC value is to 1, the stronger the model's generalization ability. MCC comprehensively considers the sample distribution and ranges from -1 to 1, where 1 represents perfect classification, 0 represents random prediction, and -1 represents completely incorrect prediction; it is suitable for datasets with uneven sample distribution. R² is used to evaluate the model's fit to the data in regression tasks, ranging from 0 to 1; the closer to 1, the stronger the model's interpretability.

[0041] This invention comprehensively analyzes and describes the model's performance through these multi-dimensional evaluation metrics, ensuring an accurate assessment of the model's overall capabilities. Taking MCC as an example, our model outperforms the currently reported best model on most datasets, and on a few datasets, its performance is essentially on par with other models, fully demonstrating the superiority of our model's performance. Specific results are as follows: Figure 5 As shown.

[0042] Figure 5In the diagram, the vertical axis represents the model's MCC result on the bioactive peptide dataset for the binary classification task; each letter represents a different type of bioactive peptide dataset, including: Antibacterial peptides (ABP): inhibit or kill bacteria (such as Escherichia coli and Staphylococcus aureus); Anticancer peptides (ACP): inhibit tumor cell proliferation or induce apoptosis; Antifungal peptides (AFP): inhibit or kill fungi, such as Candida albicans; Angiotensin-converting enzyme inhibitory peptides (ACEIP): lower blood pressure by inhibiting ACE enzyme activity; Antimalarial peptides (AMAP): inhibit the growth or infection of Plasmodium; Antimicrobial peptides (AMP): inhibit microbial activity; Antiparasitic peptides (APP): inhibit the activity of parasites (such as Leishmania). Antiviral peptides (AVPs): Inhibit viral replication or infection, such as influenza virus and HIV. Blood-Brain Barrier Peptides (BBBPs): Able to cross the blood-brain barrier for central nervous system drug delivery. Bitter peptides (BPs): Characterized by a bitter taste, affecting the sensory quality of food. DPPIV Inhibitory Peptides (DPPIVIPs): Inhibit dipeptidyl peptidase IV activity, regulating blood glucose levels. Antioxidant peptides (AOPs): Scavenger free radicals, mitigating oxidative stress damage. Hemolytic peptides (HPs): Assess the damaging effects of peptides on erythrocytes. Neuropeptides (NPs): Regulate nervous system function. Nonfouling peptides (NFPs): Inhibit the nonspecific adsorption of proteins or cells onto material surfaces. Quorum-sensing peptides (QSP): Interfere with bacterial quorum sensing systems, inhibiting biofilm formation. Anti-methicillin-resistant Staphylococcus aureus peptides (SCMRSA): Specifically inhibit resistant strains. Soluble peptides (SP): Evaluate the solubility of peptides in aqueous solutions. Toxic peptides (TP): Evaluate the toxic effects of peptides on cells or organisms.Tumor T-cell antigens (TTCA): activate T-cell immune responses for tumor immunotherapy. Umami peptides (UP): possess umami characteristics and are used for food flavor enhancement. ACP, AMAP, and BBBP each contain two independent subsets (original and alternative versions) with slightly different data compositions to evaluate the impact of different data construction strategies on model performance. BBBP-Alternative is a manually constructed high-quality BBBP dataset, combined with literature retrieval and data cleaning methods to ensure data accuracy and reliability. Experimental results on the blood-brain barrier permeability peptide dataset (BBBP and its high-quality variant BBBPV) show that the PepUniBERT peptide big language model of this invention outperforms existing methods in key metrics such as AUC and MCC, demonstrating that this model has significant advantages in the blood-brain barrier permeability prediction task.

[0043] An online tool for predicting peptides based on a general large language model is characterized by: integrating the trained model into a web platform, supporting input of sequences in FASTA, CSV, or XLSX formats; users input peptide sequences and select prediction tasks, and the platform automatically calculates and sends the results back via email.

[0044] This embodiment designs and develops an integrated peptide function prediction platform, PepMol. This platform aims to lower the technical barrier and improve prediction efficiency. By integrating the optimal model trained in this paper with an engineered system design, it achieves fully automated services from data input and model calculation to result feedback. After the user submits a task, the calculation results will be sent back to the user via email. Prediction time is directly proportional to sequence length and data volume. This platform can help researchers conduct large-scale early peptide screening, greatly improving work efficiency. The network server usage and automation process are as follows... Figure 6 As shown. To accommodate the operating habits of users without a computer background, an intuitive three-stage guided interactive process was designed: (1) Users can directly input data or upload files in fasta, csv, or xlsx formats; (2) Select the function or property of the bioactive peptide you want to predict, submit a request, and leave your email address; (3) The results will be returned to the user via email.

[0045] It should be noted that input sequences exceeding 128 amino acids will be randomly truncated. The current version does not yet support verification of the biological validity of the prediction results, and cannot determine whether the input data represents a real peptide; the scientific validity of the prediction results still requires manual evaluation.

[0046] The technical means disclosed in this invention are not limited to those disclosed in the above embodiments, but also include technical solutions composed of any combination of the above technical features. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this invention, and these improvements and modifications are also considered within the scope of protection of this invention.

Claims

1. A method for predicting peptide function based on a general large language model, characterized by: Includes the following steps: S1): Data collection, cleaning and organization, collecting pre-training data and fine-tuning data, and removing sequences containing illegal characters or non-natural amino acids, and using similarity clustering to remove redundancy to ensure the diversity and representativeness of the data; S2): Data preprocessing, using byte-to-byte encoding (BPE) for word segmentation, and using a greedy algorithm to generate vocabularies of different sizes; S3): Model building, based on the BERT architecture, design the PepUniBERT model, treat peptide sequences as natural language, and model the contextual dependencies between amino acids; S4): Pre-training, using the masked language model MLM to model the task and learn a general sequence representation; S5): Optimize the PepUniBERT model architecture by finding the optimal configuration from three dimensions: vocabulary size, model depth, and normalization method, to balance computational cost and performance. S6): Fine-tuning. During fine-tuning, the PepUniBERT model inherits the weights from pre-training, follows the principle of transfer learning, and uses [token] tags to aggregate global information of the sequence for classification / regression. To address overfitting on small samples, amino acid random masks are used for data augmentation, and stratified random sampling is implemented. To address class imbalance, improved FocalLoss and data augmentation strategies are used to improve the model's performance on small and imbalanced datasets. S7): Peptide attribute prediction. The PepUniBERT peptide big language model, which has been pre-trained and fine-tuned, encodes the input peptide sequence and constructs binary, multi-class, or regression task heads based on the special [token] representation of the sequence start position, to predict the functions of various peptides, including blood-brain barrier permeability peptides (BBBP / BBBPV). S8): Model Evaluation: The model is evaluated on independent test sets and external validation sets using multiple metrics including accuracy (ACC), area under the curve (AUC), precision (Precision), sensitivity (Sen), specificity (Spe), F1 score, and Matthews correlation coefficient (MCC).

2. The method for predicting peptide function based on a general large language model according to claim 1, characterized in that: The pre-training data collection involves obtaining large-scale non-redundant protein sequences from the UniRef50 database, screening entries with a length ≤128 amino acids, and constructing a pre-training dataset. The pre-training dataset contains no fewer than 20 million sequences and is divided into a training set and a test set in a 9:1 ratio. The pre-training dataset consists of unlabeled peptides.

3. The method for predicting peptide function based on a general large language model according to claim 1, characterized in that: The fine-tuning data consists of experimentally validated functional peptide sequences collected and organized from the PubChem database. This includes retrieving literature from the PubChem database based on functional keywords, filtering the literature using Tongyi Qianwen AI, extracting sequences, parsing them with Python scripts, and manually verifying them to remove redundancy, non-natural modifications, and D-type amino acids to ensure the data's authenticity and validity. The fine-tuning data comprises no fewer than 20 categories and consists of tagged peptides.

4. The method for predicting peptide function based on a general large language model according to claim 1, characterized in that: The PepUniBERT model consists of an input layer, a multi-layer Transforer encoder, and a fully connected layer. The input layer includes an input embedding layer and a positional encoding layer. The input embedding layer transforms the amino acid sequence into a vector representation; Positional Encoding injects sequence information into each position in the sequence through learnable absolute position embeddings; The Transformer encoder includes a multi-head attention layer, a feedforward network layer, and residual connections and layer normalization (Add & Norm) to extract and fuse sequence features layer by layer. The fully connected layer is used to predict the occluded token during the pre-training phase. During the fine-tuning phase, the output layer is adjusted so that only the special token at the beginning of the sequence is input into the fully connected layer, and the results are generated according to the specific task. The result generation includes outputting 0 or 1 for binary classification tasks, outputting class probabilities for multi-class tasks, and predicting numerical values ​​for regression tasks.

5. The method for predicting peptide function based on a general large language model according to claim 1, characterized in that: The masked language model MLM described in S4) uses an improved dynamic mask, in which 15% of the [token] in each sequence is randomly selected as the prediction target, of which 80% is replaced with the special symbol [MASK], 10% is replaced with a random [token], and the rest remain unchanged.

6. The method for predicting peptide function based on a general large language model according to claim 1, characterized in that: In S5), the optimization of the PepUniBERT model architecture includes the following steps: Q1): Vocabulary optimization: Fix an 8-layer Transformer as the baseline model and test the impact of different vocabulary sizes on model performance in order to find an optimal vocabulary size that can fully cover the semantic units of the peptide sequence, while avoiding sparse word vectors or redundant model parameters due to an excessively large vocabulary. Q2): Depth optimization: After determining the optimal vocabulary, compare and test the performance of models with depths of 8, 12, and 16 layers to find the best balance between computational cost and model performance, and select the optimal number of layers as the standard configuration. Q3): Normalization optimization. Drawing on the experience of the advanced model Llama2.0, we compared the root mean square normalization (RMSNorm) and the traditional layer normalization (LayerNorm). Through experiments, we compared the differences between the two in terms of convergence speed, final accuracy, and generalization ability, and selected the most suitable normalization scheme for the peptide language model.

7. The method for predicting peptide function based on a general large language model according to claim 1, characterized in that: In S6), the method for data augmentation using the amino acid random mask is to randomly replace one residue in each sequence with a placeholder X, generate a new sample each time, and the number of replacements does not exceed the total length of the sequence. The augmentation factor is set to 2, 4, 6, 8 or 10 times. In S6), the improved FocalLoss function is: in, p t It is the model's predicted probability of the true class. a t These are class weight parameters used to balance the proportion of positive and negative samples. γ It is a moderating factor used to reduce the loss contribution of easily classified samples. γ The larger the value, the more attention the model pays to samples that are difficult to classify.

8. The method for predicting peptide function based on a general large language model according to claim 1, characterized in that: Fine-tuning includes at least the blood-brain barrier permeability peptide prediction task. In fine-tuning, a blood-brain barrier permeability dataset (BBBP and / or its high-quality variant BBBPV) containing both blood-brain barrier-penetrating and non-penetrating peptides is used as the training set and / or independent validation set to improve the model's discriminative ability in blood-brain barrier permeability prediction scenarios.

9. An online tool for predicting peptide function based on a general large language model as described in any one of claims 1 to 8, characterized in that: The trained model is integrated into a web platform, supporting input sequences in FASTA, CSV, or XLSX formats; users input peptide sequences and select prediction tasks, and the platform automatically calculates and sends the results back via email.

Images