Method for identifying succinylation sites based on language model driven by entity knowledge

Abstract

The invention belongs to the crossing field of bioinformatics and computational biology, and relates to a method for identifying succinylation sites based on a language model driven by entity knowledge. According to the method, firstly, amino acid composition (AAC), One-Hot coding and residue distance-based features (DR) are fused to construct a multi-dimensional composite feature set, then an entity knowledge enhanced LUKE model and a robustness optimized RoBERTa model are respectively coupled with a bidirectional long-short term memory network (BiLSTM), and context dynamic modeling is realized by extracting deep features of a pre-training model. And a final prediction model is obtained through model fine tuning and integration strategy fusion. Finally, according to the method, t-SNE feature visualization and SHAP interpretability are combined to analyze and verify biological, physical and chemical reasonability of model decision, and accurate recognition of protein succinylation sites is achieved.

Description

Technical Field

[0001] This invention belongs to the interdisciplinary field of bioinformatics and computational biology, and relates to a method for identifying succinylation sites based on an entity knowledge-driven language model. Background Technology

[0002] Lysine succinylation, as a key post-translational modification (PTM), plays a multidimensional regulatory role in life activities. Through precise regulation of core biological pathways such as cell cycle progression and metabolic signal transduction, it is closely related to the pathogenesis of various human diseases, including malignant tumors and neurodegenerative diseases. Especially during viral infections such as SARS-CoV-2, the dynamic changes in succinylation modifications at the host-virus interface can bidirectionally regulate the viral replication cycle and the host immune response, providing a new perspective for elucidating pathogen-host interaction mechanisms. Precise identification of succinylation sites has become an important direction in biomedical research. However, current research still faces many core problems that urgently need to be solved: 1. Traditional mass spectrometry identification techniques have inherent throughput limitations and sequence bias bottlenecks. When processing massive proteomics data, they are difficult to effectively capture the highly heterogeneous characteristics of succinylation modifications. In addition, the detection cost is high and the operation is complicated, which cannot meet the needs of large-scale site identification. 2. Although existing computational prediction methods have made some progress, they still have significant shortcomings: some methods rely on traditional machine learning models such as random forests and SVMs, which have limited feature representation capabilities and can only integrate shallow sequence features, making it difficult to mine deep semantic associations and long-range dependencies in protein sequences. 3. Some models have not effectively addressed the imbalance between positive and negative samples, which makes the model training easily dominated by the majority class samples, resulting in insufficient ability to identify minority class succinylation sites and limited generalization performance. 4. Existing methods generally lack the ability to collaboratively capture local contextual features and global semantic features, and are mostly "black box models" that cannot explain the biochemical mechanisms behind the prediction results, making it difficult to meet the research needs for precision and interpretability.

[0003] Therefore, existing technologies have significant shortcomings in terms of accuracy, robustness, interpretability, and adaptability to large-scale applications for succinylation site identification. There is an urgent need to develop an efficient prediction technology that can integrate multi-dimensional features, solve the problem of sample imbalance, coordinate global and local feature modeling, and has interpretability, so as to meet the urgent needs of biomedicine, virology and other fields for accurate identification of lysine succinylation sites. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention provides a method for identifying succinylation sites based on an entity knowledge-driven language model.

[0005] The technical solution of the present invention is as follows: The data used in this invention are all derived from the internationally recognized protein post-translational modification database (dbPTM) and the universal protein knowledge base (UniProt). Addressing the common problem of positive and negative sample imbalance in protein succinylation site data, this invention employs an optimization strategy combining sequence expansion and intelligent sampling during the sample construction stage. First, positive samples are bidirectionally expanded to 101 positions centered on lysine (K). Insufficiently long samples are handled through mirror completion and padding with the special symbol "X". Then, negative samples are selected from complete sequences, and the NearMiss1 downsampling algorithm is used to balance the sample distribution, thereby improving the model's generalization performance and robustness. Building upon this foundation, this invention proposes an ensemble learning framework called EKROSuc. This method first integrates amino acid composition (AAC), One-Hot encoding, and residue distance-based features (DR) to construct a multidimensional composite feature set. Then, it couples the entity knowledge-enhanced LUKE model and the robustness-optimized RoBERTa model with a bidirectional long short-term memory network (BiLSTM), respectively. Contextual dynamic modeling is achieved by extracting deep features from the pre-trained model. The final prediction model is obtained through model fine-tuning and ensemble strategy integration. Finally, this invention verifies the biophysical and chemical rationality of the model's decisions by combining t-SNE feature visualization and SHAP interpretability analysis, achieving accurate identification of protein succinylation sites.

[0006] The method for identifying succinylation sites based on entity knowledge-driven language models comprises the following steps: Step 1: Data Collection and Preprocessing Dataset construction and preprocessing are fundamental steps in achieving accurate prediction of protein lysine succinylation sites in this invention. The core objective is to address issues such as incomplete sequence information, imbalanced positive and negative samples, and insufficient data reliability in traditional datasets through standardized data collection, sequence optimization, sample selection, and balancing strategies. This provides high-quality, unbiased input data for subsequent model training. The specific implementation is as follows: 1.1 Data Sources and Initial Screening The benchmark data for this invention comes from the internationally recognized authoritative database of protein post-translational modifications, dbPTM (Database of Post-Translational Modifications). This database system integrates experimentally validated protein post-translational modification data from around the world, possessing high reliability and completeness. Through rigorous screening, 2599 high-quality protein sequences were obtained from this database, including 5049 experimentally confirmed lysine succinylation sites and 5526 non-succinylation sites. To obtain complete protein sequence information to support subsequent sequence expansion operations, this invention also extracts the complete sequences of the corresponding proteins from the UniProt database.

[0007] In the initial stage of feature extraction, this invention employs a sliding window method, using lysine (K) as the central site and setting the window size to 2r+1, where the window radius r = 10 amino acid residues, thereby obtaining a feature sequence fragment of length 21 amino acids (sequence structure K±10). To standardize the sequence representation, a standardized expression method is adopted as shown in equation (1): (1) Where K represents the central lysine site, and Representing the first This method represents 21 upstream and downstream amino acid residues. It can clearly show the spatial structure characteristics of the 21-peptide sequence with succinylated lysine as the core, providing a standardized data foundation for subsequent data processing and machine learning modeling.

[0008] 1.2 Positive Sample Sequence Expansion and Filling Processing Given that protein sequences typically contain hundreds to thousands of amino acid residues, the regulatory mechanisms of post-translational modification sites not only depend on neighboring amino acids but may also be subject to synergistic regulation by distant sequence regions. Traditional 21-amino acid sequence windowing methods tend to overlook important distant regulatory information and fail to fully leverage the long-context modeling advantages of large language models. Therefore, this invention designs a systematic sequence expansion scheme that extends the positive sample sequence from 21 positions to 101 positions to fully preserve long-range amino acid interactions around the modification site.

[0009] Specifically, the original protein sequence is first located using the UniProt database. The collected original positive sample sequences are then matched with the complete protein sequence, and 40-position extensions are performed before and after the central lysine site. For samples where the modification sites are located at both ends of the sequence and equal-length bidirectional extension cannot be achieved, the missing residues are filled using a mirror method, as shown in Equation (2). (2) After mirroring and padding, if the sequence fragment length still does not reach the preset length of 101 bits, this invention uses the special symbol "X" as specified by the International Union of Biochemistry and Biotechnology (IUPAC) for length completion to maintain data consistency. It should be noted that "X" not only represents unknown types beyond the 20 standard amino acids, but is also commonly used in authoritative bioinformatics databases such as UniProt to identify undetermined amino acid sites, conforming to industry standards. Throughout the expansion and padding process, the total number of positive samples remains constant at 5049 to ensure data stability.

[0010] 1.3 Negative Sample Construction and Screening To ensure consistency in sequence length and center site type between positive and negative samples and to meet the homology requirements for model training, the present invention constructs negative samples according to the following process: First, from the complete protein sequences obtained from the UniProt database, sequence fragments that have been used for positive sample expansion and have not been padded are removed to avoid overlap between positive and negative samples; then, using lysine (K) as the reference point, sequence fragments of 101 positions before and after it are taken as the negative sample candidate set.

[0011] To ensure the authenticity and validity of negative samples, this invention optimizes the process through two core screening criteria: first, segments with a sequence similarity higher than 80% to positive samples are removed from the candidate set to avoid data redundancy and label confusion; second, segments containing known succinylation site motifs are removed to ensure that negative samples are genuine non-succinylation site sequences. After rigorous screening, 40,000 high-quality negative samples that meet the requirements are obtained, laying the foundation for subsequent sample balancing.

[0012] 1.4 Sample Balancing Process (NearMiss-1 Downsampling Strategy) The initial dataset had a positive-to-negative sample ratio of 5049:40000. This severely imbalanced sample distribution caused the model training process to be dominated by the majority class samples, shifting the decision boundary towards the minority class. This reduced the model's ability to identify minority class samples and made it prone to overfitting. To address this issue, this invention implements a downsampling strategy for negative samples. After systematically evaluating various algorithms, the NearMiss-1 algorithm was selected as the core strategy. This algorithm possesses boundary sensitivity, structure preservation, information integrity, neighborhood awareness, and high-dimensional adaptability, perfectly meeting the needs of biological sequence downsampling.

[0013] Suppose the dataset contains a set of majority class samples. and minority class sample set Each sample This represents a feature vector of a protein sequence (such as a vector concatenated from One-Hot, AAC, and DR sequences). NearMiss-1 aims to extract features from... Select a subset from This makes each selected majority class sample In relation to the recent k The average distance between minority class samples is the smallest: (3) in: express Recent k A set of indexes for minority class neighbors; This is the distance metric function, which can be Euclidean distance, cosine distance, or Mahalanobis distance. Ultimately, from... Select the average distance The smallest front Each sample constitutes a subset of the downsampling set. .

[0014] Parameter setting: k is a positive integer (e.g., 3 to 10). The number of samples in the minority class can be kept consistent with the number of samples in the minority class to achieve a 1:1 balance; the cosine distance can be preferred as the dist distance metric.

[0015] Step 2: Multi-dimensional feature extraction and fusion The core objective of this stage is to extract multi-dimensional information from protein sequences, including global frequency, precise site, and residue position, by screening complementary feature extraction methods. This information is then organically fused to form a composite feature set, providing comprehensive and discriminative input data for subsequent models. This addresses the limitations of single feature expression and the omission of key information. The specific implementation is as follows: 2.1 Feature Extraction Strategy To obtain the most discriminative sequence features, this invention compared and analyzed five feature extraction methods: Amino Acid Composition (AAC), One-Hot Encoding, Distance-based Residue (DR), PseAAC (pseudo amino acid composition), and Density Peaks Clustering (DPC). After systematic experimental verification, three feature extraction methods were ultimately selected: AAC, One-Hot, and DR. The specific implementation process involves performing these three feature extraction operations in parallel on the target succinylated sequence. The following is a systematic explanation of the principles and implementation of the three feature extraction methods: 1. Amino Acid Composition (AAC) Feature Extraction: AAC features are used to measure the relative abundance of each amino acid in a sequence, extracting a protein sequence of length L. Mapped to a 21-dimensional vector: ,in , No. j Frequency of amino acids in the sequence , .

[0016] 2. One-Hot Encoding Feature Extraction: One-Hot encoding uses a binary vector to represent the category information of each residue. For 20 standard amino acids and the filler letter "X", a mapping is defined. The mapping rules remain unchanged. When a certain position is "X", its corresponding 21-dimensional vector has only the 21st component as 1, and the rest as 0. The entire sequence is still represented as an L×21 sparse matrix.

[0017] 3. Residue Distance (DR) Based Feature Extraction: The DR algorithm aims to reflect the positional information between amino acids in a protein sequence, focusing particularly on the relative positions and distribution patterns of residues within the sequence. Its core idea is to utilize the positional distribution of each amino acid in the sequence to calculate the average relative position of each amino acid, thereby obtaining a statistical feature representation of the relative positional information of amino acids. For a given protein sequence... , Suppose a certain amino acid (For example, "A", "C", ..., "Y", "X") appear 100 times in this sequence. The locations where it appears are denoted as: For each amino acid, calculate two statistics: (1) Average position (AP): , ; (2) Standardized AP (SAP) representation of location information: To avoid the influence of sequence length differences on features, the location is generally standardized to the range [0,1]. , At this point, the DR feature vector of the entire sequence is: If a certain amino acid is not present in the sequence, the corresponding dimension can be recorded as 0 or other predefined values.

[0018] 2.2 Feature Fusion Strategy To fully leverage the complementary advantages of each individual feature, this invention organically integrates the feature vectors generated by various methods through feature fusion technology, forming a composite feature set with multidimensional representation capabilities. This fusion strategy achieves deep integration of global amino acid frequency information, local residue category information, and residue relative position information, effectively avoiding the information limitations of a single feature. This enables the fused feature set to possess more comprehensive sequence representation capabilities and stronger discriminative power, providing high-quality input for subsequent pre-trained models and feature modeling of BiLSTM networks.

[0019] Step 3: Model Building and Optimization This invention employs a hybrid architecture design of "pre-trained large language model + BiLSTM + ensemble learning". It leverages the deep semantic representation capabilities of the pre-trained large language model to capture long-range dependencies in protein sequences, while enhancing local temporal feature modeling through the BiLSTM network. Finally, it combines the advantages of both models through an ensemble strategy, addressing the problems of single models being unable to simultaneously capture global and local features and lacking generalization ability. The specific implementation is as follows: 3.1 Pre-trained large language model This invention employs two improved BERT language models: the LUKE model and the RoBERTa model for protein succinylation sequence modeling.

[0020] 3.1.1 LUKE Model: Deep Contextual Language Modeling Integrating Entity Perception LUKE (Language Understanding with Knowledge-based Embeddings) is a deep language model that co-encodes textual words with knowledge base entities. Through dual-channel input representation (word sequences and entity sequences) and an entity-aware self-attention mechanism, it simultaneously models word-word, word-entity, entity-word, and entity-entity interactions within the same Transformer framework. Combined with a joint pre-training strategy of Masked Language Modeling (MLM) and Masked Entity Modeling (MEM), it achieves deep fusion of contextual semantic information and structured knowledge, significantly improving the understanding of entity semantics and their associated structures. This invention applies LUKE to protein sequence modeling, enhancing the model's entity recognition ability by incorporating information related to protein functional domains and structural domains. For example, specific amino acid residues or structural domains in a protein sequence can be considered "entities." By integrating knowledge from these specific domains during pre-training, LUKE can more effectively capture the complex details of protein function and the interdependencies between different regions.

[0021] 3.1.2 RoBERTa Model: A Robust Optimized BERT Pre-training Method RoBERTa (A Robustly Optimized BERT Pretraining Approach) is a pre-trained model that systematically optimizes the BERT pre-training process while retaining the basic structure of the bidirectional Transformer Encoder. By removing the Next Sentence Prediction (NSP) task, employing a dynamic masking strategy, significantly expanding the pre-training corpus size, and optimizing training hyperparameters and training time, the model can learn contextual semantic representations more fully under larger data conditions, thereby improving its robustness and generalization ability in various understanding tasks. This invention applies this architecture to protein sequence modeling, treating amino acid sequences as language sequences and learning the contextual representation of residues based on masking prediction. Leveraging the Transformer's ability to model long-distance dependencies, it effectively captures the interactions between residues and functional region features, thereby improving the modeling accuracy and applicability for protein functional annotation, classification, and structure-related prediction tasks.

[0022] 3.2 Deep Learning-Based Natural Language Processing Techniques In the field of biological sequence analysis, protein sequences and natural language text share significant structural isomorphism: a linear sequence composed of 20 standard amino acids can be viewed as a symbolic flow similar to "words," and its specific permutation and combination patterns determine the higher-order folding structure and biological function of proteins. Therefore, deep learning-driven natural language processing (NLP) technology has become an important means of revealing the potential patterns of protein post-translational modifications (PTM). Although pre-trained models based on the Transformer architecture excel in capturing long-range dependencies and global semantic structure, their inductive bias in attention mechanisms may be less sensitive than recurrent neural networks when resolving the fine local microenvironmental features around modification sites. Transformers tend to focus on global associations, while the chemical properties of modification sites are often limited by the strict order and biochemical properties of their immediate upstream and downstream residues. To capture global semantics without losing these crucial local temporal features, the EKROSuc framework proposed in this invention employs a "global-local" complementary enhancement strategy. Based on constructing a global semantic skeleton using a pre-trained model, this invention specifically introduces a bidirectional long short-term memory network (BiLSTM) as an independent local feature extraction module. This module aims to compensate for the shortcomings of attention mechanisms in local sequence dynamic modeling by explicitly capturing sequence motif patterns on both sides of the modification site, thereby enhancing the ability to perceive the microenvironment.

[0023] 3.2.1 Bidirectional Long Short-Term Memory (BiLSTM) This invention employs a bidirectional long short-term memory (BiLSTM) network to construct local dependency layers, aiming to comprehensively capture the dynamic temporal features surrounding modification sites in protein sequences. BiLSTM is an advanced extension architecture based on long short-term memory (LSTM) networks. When processing protein sequences, traditional recurrent neural networks (RNNs) rely on historical hidden states to update the current state, which easily leads to gradient vanishing or exploding problems when processing long sequences, making it difficult to effectively capture long-term dependencies. LSTM networks, by introducing forget gates, input gates, output gates, and independent memory units (cell states), can maintain effective gradient propagation and retain key information over longer time spans. However, the standard unidirectional LSTM structure limits its processing to one direction (N-terminus → C-terminus or C-terminus → N-terminus), failing to simultaneously utilize the complete amino acid environment upstream and downstream of modification sites. BiLSTM overcomes this limitation by coupling two independent LSTM layers, enabling simultaneous utilization of amino acid environment information upstream and downstream of modification sites. Its specific dynamic modeling process is as follows: First, the forward LSTM layer scans the sequence from the N-terminus to the C-terminus (i.e., from t=1 to T). This layer is responsible for capturing the amino acid sequence features upstream of the modification site, utilizing the input at the current time step. Compared to the previous time step, the forward hidden state The update is performed using the following formula: (4) Meanwhile, the backward LSTM layer processes the sequence in the opposite direction (i.e., from the C-end to the N-end). This layer is primarily responsible for capturing the biochemical environmental features downstream of the site, combined with the current input. Backward hidden state with future time Its update formula is: (5) To obtain the complete spatiotemporal context semantics of this location, the model concatenates the forward and backward hidden states at the same time step t. This operation integrates upstream and downstream information into a unified dense vector. This not only preserves directional dependence but also enhances the expressive power of features: (6) Through the above steps, BiLSTM effectively simulates the recognition process of the stereochemical environment on both sides of the modification site when an enzyme binds to a substrate. In this invention, the BiLSTM module further dynamically and temporally enhances the static semantic representation output by the pre-trained language model (LUKE / RoBERTa), enabling the generated embedding vector to not only contain global contextual relevance but also possess a keen response to local structural changes. This mechanism significantly improves the model's recognition accuracy in regions with weak modification signals or high background noise, thereby optimizing the overall prediction performance.

[0024] 3.2.2LUKE+BiLSTM The architecture uses the output of the 12th layer of LUKE (the last hidden layer of LUKE-base) as the input to BiLSTM. The output of the 12th layer usually contains rich contextual information, which can capture potential functional regions in protein sequences and their interrelationships. Passing this output to the BiLSTM network enables the model to not only model based on contextual information, but also to use the bidirectional sequence modeling capability of BiLSTM to capture long-range dependencies and local structural features in protein sequences.

[0025] First, the protein sequences are preprocessed. The sequences are encoded into a format suitable for model input, and each amino acid is converted into a token using a word segmentation method. For each protein sequence, the model not only obtains the amino acid representation but also annotates entity information such as domains within the sequence and adds it to the input. The preprocessed protein sequences are then input as token sequences into the LUKE model. The LUKE model's input includes not only word embeddings but also entity embeddings. This information is processed in the Encoder using a self-attention mechanism to obtain long-distance dependencies in the context. During training, after each batch of training, the LUKE model extracts the output of its 12th layer as a feature representation of the sequence residues. This is then used as the input to the BiLSTM module. This model has bidirectional information flow capabilities, enabling parallel modeling in both forward and backward directions, effectively enhancing the representation of protein sequences between local structure and overall dynamics. The output of the BiLSTM is further mapped to a binary classification space through a fully connected layer and outputs a predicted probability via an activation function to determine whether each site is a succinylation site. In this process, LUKE and BiLSTM are jointly optimized. While maintaining LUKE's fine-tuning capabilities, the overall model introduces a binary cross-entropy loss function (BCELoss) suitable for binary classification tasks, and uses the AdamW optimizer for gradient updates. The learning rate is gradually decayed using the torch.optim.lr_scheduler.LinearLR scheduler to stabilize the training process and prevent overfitting.

[0026] 3.2.3 RoBERTa+BiLSTM The architecture also uses the output of the 12th layer of RoBERTa (the last hidden layer of RoBERTa-base) as the input of BiLSTM; BiLSTM effectively complements the semantic features extracted by RoBERTa, enhancing the ability to identify the local structure of succinylation sites and integrate global information.

[0027] First, the original protein sequence is segmented at the amino acid level and then input into the RoBERTa model. RoBERTa employs a multi-layer Transformer Encoder, generating a context-aware representation for each residue in the amino acid sequence through a self-attention mechanism. Since the output of RoBERTa's twelfth layer fully captures the long-range dependencies and latent semantic structures between residues in the sequence, this layer's output is used as a deep feature representation of the sequence. Subsequently, consistent with LUKE+BiLSTM, the entire model uses techniques such as AdamW, BCELoss, and learning rate scheduling to optimize model parameters, ultimately improving the model's generalization performance.

[0028] 3.3 Model Integration To further improve the robustness and generalization performance of the model in identifying succinate sites, this invention introduces a model ensemble strategy. The LUKE+BiLSTM and RoBERTa+BiLSTM models are integrated after independent training to obtain EKROSuc, which is then evaluated on an independent test set. The core idea of ​​the ensemble method is to leverage the complementarity of different models in feature learning, improving overall predictive ability while maintaining their respective advantages. Specifically, two models are used to predict the same protein sequence input, obtaining corresponding classification outputs. While maintaining the model structure, a probability-weighted average is used to calculate the results. The prediction results of the LUKE+BiLSTM and RoBERTa+BiLSTM models at the output layer are integrated to finally calculate the comprehensive prediction result. For a single sample... Integration results The definition is as follows: (7) in, and These represent the LUKE+BiLSTM and RoBERTa+BiLSTM models for the sample, respectively. The predicted probability, , is a weighting parameter, which is set in this invention. The value is 0.5. This ensemble strategy not only effectively alleviates the limitations of a single model in certain feature dimensions, but also significantly improves the stability and robustness of the model on independent test sets, providing a more practical solution for the task of identifying post-translational succinylation modification sites in protein sequences.

[0029] Step 4: Model Evaluation After the model training is completed, the fourth step aims to systematically test its performance and demonstrate its value in practical applications. This stage first quantitatively evaluates the model's generalization ability and prediction accuracy through rigorous cross-validation and multi-dimensional evaluation metrics.

[0030] 4.1 Evaluation Indicator Design To comprehensively measure prediction performance, this invention selects six core evaluation metrics, including accuracy (ACC), precision, recall, F1 score (F1), Matthews correlation coefficient (MCC), and area under the ROC curve (AUC), covering multiple dimensions such as positive and negative sample identification ability, overall prediction accuracy, class balance adaptability, and probabilistic discrimination efficiency.

[0031] 4.2 Evaluation Method Design To ensure the reliability and persuasiveness of the evaluation results, this invention employs a dual validation strategy of 10-fold cross-validation (CV) and independent test set validation, and introduces existing mainstream prediction methods as benchmarks. 10-fold cross-validation comprehensively evaluates the model's stability and generalization ability under different data partitioning scenarios, avoiding evaluation bias caused by a single data partition. The comparison targets are mainstream prediction methods in the field (IFS-LightGBM, SuccSPred, pSuc-FFSEA, etc.), all based on the original dataset or the dataset constructed in this invention, and trained and validated under identical conditions to ensure fairness and objectivity in the comparison. The reconstructed balanced dataset is randomly split into a training set and an independent test set in an 8:2 ratio; LUKE+BiLSTM, RoBERTa+BiLSTM, and the ensemble model EKROSuc are all fine-tuned and optimized on the training set, and then their performance is evaluated on the independent test set to verify their generalization ability to unknown data. The newly added protein modification site prediction model GPSuc was used as a comparison object. It was directly transferred to the dataset constructed in this invention for testing, and its adaptability under the new data distribution was evaluated, further highlighting the technical superiority of the model in this invention.

[0032] Step 5: Model Interpretability Analysis While deep learning models have achieved significant breakthroughs in predictive performance, they are often considered "black box" systems, lacking transparent internal decision-making logic. To bridge the gap between high performance and interpretability, and to verify whether the EKROSuc framework truly captures the biochemical essence related to succinylation modification, this invention employs two techniques: t-SNE visualization and SHAP attribution analysis. The former assesses the discriminative power of the feature space from a macroscopic perspective, while the latter analyzes the model's focus on specific amino acid sequences from a microscopic perspective. These two techniques complement each other, comprehensively revealing the model's intrinsic operating mechanism.

[0033] 5.1 Feature Space Visualization Based on t-SNE To verify the effectiveness of pre-trained large language models in transforming discrete amino acid sequences into high-dimensional dense semantic vectors, this invention utilizes the t-SNE (t-Distributed Stochastic Neighbor Embedding) algorithm to perform dimensionality reduction projection on the feature space, intuitively comparing the essential differences between traditional coding and semantic coding in terms of sample separability.

[0034] 5.2 Analysis of Decision-Making Mechanism Based on SHAP Value Based on the SHapley Additive exPlanations (SHAP) method, 10 background datasets of different sizes (100-1000 samples, step size 100) were constructed. The mean absolute SHAP value of each amino acid residue and residue combination was calculated, the residue combinations that play a key role in the prediction results were identified, and the biochemical mechanism behind the model decision was revealed.

[0035] The beneficial effects of this invention are: The protein lysine succinylation site prediction system EKROSuc proposed in this invention solves the core problems of existing technologies, such as insufficient data quality, one-sided feature representation, limited prediction accuracy, and uninterpretable models, through a complete process design including dataset optimization, multi-dimensional feature fusion, innovative model architecture, and interpretability analysis. It achieves the following significant beneficial effects: 1. Data Level: Achieving high-quality, unbiased dataset construction lays the foundation for accurate predictions. This invention extends traditional 21-position sequences to 101 positions through a sequence extension strategy. Combined with mirror completion and "X" padding techniques, it fully preserves long-range amino acid interaction information at modification sites, compensating for the lack of distant regulatory information. Simultaneously, it employs the NearMiss-1 downsampling algorithm to efficiently address the severe imbalance between positive and negative samples, constructing a balanced dataset with a 1:1 ratio of positive to negative samples. This dataset ensures both the representativeness and completeness of the samples while avoiding class bias during model training, providing solid data support for subsequent high-performance models. Compared to traditional datasets, it effectively enhances the model's ability to learn complex sequence patterns.

[0036] 2. Feature level: Multi-dimensional feature fusion enables comprehensive sequence representation and enhances discriminative power. This invention, through systematic verification and screening, identified three highly complementary feature extraction methods: AAC, One-Hot, and DR. By splicing features, it achieves deep integration of three types of information: global amino acid frequency, local residue category, and relative residue position. The fused composite feature set possesses both global statistical properties and fine-grained site information and spatial distribution characteristics, effectively avoiding the information limitations of single features. Experiments demonstrate that this feature fusion strategy outperforms single features and other combinations in all six core evaluation metrics, significantly enhancing the characterization ability of protein sequences and providing ample feature support for the model to accurately identify modification sites.

[0037] 3. Predictive performance: The innovative model architecture achieves breakthroughs in both accuracy and robustness, outperforming existing methods. This invention employs a hybrid architecture of "large language model + BiLSTM + ensemble learning," leveraging LUKE's entity knowledge enhancement capabilities and RoBERTa's robust optimization characteristics to accurately capture long-range dependencies and deep semantics of protein sequences. It also strengthens local temporal feature modeling through a BiLSTM network, achieving "global-local" feature complementarity. Finally, an ensemble strategy combines the advantages of both models to further improve prediction stability. Through 10-fold cross-validation and independent test set validation, the model achieved leading levels in all six core metrics, significantly outperforming existing mainstream methods such as IFS-LightGBM, SuccSPred, and GPSuc, and demonstrating strong generalization ability and excellent adaptability to new data distributions.

[0038] 4. Interpretability level: Breaking through the limitations of the "black box," verifying biological rationale, and improving the credibility of the model. This invention innovatively introduces t-SNE feature visualization and SHAP attribution analysis to construct a two-tiered interpretability system of "macro-micro". t-SNE visualization intuitively verifies the strong discriminative ability of the pre-trained model in extracting features, while SHAP analysis accurately locates residue combinations such as RA, AY, and IP that play a key role in prediction. Furthermore, through biochemical mechanism verification, it is confirmed that these residue combinations are highly consistent with the core mechanisms of succinylation modification, such as charge interactions and hydrogen bond networks. This module not only breaks down the "black box" barrier of deep learning models, providing reliable scientific evidence for prediction results, but also offers new clues for the study of the molecular mechanisms of succinylation modification, possessing both practical and academic value. Attached Figure Description

[0039] Figure 1 This is the overall framework diagram of the present invention.

[0040] Figure 2 This is a radar chart comparing the performance of five feature extraction methods.

[0041] Figure 3 This is a line graph showing the model performance under different sequence lengths.

[0042] Figure 4a This is the t-SNE visualization result of the feature extraction using LUKE in this invention.

[0043] Figure 4b This is the t-SNE visualization result of the protein sequence features encoded by One-Hot in this invention.

[0044] Figure 5 This is an interpretation analysis of the SHAP values ​​of the LUKE model at different sample sizes (100–200).

[0045] Figure 6 This is a statistical heatmap of residue pairs that appear multiple times in the LUKE model SHAP analysis. Detailed Implementation

[0046] The specific implementation of the present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0047] This invention proposes a method for identifying succinylation sites based on an entity knowledge-driven language model, the overall architecture of which is as follows: Figure 1As shown, this method mainly includes five stages: dataset collection and preprocessing, multi-dimensional feature extraction and fusion, model construction and optimization, model evaluation, and interpretability analysis. First, in the data collection and preprocessing stage, this invention uses the internationally authoritative databases dbPTM and UniProt as data sources, selecting experimentally verified positive samples of succinylated sites and initial negative samples of non-succinylated sites. Then, with lysine (K) as the center, the positive samples are bidirectionally expanded to 101 positions. Insufficient length samples are processed by mirror completion and filling with the special symbol "X". Negative samples are then selected from the complete sequence, and the NearMiss1 downsampling algorithm is used to balance the sample distribution, forming a balanced dataset that provides a reliable data foundation for subsequent modeling. Second, in the multi-dimensional feature extraction and fusion stage, this invention performs AAC, One-Hot, and DR feature extraction operations in parallel on the target succinylated sequence for the preprocessed 101-position sequence. Subsequently, feature fusion technology is used to organically integrate the feature vectors generated by each method, forming a composite feature set with multi-dimensional representation capabilities, laying the data foundation for the subsequent construction of machine learning models. In the model building and optimization phase, this invention selects two pre-trained large language models, LUKE and RoBERTa, and fine-tunes them using the AdamW optimizer, BCELoss loss function, and LinearLR learning rate scheduler to adapt to the succinylated site binary classification task. The output of the 12th layer of the fine-tuned model is used as the input of a BiLSTM network. Local temporal features are captured through bidirectional LSTM layers (forward N-end → C-end, backward C-end → N-end). After concatenating the hidden states, the single-model prediction probability is output through a fully connected layer and a sigmoid activation function. Finally, the two sub-models are integrated using a probability-weighted average method with a weight of α=0.5 to obtain the final EKROSuc prediction model. In the performance evaluation phase, this invention adopts a dual strategy of 10-fold cross-validation and independent test set validation. First, the balanced dataset is divided into 10 folds for alternating training and validation to evaluate model stability. Then, the training set and independent test set are split in an 8:2 ratio to verify the model's generalization ability. Subsequently, the performance differences between the model of this invention and existing methods such as IFS-LightGBM, SuccSPred, and GPSuc are comprehensively compared using six core indicators: sensitivity (Sn), specificity (Sp), accuracy (Acc), MCC, F1-score, and AUC. In the interpretability analysis stage, this invention analyzes the model's decision-making logic from both macro and micro dimensions. At the macro level, t-SNE dimensionality reduction projection (30 perplexity, 1000 iterations) is performed on the high-dimensional features extracted by the pre-trained model and the traditional One-Hot encoded features, and the positive and negative sample clustering effects of the two types of features are visualized and compared. At the micro level, 10 background datasets of different sizes are constructed, and the average absolute SHAP value of each amino acid residue and combination is calculated using the SHAP method to identify key residue combinations and verify their biochemical rationality.

[0048] In summary, this invention constructs a structurally complete, predictively accurate, robust, and multi-level interpretable framework for identifying protein succinylation sites through multi-dimensional feature fusion, model construction, and optimization. See the following implementation examples for details: Example 1: Dataset Collection and Standardization Preprocessing This embodiment uses the dbPTM database as the baseline data source. After screening, a high-quality protein sequence set for training is obtained, which includes experimentally validated succinylated lysine sites and non-succinylated sites. To form a uniform sample expression format, this embodiment uses the sliding window method, with lysine K as the central site, setting the window size to 2r+1 and r to 10, thereby obtaining a fragment sequence of length 21 (K±10). This fragment is then standardized according to uniform rules to provide a consistent data input format for subsequent feature encoding and model training.

[0049] Example 2: Sample Structure Reconstruction and Sequence Length Extension To address the issue that a 21-peptide fragment alone is insufficient to fully express long-range contextual information, this embodiment implements bidirectional extension of positive samples during the data optimization phase: First, complete protein sequence information is collected, and positive samples are extended upstream and downstream by 40 positions centered on the K site, unifying the sample sequences to 101 positions. For sequences that are too short to be directly extended, this embodiment introduces mirror completion and X-filling to achieve length regularization. Subsequently, this embodiment selects negative samples centered on the K site from the complete sequences and constructs a balanced training set using a subsequent balanced sampling strategy. The aforementioned progressive extension from 21 positions to 101 positions is also used to verify the relationship between sequence length and performance, such as... Figure 3 As shown, as the sequence length increases, multiple evaluation indicators show a stable upward trend. Considering both computational cost and application requirements, 101 bits was ultimately determined to be the optimal length, and a dataset containing 10,098 balanced samples (5,049 positive and 5,049 negative) was formed accordingly.

[0050] Example 3: Construction of Multi-Dimensional Feature Encoding and Fusion Representation This embodiment employs a multi-dimensional encoding strategy at the feature level, generating vector representations from different perspectives for the same 101-bit sequence, and comparing various single-feature encoding methods with fusion methods. Specifically, such as... Figure 2As shown, individual features such as AAC, One-Hot, DR, PseAAC, and DPC exhibit varying degrees of performance loss compared to the AAC-OneHot-DR three-feature concatenation baseline. Furthermore, the fused features outperform individual features across all six metrics, demonstrating the complementarity between global frequency information, precise site information, and residue relative position statistics. Based on this comparison, this embodiment selects AAC, One-Hot, and DR features for concatenation and fusion to construct a multi-view input representation for subsequent model learning.

[0051] Example 4: Class Imbalance Handling and NearMiss-1 Downsampling Implementation Since the negative sample size is much larger than the positive sample size during the training phase, it can easily lead to classifier decision bias and a decrease in minority class recognition ability. In this embodiment, negative training samples are downsampled, and NearMiss-1 is selected as the core strategy among various downsampling algorithms. NearMiss-1 uses the "average distance between the majority class sample and its k nearest minority class samples" as the sorting criterion, and selects a subset of samples with the smallest average distance from the majority class to better preserve the discriminative information near the class boundary and take into account the structural preservation of the high-dimensional feature space. The pseudocode flow of the algorithm is shown in Table 1.

[0052] Table 1 NearMiss-1 Algorithm Flow

[0053] Table 2 Performance Comparison of Different Downsampling Methods

[0054] As shown in Table 2, NearMiss-1, used in this invention, exhibits the most balanced and optimal performance across metrics such as Sn=0.9219, Sp=0.9114, Acc=0.9170, MCC=0.8350, F1-score=0.9175, and AUC=0.9654. Compared to ALLKNN and ENN, NearMiss-1 shows significant improvements in comprehensive metrics such as Sp, MCC, and AUC, indicating that it can effectively reduce false positives and improve overall discrimination quality while maintaining minority class recognition capabilities. Compared to random downsampling, NearMiss-1 shows particularly significant differences in key metrics such as Sp and MCC, indicating that random strategies are prone to disrupting data structures and causing a loss of discriminative information. In contrast, NearMiss-1, by prioritizing samples closer to the minority class boundary, is more conducive to preserving inter-class boundary information, thereby improving the model's generalization performance. Therefore, this invention determines NearMiss-1 as the final balanced sampling scheme.

[0055] Example 5: Selection of Pre-trained Large Language Models and Generation of Semantic Representations This embodiment introduces two types of pre-trained large language models as feature extractors: LUKE based on entity knowledge enhancement and RoBERTa after robustness optimization. Performance comparison is used to determine the base version as the main body of the subsequent structure. Table 3 shows the comparison results of various large language models on this task. RoBERTa-base performs outstandingly in comprehensive metrics such as MCC=0.8566 and AUC=0.9702, while LUKE-base shows balanced performance in metrics such as Sn=0.9219, Sp=0.9114, Acc=0.9170, and AUC=0.9654. Both have good overall performance. Further comparison shows that although LUKE-large has a slightly improved Sp=0.9275, its Sn and MCC metrics are lower than LUKE-base; RoBERTa-large also performs worse than RoBERTa-base in Sn, MCC, and AUC, indicating that under the data scale and task conditions of this invention, a larger parameter scale does not necessarily bring comprehensive benefits and may even lead to performance degradation. Based on the above comparison results, the present invention preferably uses LUKE-base and RoBERTa-base as two semantic coding backbones to balance performance and stability, and then couples them with BiLSTM to achieve collaborative optimization.

[0056] Table 3 Performance Comparison of Different Large Language Models

[0057] In the LUKE branch, this embodiment utilizes its "word-entity-entity inter-entity" joint modeling mechanism to incorporate entity information into self-attention computation, thereby achieving a fusion representation of contextual semantics and entity knowledge. This mechanism introduces attention channels such as word-entity, entity-word, and entity-entity, allowing the model to move beyond traditional word-word relationship modeling. In the RoBERTa branch, this embodiment leverages its larger corpus, larger batch size, and longer training time to enhance the modeling effect on sequence context dependencies, resulting in more robust representation capabilities.

[0058] Example 6: BiLSTM Coupling and End-to-End Co-optimization Training In this embodiment, LUKE and RoBERTa are used as two parallel semantic encoding branches, and a BiLSTM is connected after each branch to enhance the learning ability of local context and sequence dynamic patterns. In specific implementation, this embodiment extracts hidden features of a specified layer from the pre-trained model (using the features of the twelfth layer as the input of BiLSTM), and updates the BiLSTM parameters while fine-tuning the parameters of the pre-trained model, so as to achieve synergistic optimization of semantic representation and sequence context modeling.

[0059] Table 4 Results of 10-fold cross-validation using different methods

[0060] To verify the effectiveness of the "pre-trained language model + BiLSTM" structure, this embodiment performs 10-fold cross-validation on a constructed balanced dataset, and the results are shown in Table 4. Compared with existing methods (IFS-LightGBM, SuccSPred, and pSuc-FFSEA), the method of this invention significantly improves MCC and AUC under 10-fold cross-validation conditions, demonstrating stronger comprehensive discriminative ability and better stability. Furthermore, it can be seen that after introducing BiLSTM into the two backbone models, the MCC of LUKE+BiLSTM increases from 0.8350 to 0.8511, and the AUC increases from 0.9654 to 0.9763; the AUC of RoBERTa+BiLSTM increases to 0.9763, and Sn increases to 0.9355. This indicates that BiLSTM can further enhance the modeling of dynamic patterns in local contexts, improve the quality of discriminative boundaries based on pre-trained semantic representations, and thus enhance the generalization ability and robustness of the model.

[0061] Example 7: Weighted ensemble output of dual models and independent test verification After independently training LUKE+BiLSTM and RoBERTa+BiLSTM, this embodiment further employs probability-weighted averaging to achieve ensemble integration, thereby improving overall stability and generalization performance. For sample x, the ensemble output is defined as the weighted sum of the predicted probabilities of the two models, with the weight parameter α set to 0.5, thus achieving complementary fusion while maintaining the structural integrity.

[0062] Table 5 Independent test results of different methods on the new dataset

[0063] To verify the practical application effect of the ensemble model, this embodiment divides the dataset into a training set and an independent test set in an 8:2 ratio, and compares it with existing methods on the independent test set. As shown in Table 5, EKROSuc achieves the best results in six metrics: Sn, Sp, Acc, MCC, F1, and AUC, demonstrating that the ensemble strategy can effectively integrate the advantages of the two branches and achieve a comprehensive performance improvement.

[0064] Example 8: Model Interpretability Analysis and Key Residue Combination Mining To enhance model transparency and verify whether it captures discriminative information related to succinylation, this embodiment combines t-SNE visualization and SHAP attribution analysis for interpretation. For example... Figure 4a , Figure 4bAs shown, after dimensionality reduction, the high-dimensional semantic features extracted based on the pre-trained large language model (taking LUKE as an example) exhibit clearer clustering and inter-class boundaries in the two-dimensional space for positive and negative samples; while when using traditional One-Hot encoding, the sample distribution overlaps more significantly and the class boundaries are relatively blurred, indicating that the semantic representation has stronger discriminativeness and separability.

[0065] In terms of SHAP attribution analysis, this invention calculates the average absolute SHAP value of the Top-30 important residues and residue pairs under different background sample sizes (100 to 1000, step size 100). Figure 5 It can be seen that important features fluctuate to some extent under different background scales, but multiple residue pairs appear repeatedly in multiple sets of analyses, indicating that the model's focus mechanism is stable.

[0066] Furthermore, this invention performs frequency statistics on multiple groups of Top-30 residue pairs and generates heatmaps, such as... Figure 6 As shown, the RA, AY, and IP pairs appeared 6 times in each of the 10 analyses, RI, RL, and IL appeared 5 times, and NY, LG, RT, and RY residue pairs appeared 4 times. This indicates that these residue combinations are highly important in the "attention mechanism" of the LUKE model and are likely related to the local sequence context of succinylation modification. From a biological perspective, these high-frequency residue combinations may constitute structural fragments in protein sequences that are conducive to enzyme recognition or modification. For example, arginine (R) and alanine (A) are common amino acid residues that frequently participate in protein folding and functional regulation. Arginine has a strong positive charge, enabling it to interact with other negatively charged residues to form stable salt bridges, while alanine is a small, nonpolar amino acid that often appears in structurally compact locations. The above results demonstrate that this invention not only achieves high predictive performance but also outputs sequence features that contribute significantly to discrimination, providing candidate key sequence fragments for subsequent biological interpretation and experimental verification.

Claims

1. A method for identifying succinylation sites based on an entity knowledge-driven language model, characterized in that, The steps are as follows: Step 1: Data Collection and Preprocessing Using the dbPTM and UniProt databases as data sources, experimentally validated positive samples of succinylated sites and initial negative samples of non-succinylated sites were obtained. The sample sequences were bidirectionally extended to 101 positions centered on lysine K, and samples with insufficient length were mirrored and padded with the special symbol "X". Negative samples were constructed and screened from complete protein sequences. The NearMiss-1 downsampling algorithm was used to balance the distribution of positive and negative samples to form a balanced dataset. Step 2: Multi-dimensional feature extraction and fusion AAC features, One-Hot features, and DR features are extracted in parallel from the preprocessed 101-bit sequence, and the features are fused to form a composite feature set. Step 3: Model Building and Optimization The LUKE semantic encoding branch and the RoBERTa semantic encoding branch are constructed separately to fine-tune the pre-trained model. The output of the 12th layer of the fine-tuned model is extracted as the input of the BiLSTM network, and the prediction probability of each branch is obtained by passing through a fully connected layer and a Sigmoid activation function. Then, the prediction probabilities of the outputs of the LUKE semantic encoding branch and the RoBERTa semantic encoding branch are weighted and averaged according to the weight α to obtain the final prediction result. Step 4: Model Evaluation The model's performance was evaluated using cross-validation and an independent test set. Step 5: Interpretability Analysis SHAP was used to perform interpretability analysis on the model output, and the biophysical and chemical rationality of the model's decisions was verified by combining t-SNE feature visualization.

2. The method for identifying succinylation sites based on an entity knowledge-driven language model as described in claim 1, characterized in that, The first step, as described above, is performed as follows: 1.1 Data Sources and Initial Screening Using the dbPTM database as the data source for succinylation sites, experimentally validated positive samples of lysine succinylation sites and their corresponding protein entries were obtained, and the complete amino acid sequences of these protein entries were acquired from the UniProt database. A positive sample sequence fragment of length 2r+1 was constructed, centered on lysine K and with a window radius r=10. Its standardized representation is as follows: ; 1.2 Positive Sample Sequence Expansion and Filling The 21-position positive sample sequence fragment is extended by 40 positions on both sides of the central site K to expand it to 101 positions. When the central site K is close to both ends of the protein sequence, making equal-length extension impossible, mirror completion is used, and the mirror completion relationship is as follows: ； If the length is still less than 101 bits after mirror completion, use the special symbol "X" to pad it to 101 bits; 1.3 Negative Sample Construction and Screening Sequence fragments used for positive sample expansion but not padded were removed from the complete protein sequence; then, 101 sequence fragments were truncated before and after lysine K to form a negative sample candidate set; the negative sample candidate set was screened to remove candidate fragments with a similarity of more than 80% to any positive sample sequence and candidate fragments containing known succinylation site motifs, thus obtaining the negative sample set. 1.4 Sample Balancing The NearMiss-1 downsampling algorithm is used to downsample the negative sample set, and the k nearest neighbors of the minority class are determined. The average distance from the majority class sample to the k nearest neighbors of the minority class is calculated. The majority class samples are sorted in ascending order of the average distance, and the samples with the highest average distance are selected. The majority class samples are used to form a subset of the majority class samples after downsampling, thereby achieving a balance between the number of positive and negative samples.

3. The method for identifying succinylation sites based on an entity knowledge-driven language model as described in claim 1, characterized in that, The second step, as described above, is performed as follows: 2.1 Amino Acid Composition Characteristics AAC Extraction Suppose the input protein sequence Map it to a 21-dimensional vector The frequency of the j-th amino acid , ; 2.2 One-Hot Encoding Feature Extraction Define mapping Each residue in the sequence is mapped to a 21-dimensional binary vector, and the sequence is represented as an L×21 sparse matrix; 2.3 Feature DR Extraction Based on Residue Distance Let the position of the j-th amino acid in the sequence be . Then the average position Standardized average position Distance residue features ; 2.4 Feature Fusion AAC features, One-Hot features, and DR features are concatenated and fused to form a composite feature vector, which is then used as the model input.

4. The method for identifying succinylation sites based on an entity knowledge-driven language model as described in claim 1, characterized in that, The third step, as described above, is performed as follows: 3.1 Constructing the LUKE semantic encoding branch The protein sequence was segmented and entity-labeled with the functional or structural domain information corresponding to the amino acid residues. The results were then input into the LUKE model and fine-tuned. The output of the 12th layer of the fine-tuned LUKE model was extracted as the temporal input. The input is fed into a bidirectional long short-term memory network (BiLSTM), where: Forward hidden state update: ； Backward hidden state update: ； Simultaneous step-hidden state splicing: ； Will The input is fed into a fully connected layer and activated by Sigmoid to obtain the prediction probability of the LUKE semantic coding branch. ; 3.2 Constructing the RoBERTa semantic encoding branch The protein sequence was input into the RoBERTa model and fine-tuned. The output of the 12th layer of the fine-tuned RoBERTa model was extracted as the input to a BiLSTM, which was then passed through a fully connected layer and activated by a Sigmoid to obtain the prediction probability of the RoBERTa semantic coding branch. ; 3.3 Parameter Optimization The LUKE and RoBERTa models were fine-tuned using the binary cross-entropy loss function BCELoss, with the AdamW optimizer and the LinearLR learning rate scheduler used to decay the learning rate and update the model parameters. 3.4 Model Integration and Output The predicted probabilities of the LUKE semantic coding branch and the RoBERTa semantic coding branch are weighted. Perform a probability-weighted average to obtain the final predicted probability: ； in and ;according to Output the results of succinylation site identification.

5. The method for identifying succinylation sites based on an entity knowledge-driven language model as described in claim 1, characterized in that, The fourth step, as described above, is performed as follows: We employ a dual validation strategy of 10-fold cross-validation and independent test set validation, and introduce existing mainstream prediction methods as a benchmark for comparison. We select six core evaluation indicators, including accuracy, precision, recall, F1 score, Matthews correlation coefficient, and area under the ROC curve, covering multiple dimensions such as positive and negative sample identification ability, overall prediction accuracy, class balance adaptability, and probabilistic discrimination performance.

6. The method for identifying succinylation sites based on an entity knowledge-driven language model as described in claim 1, characterized in that, The fifth step, as described above, is performed as follows: 5.1t-SNE Feature Visualization Analysis The high-dimensional semantic feature vector generated by the pre-trained large language model for the test sample sequence is obtained, and the high-dimensional semantic feature vector is input into the t-SNE dimensionality reduction algorithm to obtain a two-dimensional projection representation. In the two-dimensional projection representation, positive samples and negative samples are visualized and compared to characterize the separability of semantic features for the two types of samples. 5.2 SHAP Interpretability Analysis A background dataset for attribution calculation is constructed, and the SHAP method is used to attribute the input features of the pre-trained large language model to obtain the SHAP value corresponding to each amino acid residue and residue combination. The SHAP value is statistically analyzed to obtain the importance of each residue and residue combination, and the key residues and key residue combinations that contribute the most to the succinylation site identification results are output accordingly.

7. The method for identifying succinylation sites based on an entity knowledge-driven language model as described in claim 1, characterized in that, The parameters of the t-SNE dimensionality reduction algorithm are set as follows: The perplexity was set to 30 and the number of iterations was 1000. The high-dimensional semantic features extracted by the pre-trained large language model and the traditional One-Hot encoded features were subjected to t-SNE dimensionality reduction projection to compare the clustering and inter-class boundary differences of positive and negative samples in two-dimensional space under different feature representations.

8. The method for identifying succinylation sites based on an entity knowledge-driven language model as described in claim 6, characterized in that, The SHAP interpretability analysis specifically includes: Multiple background datasets of different sizes were constructed. The average absolute SHAP value of each amino acid residue and residue pair was calculated under the condition that the background sample size was 100 to 1000 and the step size was 100. The top-30 important residues and top-30 important residue pairs with the highest contribution were selected in each background dataset. The frequency of the top-30 important residue pairs obtained from multiple background datasets was statistically analyzed, and the high-frequency key residue pairs were output as candidate key sequence features for model discrimination based on the frequency statistical results.

Images