A method of characterizing a polypeptide

Abstract

The present application relates to the technical field of polypeptide research and development, in particular to a polypeptide characterization method, which can input the polypeptide sequence to be characterized into a constructed polypeptide characterization learning model and output a polypeptide characterization vector. The polypeptide characterization learning model can respectively perform discrete residue Token coding and continuous physicochemical property coding on the polypeptide sequence; then, a multi-branch embedding layer is used to obtain a high-dimensional learnable vector, a position embedding vector and a physicochemical property embedding vector; then, a ternary decoupling attention layer is used to construct three different dimension feature dependencies and integrate them into a global output characterization; after being encoded by a multi-layer Transformer, a polypeptide characterization vector is obtained. By means of the double-modal numerical coding and the ternary decoupling attention architecture, the technical scheme can accurately capture the functional characteristics of polypeptides, effectively analyze the functional differences of different polypeptides, and analyze the physicochemical properties of polypeptide sequences, and has good biophysical interpretability.

Description

Technical Field

[0001] This invention relates to the field of peptide research and development technology, and specifically to a peptide characterization method. Background Technology

[0002] In recent years, bioactive peptides have been widely used in peptide drug development and functional discovery due to their high specificity, low immunogenicity, and good safety profile. With the rapid development of deep learning technology, protein language models (PLMs) based on large-scale pre-trained models have become important tools for characterizing protein sequences, predicting protein folding, and identifying functional motifs. These models, through self-supervised learning on massive amounts of protein sequences, can capture evolutionary information between amino acids and global topological features, thereby replacing expensive structural experiments.

[0003] However, despite significant progress in handling large-scale globular proteins, existing protein language models still have limitations in the functional analysis and characterization of peptides (typically short sequences of less than 50 residues). Existing models are insufficient in resolving peptide-specific functional signals. Most general-purpose protein language models (such as ESMC) are designed to optimize global folding topology. Due to the short length of peptide sequences and the often lack of stable tertiary structures, their biological activity often depends on extremely high-density local biochemical motifs. Existing peptide characterization methods often lack biophysical interpretability. Most deep learning models are considered "black boxes," and their attention mechanisms fail to effectively decouple semantic information from physicochemical priors. In practical applications, researchers need not only to predict whether a peptide is active but also to identify which key residues drive the interactions. Existing scoring or characterization methods fail to effectively nonlinearly fuse the sequence semantics and physicochemical characteristics of peptides, making it difficult to accurately reveal the deep sequence-function mapping relationship.

[0004] In summary, existing peptide characterization methods lack sufficient precision in short peptide characterization and lack biophysical interpretability. Summary of the Invention

[0005] In view of this, the purpose of the present invention is to provide a peptide characterization method to solve the problems of insufficient accuracy in short peptide characterization and lack of biophysical interpretability in existing peptide characterization methods.

[0006] According to a first aspect of the present invention, a method for characterizing a polypeptide is provided, comprising:

[0007] Obtain the peptide sequence to be characterized, input the peptide sequence into the constructed peptide characterization learning model, and output the peptide characterization vector; The polypeptide characterization learning model includes: A dual-modal numerical coding layer performs discrete residue token encoding and continuous physicochemical property encoding on the polypeptide sequence to obtain an identifier ID sequence and a multi-dimensional physicochemical feature vector. The multi-branch embedding layer derives a high-dimensional learnable vector corresponding to each residue in the polypeptide sequence based on the identifier ID sequence; it then uses sine-cosine fixed-position encoding based on the length of the identifier ID sequence to obtain a position embedding vector; and finally performs nonlinear mapping and standardization on the multi-dimensional physicochemical feature vector to obtain a physicochemical property embedding vector. The three-dimensional decoupled attention layer takes a high-dimensional learnable vector as input to the semantic autocorrelation flow to obtain the first output matrix; takes a high-dimensional learnable vector and a physicochemical property embedding vector as input to the physicochemical regulation flow to obtain the second output matrix; takes a high-dimensional learnable vector and a position embedding vector as input to the position topological deviation flow to obtain the third output matrix; and concatenates the output matrices and performs nonlinear feature refinement to obtain the global output representation. The coding layer takes the global output representation as input to a multi-layer Transformer coding structure; The output layer is characterized by extracting global sequence feature vectors from the top-level Transformer encoding structure and mapping these global sequence feature vectors into a continuous spatial vector of fixed dimensions to obtain the peptide characterization vector.

[0008] Preferably, for the length of the identifier ID sequence, sine-cosine fixed-position encoding is used to obtain the position embedding vector, including: For any position in the ID sequence, if the position is an even-numbered dimension, a sine function is used for assignment, as shown in the following formula:

[0009] If the sequence position is in an odd-numbered dimension, the cosine function is used for assignment, as shown in the following formula:

[0010] in, This is the encoded value corresponding to the even-numbered dimension at the pos-th position of the sequence; For the sequence number Each position corresponds to an odd-numbered dimension's encoded value; Embed dimensions into the model; This serves as the position index for identifying residues within the ID sequence; The encoded values ​​of each sequence position after assignment are integrated into a position embedding vector.

[0011] Preferably, the multidimensional physicochemical feature vectors are subjected to nonlinear mapping and standardization to obtain physicochemical property embedding vectors, including: A linear transformation is used to project multidimensional physicalized feature vectors onto the model embedding dimension; The projection results are nonlinearly transformed using the GELU activation function; The nonlinear transformation result is normalized using layer normalization to obtain the physicochemical property embedding vector; the physicochemical property embedding vector, the high-dimensional learnable vector, and the position embedding vector have the same dimension.

[0012] Preferably, the high-dimensional learnable vector is used as input to the semantic autocorrelation stream to obtain the first output matrix, including: The high-dimensional learnable vector is generated by linear projection. matrix, Matrix and Matrix; according to matrix, Matrix and The matrix is ​​subjected to mask scaling, dot product attention calculation to obtain the first output matrix; Using the high-dimensional learnable vector and the physicochemical property embedding vector as inputs to the physicochemical regulation flow, a second output matrix is ​​obtained, including: The high-dimensional learnable vector is generated by linear projection. A matrix is ​​generated by linearly projecting the physicochemical properties embedded in the vector. Matrix and Matrix; according to matrix, Matrix and The matrix is ​​subjected to mask scaling, dot product attention calculation to obtain the second output matrix; Using the high-dimensional learnable vector and the position embedding vector as input to the position topology bias flow, a third output matrix is ​​obtained, including: The high-dimensional learnable vector is generated by linear projection. A matrix is ​​generated by linearly projecting the position embedding vector. Matrix and Matrix; according to matrix, Matrix and The matrix is ​​subjected to mask scaling, dot product, and attention calculation to obtain the third output matrix.

[0013] Preferably, the mask scaling dot product attention calculation is performed based on the Q matrix, K matrix, and V matrix, including: Perform matrix multiplication on the Q matrix and the transposed K matrix to obtain the similarity score matrix; The similarity score matrix is ​​scaled according to the dimensions of the K matrix; Add masks to invalid locations on the scaled similarity score matrix; Perform Softmax normalization on the similarity score matrix after adding the mask to obtain the attention weight matrix; Perform matrix multiplication between the attention weight matrix and the V matrix to obtain the attention output matrix.

[0014] Preferably, the global output representation is input into a multi-layer Transformer coding structure, including: Each Transformer coding structure first performs residual connection and layer normalization on the input data, then inputs the layer normalization result into the feedforward neural network for nonlinear feature refinement, and then performs residual connection and layer normalization again on the output result of the feedforward neural network to obtain the output result of the Transformer coding structure of this layer. The input data of the first layer Transformer coding structure is the global output representation, and the input data of other layers Transformer coding structures is the output result of the previous layer Transformer coding structure.

[0015] Preferably, the polypeptide characterization method further includes: Obtain a dataset containing multiple known peptide sequences, and divide the dataset into a training set and a validation set; Obtain the mask sequence label, physicochemical property features, and attention mask vector for each polypeptide sequence in the training set to construct the model training set; The training set is input into the peptide characterization learning model to obtain sequence prediction results; the weighted loss value of the sequence prediction results and the corresponding peptide sequence is calculated, and mean normalization is performed according to the mask sequence label to obtain the final loss value; the model weights of the peptide characterization learning model are updated according to the final loss value; the validation set is input into the updated peptide characterization learning model to calculate the validation set loss value; using the updated peptide characterization learning model, this step is repeated until the preset maximum number of iterations is reached; The optimal model weights are determined by the model weights in the iterations corresponding to the optimal validation set loss value.

[0016] Preferably, the dataset is divided into a training set and a validation set, including: The dataset contains multiple known polypeptide sequences that are divided into a first part and a second part in a fixed ratio. The first part is used as the training set and the second part is used as the validation set.

[0017] Preferably, a dataset containing multiple known polypeptide sequences is obtained, including: Obtain raw sequence data from a publicly available protein sequence database, extract polypeptide sequences whose length is less than a preset limit from the raw sequence data, and sequentially remove invalid sequences and duplicates to obtain the UniProt screening set; Obtain ColabFoldDB sequence resources, extract peptide sequences whose length is less than a preset limit from the ColabFoldDB sequence resources, and sequentially remove invalid sequences and duplicates to obtain the ColabFoldDB screening set; Merge the UniProt filter set and the ColabFoldDB filter set, and perform deduplication on the merged dataset to obtain the original corpus. Site-directed mutagenesis was performed on each amino acid site of the polypeptide sequences in the original corpus to generate mutant data including various amino acid substitutions. Using affinity index as a quantitative screening criterion, invalid mutants were removed from all mutant data to obtain the dataset.

[0018] Preferably, the peptide characterization method further includes: fine-tuning the peptide characterization learning model according to a preset downstream task.

[0019] The technical solution provided by this invention may include the following beneficial effects: It is understood that the technical solution presented in this invention can input the polypeptide sequence to be characterized into a constructed polypeptide characterization learning model and output a polypeptide characterization vector. The polypeptide characterization learning model can encode discrete residue tokens and continuous physicochemical properties of the polypeptide sequence; then, it uses a multi-branch embedding layer to obtain a high-dimensional learnable vector, a position embedding vector, and a physicochemical property embedding vector; next, it uses a ternary decoupled attention layer to construct three different dimensions of feature dependencies and integrates them into a global output representation; finally, it uses multi-layer Transformer encoding to obtain the polypeptide characterization vector. This technical solution, through dual-modal numerical encoding and a ternary decoupled attention architecture, can accurately capture the functional characteristics of polypeptides, effectively analyze the functional differences between different polypeptides, and simultaneously analyze the physicochemical properties of the polypeptide sequence, possessing good biophysical interpretability.

[0020] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit the invention. Attached Figure Description

[0021] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0022] Figure 1 This is a schematic diagram illustrating the steps of running a peptide characterization learning model according to an exemplary embodiment; Figure 2 This is a flowchart illustrating the data processing of a peptide characterization learning model according to an exemplary embodiment; Figure 3This is a diagram illustrating a dual-modal numerical coding architecture according to an exemplary embodiment; Figure 4 This is a schematic diagram of a ternary decoupling attention mechanism according to an exemplary embodiment; Figure 5 This is a schematic diagram illustrating a data collection and processing process according to an exemplary embodiment; Figure 6 This is a comparison chart of model results illustrated according to an exemplary embodiment; Figure 7 This is a comparison chart of independent test set results illustrated according to an exemplary embodiment; Figure 8 This is a comparison chart of ablation experiment results shown according to an exemplary embodiment; Figure 9 This is a graph of AMP result data illustrated according to an exemplary embodiment; Figure 10 This is a graph of AIP result data illustrated according to an exemplary embodiment. Detailed Implementation

[0023] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the invention as detailed in the appended claims.

[0024] In one embodiment, see Figure 1 and Figure 2 A peptide characterization method is provided, comprising: obtaining a peptide sequence to be characterized, inputting the peptide sequence into a constructed peptide characterization learning model, and outputting a peptide characterization vector.

[0025] The peptide characterization learning model includes: a bimodal numerical coding layer, a multi-branch embedding layer, a ternary decoupled attention layer, a coding layer, and a characterization output layer.

[0026] A dual-modal numerical coding layer performs discrete residue token encoding and continuous physicochemical property encoding on the polypeptide sequence to obtain an identifier ID sequence and a multidimensional physicochemical feature vector.

[0027] In practical application, see Figure 3The dual-modal numerical coding layer is used to encode the original amino acid string form of a polypeptide sequence into a structured code with two complementary information modalities. Among them, the discrete residue token encoding maps each amino acid residue in the sequence to a corresponding unique integer identifier ID through a word segmenter, and arranges them sequentially to form an identifier ID sequence, which is used to carry the semantic association and evolutionary pattern features of the polypeptide sequence.

[0028] The continuous physicochemical property encoding generates a multidimensional physicochemical feature vector for each residue in the sequence, which includes a 19-dimensional multi-scale hydrophobicity index and a 1-dimensional residue flexibility parameter, and quantitatively characterizes the key physicochemical constraints on which the peptide exerts its biological function.

[0029] Through the above two-way parallel encoding, this layer achieves complementary encoding of discrete semantic information and continuous physicochemical property information, which not only preserves the evolutionary semantic information of the sequence, but also supplements the physicochemical constraint information that the short peptide function is highly dependent on.

[0030] The multi-branch embedding layer derives a high-dimensional learnable vector corresponding to each residue in the polypeptide sequence based on the identifier ID sequence; it then uses sine-cosine fixed-position encoding based on the length of the identifier ID sequence to obtain a position embedding vector; and finally performs nonlinear mapping and standardization on the multidimensional physicochemical feature vector to obtain a physicochemical property embedding vector.

[0031] In practice, the multi-branch embedding layer receives the output of the bimodal numerical coding layer and maps the coding results of different modalities to a continuous representation space of a unified dimension through three independent embedding branches.

[0032] Specifically, the embedding branch based on residue identifier IDs maps each discrete residue ID to a corresponding high-dimensional learnable vector E through a learnable embedding lookup table. token , used to characterize the global evolutionary semantic features of polypeptide sequences.

[0033] To address the spatial location information of the sequence, a sine-cosine fixed-position encoding mechanism is adopted. The encoding is performed based on the actual length of the polypeptide sequence after bimodal encoding, and a corresponding position embedding vector is generated for each sequence position, enabling the model to perceive the sequence arrangement order and topological position relationship of residues.

[0034] In a preferred embodiment, for any sequence position in the identifier ID sequence, if the sequence position has an even dimension, a sine function is used for assignment, as shown in the following formula:

[0035] If the sequence position is in an odd-numbered dimension, the cosine function is used for assignment, as shown in the following formula:

[0036] in, This is the encoded value corresponding to the even-numbered dimension at the pos-th position of the sequence; For the sequence number Each position corresponds to an odd-numbered dimension's encoded value; Embed dimensions into the model; This is used to identify the position index of residues in the ID sequence.

[0037] The encoded values ​​of each sequence position after assignment are integrated into a position embedding vector.

[0038] Preferably, the maximum length L can be preset. max The polypeptide sequence, with a corresponding maximum length L max The maximum-length ID sequence is given, and sine-cosine fixed-position encoding is performed to obtain the encoded values ​​of each position. These values ​​are then integrated into a position encoding matrix, which is stored as a model constant. During forward propagation, this position encoding matrix can be dynamically truncated according to the actual length of the input ID sequence and expanded by batching to obtain a position embedding vector E that matches the input ID sequence. pos This provides precise one-dimensional spatial location constraints for polypeptide sequences.

[0039] In a preferred embodiment, for multidimensional physicochemical feature vectors, a linear transformation is used to project the multidimensional physicochemical feature vectors onto the model embedding dimension, and the calculation formula is as follows:

[0040] in, This is the intermediate feature vector output by the linear transformation; It is a linear transformation operation function; The input is a multidimensional physicochemical feature vector; is the learnable weight matrix of the linear transformation layer, and is the trainable parameters of the model; Let be the learnable bias vector of this linear transformation layer, and be the trainable parameters of the model.

[0041] The GELU activation function is used to perform a nonlinear transformation on the projection results to enhance the feature representation ability: ,in, This is the result of a nonlinear transformation. This is the GELU activation function.

[0042] Layer normalization is used to normalize the nonlinear transformation results to stabilize the training process and obtain the physicochemical property embedding vector. : ,in, This is the layer normalization function.

[0043] The dimensions of the physicochemical property embedding vector, the high-dimensional learnable vector, and the position embedding vector are consistent.

[0044] This technical solution sequentially maps low-dimensional continuous physicochemical features into physicochemical property embedding vectors with the same embedding dimensions as the first two types through linear transformation projection, nonlinear transformation of GELU activation function and layer normalization processing, thus preserving the physicochemical prior constraints of peptides in a unified representation space.

[0045] Preferably, the three types of embedding vectors are initially fused by adding them element by element, achieving dimensional alignment and initial integration of the three types of information: discrete semantics, spatial topology, and physicochemical properties. This preserves the independent semantics of each modality and provides a unified input basis for the subsequent multi-dimensional feature dependency modeling of the ternary decoupled attention.

[0046] The ternary decoupled attention layer takes a high-dimensional learnable vector as input to a semantic autocorrelation flow to obtain the first output matrix; takes a high-dimensional learnable vector and a physicochemical attribute embedding vector as input to a physicochemical regulation flow to obtain the second output matrix; takes a high-dimensional learnable vector and a position embedding vector as input to a position topological deviation flow to obtain the third output matrix; and concatenates the first, second, and third output matrices and performs nonlinear feature refinement to obtain the global output representation.

[0047] In practical application, see Figure 4 The ternary decoupled attention layer decouples the traditional Transformer's single attention flow into three parallel mask-scaled dot product attention flows (semantic autocorrelation flow, physicochemical conditioning flow, and positional topological deviation flow). It models feature dependencies of three different dimensions: semantic autocorrelation, semantic-physicochemical conditioning, and semantic-topological correlation. The three flows share the Query source, while the Key source and Value source correspond to different embedding branches.

[0048] Semantic autocorrelation flow: Taking a high-dimensional learnable vector as input to a semantic autocorrelation flow to obtain a first output matrix, including: generating a first output matrix from the high-dimensional learnable vector through linear projection. matrix, Matrix and Matrix; according to matrix, Matrix and The matrix is ​​subjected to mask scaling, dot product, and attention calculation to obtain the first output matrix. The calculation formula is:

[0049] in, for Matrix transpose for The dimension of the matrix is ​​used to scale the dot product result to avoid gradient vanishing. This represents Softmax activation with masking, which masks attention weights at invalid locations.

[0050] Each matrix is ​​generated by linear projection of a high-dimensional learnable vector embedded in the token:

[0051]

[0052]

[0053] Semantic autocorrelation streams use high-dimensional learnable vectors as their sole input source, from which a query matrix is ​​generated through independent linear projections. Key matrix AND-value matrix The first output matrix is ​​obtained by mask scaling dot product attention calculation, which focuses on modeling the sequence semantic context dependency between residues and capturing the evolutionary association and global semantic pattern of peptide sequences.

[0054] Physicochemical regulation flow: A second output matrix is ​​obtained by using a high-dimensional learnable vector and a physicochemical property embedding vector as inputs to the physicochemical regulation flow, including: generating the high-dimensional learnable vector through linear projection. A matrix is ​​generated by linearly projecting the physicochemical properties embedded in the vector. Matrix and Matrix; according to matrix, Matrix and The matrix is ​​subjected to mask scaling, dot product, and attention calculation to obtain the second output matrix. The calculation formula is:

[0055] in, Matrix and semantic autocorrelation flow Matrix consistency for Matrix transpose Matrix and The matrix is ​​generated by linear projection of the physicochemical property embedding vectors:

[0056]

[0057] Physicochemical regulatory flow specifically models the dynamic regulatory relationship between sequence semantics and physicochemical properties, realizing the precise constraint of physicochemical priors on semantic representation, and enhancing the model's ability to capture the physicochemical driving laws of local functional motifs of short peptides.

[0058] Location topology deviation flow: Taking a high-dimensional learnable vector and a location embedding vector as inputs to the location topology deviation flow, a third output matrix is ​​obtained, including: generating the high-dimensional learnable vector through linear projection. A matrix is ​​generated by linearly projecting the position embedding vector. Matrix and Matrix; according to matrix, Matrix and The matrix is ​​subjected to mask scaling, dot product, and attention calculation to obtain the third output matrix. The calculation formula is:

[0059] in, Matrix and semantic autocorrelation flow Matrix consistency for Matrix transpose Matrix and The matrix is ​​generated by linear projection of the position embedding vector:

[0060]

[0061] Position topology bias flow is used to explicitly lock the association constraints between the semantic features of residues and the spatial topological positions of sequences, enhance the model's perception of the arrangement of functional motifs, and adapt to the biological characteristics of peptide function being highly dependent on the distribution of motif positions.

[0062] Finally, the output matrices of the three attention branches are concatenated to achieve dimensional fusion of multi-dimensional features. After being adapted to the dimensions by a linear projection matrix, they are fed into a feedforward neural network to complete nonlinear feature refinement and depth mining, ultimately obtaining a global output representation TDA(·) that integrates semantic, physicochemical, and topological information. The overall calculation formula is as follows:

[0063] Where Concat is the concatenation operation, Wo is the linear projection matrix, and FeedForward is the feedforward neural network.

[0064] All of the above attention streams strictly follow the basic calculation process of mask-scaled dot product attention, and the specific calculation formula is as follows:

[0065] The calculation method is as follows: perform matrix multiplication on the Q matrix and the transposed K matrix to obtain a similarity score matrix; scale the similarity score matrix according to the dimension of the K matrix; add masks to invalid positions on the scaled similarity score matrix; perform Softmax normalization on the masked similarity score matrix to obtain an attention weight matrix; perform matrix multiplication on the attention weight matrix and the V matrix to obtain an attention output matrix.

[0066] Understandably, this technical solution not only achieves decoupled modeling of features depending on different dimensions, but also ensures deep fusion of multi-source information, providing a refined multimodal feature foundation for subsequent multi-layer Transformer encoding of progressively evolving features.

[0067] The encoding layer takes the global output representation as input to the multi-layer Transformer encoding structure.

[0068] It should be noted that each Transformer encoding structure first performs residual concatenation and layer normalization on the input data. The normalized result is then fed into a feedforward neural network for nonlinear feature refinement. The output of the feedforward neural network is then subjected to residual concatenation and layer normalization again to obtain the output of the current Transformer encoding structure. The input data of the first Transformer encoding structure is the global output representation, while the input data of other Transformer encoding structures are the outputs of the previous Transformer encoding structure.

[0069] In practice, each Transformer encoding layer constitutes a standard computational unit for the model to progressively extract multimodal features. The features input to this layer first undergo the first set of residual connections and layer normalization operations. The normalized features are then fed into a feedforward neural network, which performs a nonlinear depth transformation on the fused features at each residue site to uncover higher-order combinatorial relationships within the features. The output of the feedforward neural network then undergoes the second set of residual connections and layer normalization operations, ultimately generating the final output features of this layer's Transformer encoding structure, which are then used by the next layer to perform higher-level feature abstraction and extraction.

[0070] By stacking multiple layers of the aforementioned Transformer coding structure, the model can extract multi-level, high-dimensional features from local residue associations to global peptide functional patterns, enabling in-depth mining and characterization of complex biological information of peptides.

[0071] The characterization output layer extracts global sequence feature vectors from the top-level Transformer encoding structure, and maps these global sequence feature vectors into a continuous spatial vector of fixed dimensions through a linear projection layer to obtain the peptide characterization vector.

[0072] Peptide characterization vectors can comprehensively and uniformly carry semantic information, physicochemical properties, and spatial topological structure information of peptide sequences. They have strong discriminative and generalization capabilities and can be directly adapted to various bioinformatics tasks such as downstream peptide function prediction, classification and identification, and target screening.

[0073] In one embodiment, a model training method for a peptide characterization learning model is shown, as follows: First, see Figure 5 Obtain a dataset containing multiple known peptide sequences, including: Step S101: Obtain the raw sequence data from the publicly available protein sequence database UniProt, extract peptide sequences whose length is less than a preset limit (e.g., 50) from the raw sequence data, in order to focus on molecular research at the peptide scale; then, remove invalid sequences and duplicates in sequence to obtain the UniProt screening set.

[0074] Step S102: Obtain ColabFoldDB sequence resources, extract peptide sequences from ColabFoldDB sequence resources that are shorter than a preset limit (e.g., 50 sequences), and remove invalid sequences and duplicates in sequence to obtain the ColabFoldDB screening set.

[0075] ColabFoldDB sequence resources, through MMseqs2-based clustering and diversity filtering technology, deeply integrate Big Fantastic Database (BFD), MGnify, and various metagenomic sequences, providing a broad spectrum of characterization of the global protein landscape.

[0076] Step S103: Merge the UniProt filter set and the ColabFoldDB filter set, and perform deduplication on the merged dataset to obtain the original corpus containing 19,838,117 non-redundant peptide sequences.

[0077] Step S104: Perform site-directed mutagenesis on each amino acid site of the polypeptide sequence in the original corpus to generate mutant data including various amino acid substitutions.

[0078] Step S105: Using affinity index as the quantitative screening criterion, invalid mutants are removed from all mutant data to obtain the dataset.

[0079] By performing systematic site-directed mutagenesis at various amino acid sites in the peptide sequence, mutant data covering various amino acid substitutions were generated. The mutant data were then filtered using affinity indices, ultimately resulting in a high-quality target dataset of 1,324,688,140 sequences for model pre-training. The expanded and high-quality sequence data set formed by the above process provides sufficient and high-quality data foundation support for the subsequent implementation of data augmentation strategies and the pre-training of peptide characterization learning models.

[0080] After obtaining the dataset, the specific training method is as follows: Step S201: Obtain a dataset containing multiple known polypeptide sequences, and divide the dataset into a training set and a validation set.

[0081] Preferably, multiple known polypeptide sequences in the dataset are sequentially divided into a first part and a second part in a fixed ratio (e.g., 9:1), with the first part serving as the training set and the second part serving as the validation set (the ratio of the training set to the validation set is 9:1).

[0082] Step S202: Obtain the mask sequence label, physicochemical property features and attention mask vector of each polypeptide sequence in the training set, and construct the model training set.

[0083] This step uses a custom PetDataset dataset class to perform preprocessing operations such as sequence encoding, embedding of physicochemical property features, and masking. It unifies the input sequence length, physicochemical property feature dimensions, and masking probabilities to construct standardized pre-training input data. The standardized pre-training input data is the model training set, which includes multiple peptide sequences, as well as mask sequence labels, physicochemical property features, and attention mask vectors for the peptide sequences.

[0084] Step S203: Forward propagation inputs the model training set into the peptide characterization learning model to obtain sequence prediction results; calculates the weighted loss value of the sequence prediction results and the corresponding peptide sequence, and performs mean normalization based on the mask sequence label to obtain the final loss value; backpropagation updates the model weights of the peptide characterization learning model based on the final loss value; inputs the validation set into the updated peptide characterization learning model to calculate the validation set loss value; using the updated peptide characterization learning model, this step is repeated until the preset maximum number of iterations (e.g., 20) is reached.

[0085] Step S204: Take the model weights in the iterations corresponding to the optimal validation set loss value as the optimal model weights.

[0086] After each training round, the model loss is evaluated on the validation set, and the validation loss is used as the optimization metric to save the optimal model weight file. Simultaneously, the training and validation losses for each round are recorded in real time, generating an iterative loss CSV file for monitoring the training process and analyzing convergence. Through this standardized training process and optimization strategy, the model is trained to possess the ability to extract basic peptide sequence features and perform semantic representation.

[0087] Preferably, during model training, the hardware environment is configured with a GPU-accelerated device, data loading uses DataLoader for batch processing, a fixed batch size is set, and random shuffling and batch discarding strategies are enabled. The optimizer is the Adam adaptive optimizer, which updates model parameters according to the configured learning rate; the loss function is the cross-entropy loss function, and a mask bit index ignoring parameter is set to adapt to the masked language model training task.

[0088] In a preferred embodiment, the peptide characterization method further includes: fine-tuning the peptide characterization learning model according to a preset downstream task.

[0089] Based on the above model training, this embodiment designs a targeted fine-tuning strategy and verifies the effectiveness of each module through ablation experiments, and finally determines the optimal model configuration.

[0090] In this embodiment, three ablation control schemes are set up: (1) the ternary decoupling attention mechanism is removed and replaced with the traditional self-attention mechanism to verify the improvement effect of the ternary decoupling attention mechanism on peptide feature extraction; (2) the peptide physicochemical property embedding is removed and only the sequence coding information is retained to verify the contribution of physicochemical property features to the model's representation ability; (3) the peptide data augmentation strategy is removed and the original dataset is used for fine-tuning to verify the optimization effect of data augmentation on the model's generalization performance.

[0091] In the specific fine-tuning process, the output features of the penultimate layer of the model are used as the peptide sequence representation vector. After the representation vector is input into a fully connected layer (FC) for dimensionality adaptation, it is connected to a custom CNN classification model to complete the downstream task fine-tuning. The CNN classification model is a one-dimensional convolutional neural network that extracts local features of the representation vector through convolution and pooling operations, ultimately achieving functional prediction.

[0092] In practice, 16 downstream tasks can be fine-tuned, including antimicrobial peptide AMP, antiparasitic peptide APP, antiviral peptide AVP, antiangiogenic peptide AAP, anti-inflammatory peptide AIP, antioxidant peptide AOX, quorum sensing peptide QSP, anticancer peptide (main class) APM, tumor T-cell antigen TTA, blood-brain barrier penetrating peptide BBB, cell-penetrating peptide CPP, toxic peptide TOX, surface-binding peptide SBP, lysine crotonylation KCR, lysine glycosylation KGY, and phosphorylated STL. The data for these 16 downstream tasks are constructed by integrating publicly available bioinformatics databases and standardized evaluation datasets, covering various typical task scenarios such as peptide function prediction and property prediction. Specific sources include authoritative databases such as UniProt, PDB, and DBAASP. All data undergo a unified preprocessing workflow, including sequence length standardization, missing value imputation, label format alignment, and dataset partitioning, to ensure the training stability and evaluation fairness of the downstream fine-tuning tasks.

[0093] In terms of model evaluation, a multi-dimensional set of indicators is used to comprehensively assess the model's representational capabilities, including: accuracy (ACC), balanced accuracy (BACC), sensitivity (SE), specificity (SP), Matthews' correlation coefficient (MCC), area under the curve (AUC), precision (PRE), F1 score, and area under the precision-recall curve (RP-AUC). The formulas are as follows:

[0094]

[0095]

[0096]

[0097]

[0098]

[0099]

[0100] Wherein, TP is the number of samples whose true label is positive and whose model prediction is also positive, FP is the number of samples whose true label is negative and whose model incorrectly predicts as positive, TN is the number of samples whose true label is negative and whose model prediction is also negative, and FN is the number of samples whose true label is positive and whose model incorrectly predicts as negative.

[0101] Through ablation experiments and multi-index evaluation, the fine-tuning scheme incorporating data augmentation strategies achieved optimal performance across all evaluation metrics, significantly outperforming the control scheme that removed the ternary decoupling attention mechanism and physicochemical properties. Therefore, this invention uses the model that integrates data augmentation strategies while retaining the ternary decoupling attention mechanism and physicochemical properties as the final peptide characterization learning model (PepLLM model), possessing optimal peptide sequence characterization capabilities and downstream task adaptability.

[0102] Figure 6 The results show the comparison between the peptide representation learning model and ESMC across 16 downstream tasks. The figure shows the comparison of the area under the curve (AUC) and F1 score, with the vertical axis representing the AUC and F1 scores, respectively. The peptide representation learning model was then fine-tuned for downstream tasks, and its validation results were compared with the existing superior language model ESMC. The comparison results show that the peptide representation learning model consistently outperforms ESMC across all 16 downstream tasks.

[0103] To further validate the model's generalization ability, it was tested on an independent test set of 16 peptide tasks and compared with ESMC. The results are shown in [link to results]. Figure 7 , Figure 7 The solid line represents the peptide characterization learning model, and the dashed line represents ESMC. Figure 7 The results show that the peptide characterization learning model achieved excellent results in more than half of the tasks (10 / 16).

[0104] The peptide characterization learning model is based on a TDA (decoupled attention) architecture containing 1.3 × 10 9 Pre-training was performed on a large-scale corpus of sequences, and physicochemical descriptors were fused at the input stage. To systematically decompose the contributions of each core component, a multi-level ablation framework was constructed: (1) Data scale ablation: compared with 1.9×10 7 (1) Original sequences and enhanced corpus; (2) Architecture ablation: in 1.9×10 7 On the baseline data, physicochemical information or ternary decoupling attention mechanism were removed respectively.

[0105] See Figure 8 In the line graph, the lines from top to bottom represent the results of the peptide characterization learning model, 1.9 × 10⁻⁶, and 1.9 × 10⁻⁶, respectively. 7Results after removing the NotDependency Attention (NOTDA) mechanism: 1.9 × 10 7 The result after removing data augmentation: 1.9 × 10 7 The result after removing physicochemical properties (NOPRO). From Figure 8 As can be seen, the complete peptide characterization learning model framework demonstrates an absolute performance advantage in results validation, achieving the best prediction accuracy on 87.5% (14 / 16) of the evaluation tasks. This systematic advantage proves the model's architectural robustness in dealing with the inherent heterogeneity of peptide bioactivity data. Analysis shows that removing specific modules—namely, data augmentation, the TDA engine, or physicochemical descriptors—causes "local performance collapse" on multiple tasks. Simplified variants only maintain leadership in a few specific tasks, indicating that while a single modality can capture specific biochemical signals, it lacks the generalized characterization capabilities required for universal peptide modeling.

[0106] In the peptide representation learning model, the UMAP dimensionality reduction visualization method was used to validate the feature outputs at different levels of the model, intuitively evaluating its unsupervised representation discrimination ability. The results show that even without any downstream task fine-tuning, the deep representations extracted by the peptide representation learning model can clearly separate positive and negative samples and aggregate them within classes, significantly different from the mixed distribution of the initial embedding layer. This fully demonstrates that the model has autonomously learned the discriminative features of peptide sequences during the pre-training stage and possesses a natural sample discrimination ability.

[0107] To further evaluate the model's generalization ability for decoding different biological functions, we investigated anti-inflammatory peptides (AIPs) and antimicrobial peptides (AMPs) of typical short peptides (length <30 residues) and compared their performance. By calculating the model's average attention weights for 20 common amino acids, we found that decoupling attention (TDA) exhibited highly specific characteristic responses when processing different functional peptides: when processing antimicrobial peptides, TDA highly focused on leucine (L), tyrosine (Y), and arginine (R), achieving a deep alignment between the attention distribution and the molecular mechanism of antimicrobial peptides, namely "hydrophobic transmembrane driving—conformational stability support—electrostatic membrane targeted recognition"; when decoding anti-inflammatory peptides, TDA significantly enhanced the characteristic weights for leucine (L), tyrosine (Y), and aspartic acid (D), accurately capturing the biophysical characteristics of AIPs regulating signaling pathways through specific electrostatic interactions and hydrophobic motifs.

[0108] See Figure 9 and Figure 10 This indicates that the peptide characterization learning model can autonomously learn and accurately distinguish the key biophysical and structural features of different functional peptides, and has strong biological function interpretability and specific characterization capabilities.

[0109] In real-world applications, the model accurately predicted the anti-inflammatory activity of an oyster-derived anti-inflammatory peptide (sequence: QHGV), showing a high degree of consistency with experimental results and precisely capturing the peptide's anti-inflammatory potential. For a naturally derived antioxidant peptide (sequence: KGEYNK), the model successfully identified its antioxidant function, with predictions perfectly matching experimentally measured antioxidant activity, validating the model's accuracy in recognizing the function of natural active peptides. For peptide sequences with high homology (such as PFLF and IALLIPF), the model accurately distinguished their functional differences and correctly determined their activity levels, perfectly matching experimental results (IALLIPF>PFLF).

[0110] The above practical cases demonstrate that the peptide characterization learning model can accurately capture the functional characteristics of peptides and effectively analyze the functional differences of different peptides, providing reliable technical support for peptide drug development and natural bioactive peptide screening, and further demonstrating the practical value and stability of the model in real-world applications.

[0111] It is understood that the same or similar parts in the above embodiments can be referred to each other, and the contents not described in detail in some embodiments can be referred to the same or similar contents in other embodiments.

[0112] It should be noted that in the description of this invention, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Furthermore, in the description of this invention, unless otherwise stated, "a plurality of" means at least two.

[0113] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which embodiments of the invention pertain.

[0114] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0115] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0116] Furthermore, the functional units in the various embodiments of the present invention can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0117] The storage media mentioned above can be read-only memory, disk, or optical disk, etc.

[0118] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0119] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for characterizing polypeptides, characterized in that, include: Obtain the peptide sequence to be characterized, input the peptide sequence into the constructed peptide characterization learning model, and output the peptide characterization vector; The polypeptide characterization learning model includes: A dual-modal numerical coding layer performs discrete residue token encoding and continuous physicochemical property encoding on the polypeptide sequence to obtain an identifier ID sequence and a multi-dimensional physicochemical feature vector. The multi-branch embedding layer derives a high-dimensional learnable vector corresponding to each residue in the polypeptide sequence based on the identifier ID sequence; it then uses sine-cosine fixed-position encoding based on the length of the identifier ID sequence to obtain a position embedding vector; and finally performs nonlinear mapping and standardization on the multi-dimensional physicochemical feature vector to obtain a physicochemical property embedding vector. The three-dimensional decoupled attention layer takes a high-dimensional learnable vector as input to the semantic autocorrelation flow to obtain the first output matrix; takes a high-dimensional learnable vector and a physicochemical property embedding vector as input to the physicochemical regulation flow to obtain the second output matrix; takes a high-dimensional learnable vector and a position embedding vector as input to the position topological deviation flow to obtain the third output matrix; and concatenates the output matrices and performs nonlinear feature refinement to obtain the global output representation. The coding layer takes the global output representation as input to a multi-layer Transformer coding structure; The output layer is characterized by extracting global sequence feature vectors from the top-level Transformer encoding structure and mapping these global sequence feature vectors into a continuous spatial vector of fixed dimensions to obtain the peptide characterization vector.

2. The polypeptide characterization method according to claim 1, characterized in that, For the length of the identifier ID sequence, sine-cosine fixed-position encoding is used to obtain the position embedding vector, including: For any position in the ID sequence, if the position is an even-numbered dimension, a sine function is used for assignment, as shown in the following formula: If the sequence position is in an odd-numbered dimension, the cosine function is used for assignment, as shown in the following formula: in, For the sequence number Each position corresponds to an even-numbered dimension in the encoded value; For the sequence number Each position corresponds to an odd-numbered dimension's encoded value; Embed dimensions into the model; This serves as the position index for identifying residues within the ID sequence; The encoded values ​​of each sequence position after assignment are integrated into a position embedding vector.

3. The polypeptide characterization method according to claim 1, characterized in that, Nonlinear mapping and standardization are performed on the multidimensional physicochemical feature vectors to obtain physicochemical property embedding vectors, including: A linear transformation is used to project multidimensional physicalized feature vectors onto the model embedding dimension; The projection results are nonlinearly transformed using the GELU activation function; The nonlinear transformation result is normalized using layer normalization to obtain the physicochemical property embedding vector; the physicochemical property embedding vector, the high-dimensional learnable vector, and the position embedding vector have the same dimension.

4. The polypeptide characterization method according to claim 1, characterized in that, Using a high-dimensional learnable vector as input to a semantic autocorrelation stream, the first output matrix is ​​obtained, including: The high-dimensional learnable vector is generated by linear projection. matrix, Matrix and Matrix; according to matrix, Matrix and The matrix is ​​subjected to mask scaling, dot product attention calculation to obtain the first output matrix; Using the high-dimensional learnable vector and the physicochemical property embedding vector as inputs to the physicochemical regulation flow, a second output matrix is ​​obtained, including: The high-dimensional learnable vector is generated by linear projection. A matrix is ​​generated by linearly projecting the physicochemical properties embedded in the vector. Matrix and Matrix; according to matrix, Matrix and The matrix is ​​subjected to mask scaling, dot product attention calculation to obtain the second output matrix; Using the high-dimensional learnable vector and the position embedding vector as input to the position topology bias flow, a third output matrix is ​​obtained, including: The high-dimensional learnable vector is generated by linear projection. A matrix is ​​generated by linearly projecting the position embedding vector. Matrix and Matrix; according to matrix, Matrix and The matrix is ​​subjected to mask scaling, dot product, and attention calculation to obtain the third output matrix.

5. The polypeptide characterization method according to claim 4, characterized in that, Mask scaling dot product attention calculation based on the Q, K, and V matrices includes: Perform matrix multiplication on the Q matrix and the transposed K matrix to obtain the similarity score matrix; The similarity score matrix is ​​scaled according to the dimensions of the K matrix; Add masks to invalid locations on the scaled similarity score matrix; Perform Softmax normalization on the similarity score matrix after adding the mask to obtain the attention weight matrix; Perform matrix multiplication between the attention weight matrix and the V matrix to obtain the attention output matrix.

6. The polypeptide characterization method according to claim 1, characterized in that, The global output representation is input into a multi-layer Transformer encoding structure, including: Each Transformer coding structure performs residual connection and layer normalization operations on the input data. The layer normalization result is input into the feedforward neural network for nonlinear feature refinement. The output result of the feedforward neural network is then subjected to residual connection and layer normalization operations again to obtain the output result of the Transformer coding structure of this layer. The input data of the first layer Transformer coding structure is the global output representation, and the input data of other layers Transformer coding structures is the output result of the previous layer Transformer coding structure.

7. The polypeptide characterization method according to claim 1, characterized in that, Also includes: Obtain a dataset containing multiple known peptide sequences, and divide the dataset into a training set and a validation set; Obtain the mask sequence label, physicochemical property features, and attention mask vector for each polypeptide sequence in the training set to construct the model training set; The training set is input into the peptide characterization learning model to obtain sequence prediction results; the weighted loss value of the sequence prediction results and the corresponding peptide sequence is calculated, and mean normalization is performed according to the mask sequence label to obtain the final loss value; the model weights of the peptide characterization learning model are updated according to the final loss value; the validation set is input into the updated peptide characterization learning model to calculate the validation set loss value; using the updated peptide characterization learning model, this step is repeated until the preset maximum number of iterations is reached; The optimal model weights are determined by the model weights in the iterations corresponding to the optimal validation set loss value.

8. The polypeptide characterization method according to claim 7, characterized in that, The dataset is divided into a training set and a validation set, including: The dataset contains multiple known polypeptide sequences that are divided into a first part and a second part in a fixed ratio. The first part is used as the training set and the second part is used as the validation set.

9. The polypeptide characterization method according to claim 7, characterized in that, Obtain a dataset containing multiple known peptide sequences, including: Obtain raw sequence data from a publicly available protein sequence database, extract polypeptide sequences whose length is less than a preset limit from the raw sequence data, and sequentially remove invalid sequences and duplicates to obtain the UniProt screening set; Obtain ColabFoldDB sequence resources, extract peptide sequences whose length is less than a preset limit from the ColabFoldDB sequence resources, and sequentially remove invalid sequences and duplicates to obtain the ColabFoldDB screening set; Merge the UniProt filter set and the ColabFoldDB filter set, and perform deduplication on the merged dataset to obtain the original corpus. Site-directed mutagenesis was performed on each amino acid site of the polypeptide sequences in the original corpus to generate mutant data including various amino acid substitutions. Using affinity index as a quantitative screening criterion, invalid mutants were removed from all mutant data to obtain the dataset.

10. The polypeptide characterization method according to claim 7, characterized in that, Also includes: The peptide characterization learning model is fine-tuned according to the preset downstream tasks.

Images