A SMILES representation method and system for endpoint prediction and local editing ordering of non-standard cyclic peptides
By constructing the non-standard cyclic peptide SMILES representation model ncAA-RepDistill, the problem of prediction and ranking of non-standard cyclic peptides in the prior art is solved. It realizes the prediction of endpoint properties and local editing ranking of non-standard cyclic peptides, and improves the model's ability to predict permeability and analyze local editing sensitivity.
Patent Information
- Application Number
- CN202611421826.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-09-14
- Publication Date
- 2026-11-17
AI Technical Summary
Existing technologies struggle to effectively predict and sequence the local chemical properties of non-standard cyclic peptides, and existing models exhibit poor stability in extrapolation predictions across backbones and residue types, failing to achieve universal prediction across the broad chemical space of modified peptides.
A method for constructing a non-standard cyclic peptide SMILES representation model is adopted. Through training with Transformer network and knowledge distillation, combined with self-attention layer, feedforward network layer and chemi-sensory auxiliary supervision module, a SMILES representation model ncAA-RepDistill suitable for non-standard amino acids is constructed.
It enables the prediction of endpoint properties and local editing ranking of non-standard cyclic peptides, improves the model's ability to predict the permeability of non-standard amino acid cyclic peptides, and supports local editing sensitivity analysis.
Smart Images

Figure FT_1 
Figure FT_2 
Figure FT_3
Abstract
Description
Technical Field
[0001] This invention relates to the design and development of non-standard cyclic peptides and small molecule drugs, and more specifically, to a SMILES representation method for endpoint prediction and local editing ordering of non-standard cyclic peptides. Background Technology
[0002] Non-canonical amino acid (ncAA) peptides and macrocyclic peptides occupy a valuable middle ground between small molecule drugs and biologics. Introducing ncAAs can directly alter local chemical properties, expanding the molecular design space. Modifications such as D-residue substitution, N-methylation, halogenation, and backbone reconstruction can optimize molecular conformation, enzymatic stability, and membrane permeability, even increasing oral exposure levels. However, the effects of these modifications are highly context-dependent: the same modification can have vastly different effects on different peptide backbones, and increased permeability is often accompanied by decreased solubility, impacting drug development potential. Therefore, ncAA peptide modeling is a condition-dependent medicinal chemistry problem and cannot be simply equated to sequence design.
[0003] The context-dependent nature of modification effects contradicts existing datasets. CycPeptMPDB and other cyclic peptide databases have expanded the data for permeability studies, but the data distribution is uneven, with insufficient coverage across different backbones, modification types, and physicochemical properties, making model evaluation susceptible to interference from dataset partitioning. Existing research confirms that the extrapolation prediction stability of models across backbones and residue types is poor, making it difficult to achieve universal predictions across the broad chemical space of modified peptides. Furthermore, molecular representation systems also have limitations. ncAA peptides exhibit non-standard linkages, stereochemical changes, and diverse cyclization patterns, making lossless mapping to the natural 20-amino acid alphabet impossible. Residue formats such as HELM facilitate data exchange, but prediction models require atomic-level characterization such as SMILES and molecular graphs; the conversion of modified peptides from peptide sequence formats to standard molecular structures is not a neutral preprocessing, and the conversion process may lose key chemical features. While toolchains such as AAindexNC and pyPept improve conversion fidelity, they fail to eliminate this defect; graph-aware SMILES, as a compromise, can balance atomic linkage information with the scalability of sequence models.
[0004] Existing computational methods can only partially address the aforementioned challenges. For example, protein language models like ESM2 excel at capturing peptide sequence patterns, while small molecule models like ChemBERTa provide atomic-level stereochemical priors. Chemisen peptide-specific models such as PepLand and PeptideCLM are optimized for cyclic peptides and non-standard residue scenarios. Tools like HighFold3 promote ncAA structure prediction and experimental engineering. These studies demonstrate that modified peptides cannot follow the same modeling approach as natural peptides, and there is still a lack of objective methods to evaluate the merits of different characterization schemes. This shortcoming simultaneously restricts molecular prediction and drug design. Even if models can accurately predict the overall properties of a molecule, they struggle to achieve precise sequencing of structurally similar analogs with the same backbone, and the property differences brought about by minor modifications are easily masked by characterization noise.
[0005] In summary, there is an urgent need in this field for a chemically intrinsic SMILES representation method suitable for ncAA, which would enable ncAA to have local analogue ranking capabilities and be used for endpoint property prediction. Summary of the Invention
[0006] The purpose of this invention is to provide a SMILES representation method and system for endpoint prediction and local editing ordering of non-standard cyclic peptides.
[0007] A first aspect of the present invention provides a method for constructing a non-standard cyclic peptide SMILES representation model, the method comprising the steps of: (S1) Provide standard amino acid and non-standard amino acid datasets, optionally with structural diagrams corresponding to peptides in the datasets; the datasets include SMILES representations of peptides; (S2) Preprocess the SMILES representation and the optional structure graph to form the lexicalized SMILES representation and the optional bias matrix, respectively; (S3) Input the lexicalized SMILES representation and the optional bias matrix into a student model, and use the training module to perform knowledge distillation training on the student model; The student model includes a Transformer network; wherein the lexicalized SMILES representation is input into the Transformer network; the Transformer network includes a self-attention layer, a feedforward network layer, and an addition and normalization layer; wherein, in the self-attention layer, rotation position encoding and an optional bias matrix are applied to obtain the self-attention layer output; the self-attention layer output is processed sequentially through a first addition and normalization layer, the feedforward network layer, and a second addition and normalization layer to finally obtain the final layer lexical state, and the first special lexical state is used as a global feature; The training module includes a masked SMILES modeling module, a randomized SMILES consistency module, a dual-teacher projection alignment module, and a chemi-aware auxiliary supervision module; wherein, the masked SMILES modeling module includes a masked language header; the dual-teacher projection alignment module includes a protein teacher model and a molecular teacher model; the chemi-aware auxiliary supervision module includes a descriptor header; the knowledge distillation training includes: The first stage of training, which is conducted on the standard amino acid dataset, involves using the masked SMILES modeling module to impose lexical-level constraints on the final-layer lexical states; and using the randomized SMILES consistency module, the dual-teacher projection alignment module, the chemical perception-assisted supervision module, and molecular space contrastive learning to impose global representation-level constraints on the global features, thereby aligning the student model with the prior knowledge of the standard amino acids. The second stage of training is unlabeled training performed on the non-standard amino acid dataset; wherein, the mask SMILES modeling module is used to perform word-level constraints on the final layer word states; and the randomized SMILES consistency module, the molecular teacher model, and the chemical perception auxiliary supervision module are used to perform global representation-level constraints on the global features. (S4) When the training results reach the predetermined termination condition, the model training is terminated, and the obtained student model is the non-standard cyclic peptide SMILES representation model, namely ncAA-RepDistill.
[0008] In another preferred embodiment, the method further includes: (S5) freezing the downstream transfer and evaluation of the representation, wherein the student model is frozen and a lightweight downstream task head is trained on a labeled dataset to evaluate the transferability of the student model.
[0009] In another preferred embodiment, the standard amino acid dataset includes standard short peptides.
[0010] In another preferred embodiment, the standard amino acid dataset is derived from a standard short peptide dataset obtained by screening UniRef data.
[0011] In another preferred embodiment, the sequence homology of the standard short peptides in the standard amino acid dataset is ≤30%.
[0012] In another preferred embodiment, the standard short peptide refers to a peptide segment with less than 30 amino acid residues and containing only 20 standard amino acids.
[0013] In another preferred embodiment, the SMILES of the standard short peptides in the standard amino acid dataset are represented as residue-level monomers SMILES.
[0014] In another preferred embodiment, the residue-level monomers SMILES are residue-SMILES sequences connected in sequence by "." separators.
[0015] In another preferred embodiment, the non-standard amino acid dataset includes non-standard amino acids or peptide molecules.
[0016] In another preferred embodiment, the non-standard amino acid dataset includes an unlabeled non-standard amino acid dataset and a labeled non-standard amino acid dataset.
[0017] In another preferred embodiment, the non-standard amino acid dataset includes non-standard long peptides.
[0018] In another preferred embodiment, the non-standard amino acid dataset is derived from SATPdb, THPdb, CPPsite, CyclicPepedia, DRAMP, DBAASP, ConoServer, Hemolytik, Hmrbase2, and AHTPDB.
[0019] In another preferred embodiment, the non-standard long peptide refers to a peptide segment in which SMILES represents more than 766 SMILES characters.
[0020] In another preferred embodiment, the non-standard amino acid dataset includes: therapeutic peptides, cell-penetrating peptides, cyclic peptides, antimicrobial peptides, toxic peptides, and hormonal peptides.
[0021] In another preferred embodiment, the structural diagram is an RDKit molecular diagram.
[0022] In another preferred embodiment, if the SMILES representation of the peptide does not exceed the intrinsic single window length, then step (S2) specifically includes the following steps: (S2.1) The SMILES representation of the short peptide is segmented into characters using a predetermined vocabulary; one special word is added to the beginning and end of the word, and the resulting word sequence is padded or truncated to 768 words, of which a maximum of 766 are valid words, thereby obtaining the word-based SMILES representation. (S2.2) Generate the bias matrix corresponding to the peptide segment based on the lexicalized SMILES representation and the structure diagram.
[0023] In another preferred embodiment, the bias matrix for: , , , in, The distance after truncation; This is the maximum truncation value; h represents the shortest path distance between atoms i and j in the RDKit molecular diagram; h is the attention head. This represents the learnable distance embedding corresponding to the h-th attention head; is the graph bias term between lexical positions i and j in the attention head h; N is the lexical sequence length; in the RDKit molecular graph, atoms i and j correspond to lexical positions i and j, and lexical positions not aligned with atoms are retained at index 0.
[0024] In another preferred embodiment, the maximum cutoff value .
[0025] In another preferred embodiment, if the SMILES of the peptide exceeds the intrinsic single window length, then step (S2) specifically includes: The SMILES representation of the long peptide is divided into 256 overlapping character windows with a step size of 128 characters. For each character window, special lexical units are added at the beginning and end, and then independently filled up to 768 lexical units. Finally, the lexicalized SMILES representation is obtained.
[0026] In another preferred embodiment, the Transformer network is a 13-layer Transformer network.
[0027] In another preferred embodiment, the hidden layer size of the Transformer network is 768.
[0028] In another preferred embodiment, the Transformer network has 12 attention heads.
[0029] In another preferred embodiment, the dropout rate of the Transformer network is 0.12.
[0030] In another preferred embodiment, if the peptide segment does not exceed the intrinsic single window length, rotational position encoding is applied in the graph bias self-attention layer of the Transformer network, and the bias matrix is introduced into the attention score to obtain the self-attention layer output containing the bias matrix; finally, global features are obtained.
[0031] In another preferred embodiment, if the peptide segment exceeds the intrinsic single window length, rotational position encoding is applied in the graph bias self-attention layer of the Transformer network to encode each character window separately, thereby obtaining a window-level aggregated representation; based on each window-level aggregated representation, a global feature is obtained using an effective window length weighted average method.
[0032] In another preferred embodiment, the output of the self-attention layer containing the bias matrix is: , in, For self-attention terms; h is the attention head index; , and These are the query, key, and value for the attention head, respectively; Normalization is performed along the bond positions; Let N×N be the bias matrix of this attention head; The number of dimensions for the query / key of this attention head.
[0033] In another preferred embodiment, .
[0034] In another preferred embodiment, the final-level word state is: , Where N is the number of valid input tokens; Let i be the state of the i-th word element.
[0035] In another preferred embodiment, the global feature is: ,in, This represents the state of the first special term in the output of the last layer of the Transformer network.
[0036] In another preferred embodiment, the global features of peptides exceeding the intrinsic single-window length are: , in, This represents the m-th window. It is the effective character length of window m. It is a window-level aggregate representation extracted from window m.
[0037] In another preferred embodiment, the Transformer network also outputs mean-pooled global features.
[0038] In another preferred embodiment, the mean pooling global feature is: , in, Indicates whether the i-th word is valid; Let i be the state of the i-th word element.
[0039] In another preferred embodiment, the loss function of the mask SMILES modeling module is: , in, This is a cross-entropy calculation term; This is the state of the final-level word element; For masking language header; The actual token label at the masked position.
[0040] In another preferred embodiment, using the randomized SMILES consensus module for global representation-level constraints includes: (r1) Generate two chemically equivalent randomized SMILES views based on the residue sequence of a peptide. (r2) Input the two chemically equivalent randomized SMILES views into the same student model to obtain global features respectively. and This allows for the construction of a cosine similarity loss function to impose global representation-level constraints.
[0041] In another preferred embodiment, the loss function of the randomized SMILES consensus module is: , in, This is a cosine similarity calculation term; and Global features are obtained by encoding two randomized SMILES views of the same peptide using the same student model.
[0042] In another preferred embodiment, the loss function for each space in the dual-teacher projection alignment module is: , in, The vector formed by projecting the global features into the corresponding space; To freeze the target output in the corresponding space; The square of the Euclidean distance; Cosine similarity; These are the weighting coefficients.
[0043] In another preferred embodiment, the loss function of each space in the dual-teacher projection alignment module... .
[0044] In another preferred embodiment, the protein teacher model includes ESM2.
[0045] In another preferred embodiment, the dual-teacher projection alignment module converts the global features into a 1280-dimensional protein space vector and aligns it with the 1280-dimensional frozen target output by the protein teacher model.
[0046] In another preferred embodiment, the protein spatial projection of the global features is: , in, This is the protein spatial projection matrix. This refers to the global feature.
[0047] In another preferred embodiment, the distillation loss of the protein space is: , in, The spatial projection of proteins as global features; The target of the frozen protein teacher model output; The square of the Euclidean distance; Cosine similarity; These are the weighting coefficients.
[0048] In another preferred embodiment, the molecular teacher model includes GeminiMol.
[0049] In another preferred embodiment, the molecular teacher model converts the global features into a 2048-dimensional molecular space vector and aligns it with the 2048-dimensional frozen target output by the molecular teacher model.
[0050] In another preferred embodiment, the molecular space projection of the global feature is: , in, The molecular space projection matrix, This refers to the global feature.
[0051] In another preferred embodiment, the loss function of the molecular teacher model is: , in, The molecular space projection of global features; To freeze the target output of the molecular teacher model; The square of the Euclidean distance; Cosine similarity; These are the weighting coefficients.
[0052] In another preferred embodiment, when using the dual-teacher projection alignment module for constraint, the molecular spatial target is used as the anchor point to correct the protein spatial target, thereby obtaining residual features; the residual features are: , in, This is the term used to stop the gradient method calculation; The residual correction term obtained from learning; This is a projection of molecular space. This represents the spatial projection of proteins.
[0053] In another preferred embodiment, the loss function for correcting the protein spatial target using the molecular spatial target as the anchor point is: , in, This is the term for calculating the mean square error; To stop the gradient operator; To freeze the target output of the molecular teacher model; This is a projection of molecular space. This is the residual correction term after the protein spatial projection is mapped to molecular space.
[0054] In another preferred embodiment, the chemical sensing supervision module regresses a proxy target based on the Morgan fingerprint and its derived features according to the global features.
[0055] In another preferred embodiment, the derived features include: dimensionality-reduced features obtained by principal component analysis of the Morgan fingerprint, and molecular descriptors calculated using RDKit.
[0056] In another preferred embodiment, the loss function of the chemical sensing supervision module includes a primary chemical sensing proxy regression loss and an auxiliary chemical sensing proxy regression loss.
[0057] In another preferred embodiment, the loss function of the primary chemical sensing proxy regression loss is: , Where g represents the global feature; The primary chemical agent is the regression target; The regression loss between the surrogate prediction and the corresponding target; For proxy head.
[0058] In another preferred embodiment, the loss function of the auxiliary chemical sensing proxy regression loss is: , in, To assist in mean pooling representation; To assist chemical agents in regressing to the target; The regression loss between the surrogate prediction and the corresponding target; To assist the proxy head.
[0059] In another preferred embodiment, the loss function for the molecular spatial contrast learning is: , in, To compare and learn computational items; This is a projection of molecular space. To freeze the target output of the molecular teacher model; for Temperature coefficient.
[0060] In another preferred embodiment, the temperature coefficient in the loss function of the molecular spatial contrast learning .
[0061] In another preferred embodiment, the loss function for the first stage of training is: , in, Distillation loss for the protein teacher model; Distillation loss for the molecular teacher model; For masked word cross-entropy loss; For molecular spatial contrast learning loss; This is the cosine consistency loss; The primary chemical sensing proxy regression loss; To assist in the regression loss of chemical sensing agents; This is the residual loss; This indicates the weight of each loss.
[0062] In another preferred embodiment, the loss function for the second stage of training is: , in, Distillation loss for the molecular teacher model; For masked word cross-entropy loss; This is the cosine consistency loss; The primary chemical sensing proxy regression loss; To assist in the regression loss of chemical sensing agents; This is the residual loss; This indicates the weight of each loss.
[0063] A second aspect of the present invention provides a prediction system adapted for non-standard cyclic peptides, comprising: (M1) Input module, the input module being configured to input data, the data including a representation of the compound SMILES; (M2) Prediction module, the prediction module being configured to perform the following operations: inputting the SMILES representation of the compound into a non-standard cyclic peptide SMILES representation model to obtain a global representation; inputting the global representation into a lightweight downstream task head to obtain a prediction result; the non-standard cyclic peptide SMILES representation model is constructed using the method described in the first aspect of the present invention; (M3) Output module, which is configured to output the prediction results of the prediction module.
[0064] In another preferred embodiment, the compound comprises a non-standard amino acid cyclic peptide.
[0065] In another preferred embodiment, the compound comprises SMILES, which represents a long peptide of more than 766 SMILES characters.
[0066] A third aspect of the present invention provides a method for predicting permeability using a non-standard cyclic peptide SMILES representation model, the method comprising the steps of: (x1) Provides a compound SMILES; (x2) Input the SMILES representation of the compound into a non-standard cyclic peptide SMILES representation model to obtain a global representation; input the global representation into a lightweight regression head trained on labeled downstream data to obtain a permeability prediction value; The non-standard cyclic peptide SMILES representation model is constructed using the method described in the first aspect of this invention.
[0067] A fourth aspect of the present invention provides a method for local editing sensitivity analysis using a non-standard cyclic peptide SMILES representation model, the method comprising the steps of: (y1) Provide a SMILES representation of a compound, and a SMILES representation of one or more of the same molecules after local editing; (y2) Input the SMILES representation of the compound and all locally edited SMILES representations into a non-standard cyclic peptide SMILES representation model to obtain the corresponding global representation; input the global representation into a lightweight regression head trained with labeled downstream data to obtain the original predicted value and the predicted value corresponding to each local edit; wherein, the non-standard cyclic peptide SMILES representation model is constructed using the method described in the first aspect of the present invention; (y3) Evaluate the difference between the predicted value corresponding to each local edit and the original predicted value, and sort them according to the size of the difference to perform local edit sensitivity analysis.
[0068] A fifth aspect of the present invention provides an electronic device including a processor and a memory, the memory having a plurality of executable instructions, the processor being configured to read the instructions and execute steps in the method as described in any one of the first, third, or fourth aspects of the present invention.
[0069] A sixth aspect of the present invention provides a computer-readable storage medium storing computer-executable instructions, which, when read and executed by a processor, implement the steps of the method as described in any one of the first, third, or fourth aspects of the present invention.
[0070] A seventh aspect of the present invention provides a computer program product comprising computer-executable instructions which, when executed by a processor, implement the steps of the method as described in any one of the first, third, or fourth aspects of the present invention.
[0071] It should be understood that, within the scope of this invention, the above-described technical features of this invention and the technical features specifically described below (such as in the embodiments) can be combined with each other to form new or preferred technical solutions. Due to space limitations, they will not be described in detail here. Attached Figure Description
[0072] Figure 1 The overall architecture and three-stage training workflow of ncAA-RepDistill are shown. In the first stage, residue-ordered SMILES inputs and normalized molecular SMILES inputs in the second / third stage are word-segmented and paired with a graph bias matrix derived from RDKit before being input into a graph-aware Transformer student model. The student model outputs lexical-level features for masking SMILES modeling, as well as the main aggregated global features for supervision during training. During training, this shared representation is aligned with teacher models in both protein and molecular spaces and further constrained by a chemi-aware auxiliary head; in the inference phase, only the student encoder is retained. The first stage focuses on distillation of the standard peptides by two teachers, the second stage removes the protein teacher path and continues ncAA adaptation under molecular and chemi-aware supervision, and the third stage freezes the encoder for downstream evaluation.
[0073] Figure 2 This shows a comparison of the matching between the Phase1 model and the Phase2-adapted model on the ncaa_cpp dataset. Light-colored lines connect archived checkpoint pairs evaluated under the same downstream protocol, and box plots summarize the distribution among the pairs.
[0074] Figure 3 The structured first-stage ablation experiments for the five major mission benchmarks are shown in Figure A. R-values based on the primary endpoint metric ncaa_cpp are also presented. 2 The values are sorted among the 11 variants shown in the figure. Figure B summarizes the comparison of the delta matrix of each task with the full model on tasks such as ncaa_cpp, aa_binding, ncaa_binding, aa_cpp, and aa_sol. This ablation analysis panel includes perturbations such as architecture, training regularization, descriptor proxies, teacher model removal, and weight scheduling. Weight triples are reported in the order of "protein / molecule / MLM".
[0075] Figure 4This displays the ncAA-Scan benchmark test for single-edit priority sorting on the retain set ncaa_cpp. The x-axis shows the test set R. 2 The y-axis displays the mean of the scaffold-level Spearman correlation coefficients (Δŷ, Δy), and the bubble size represents E@10. Data points are categorized into representative ncAA-RepDistill models, descriptor baseline models, and external molecular encoders. Candidate editing sites were defined from the top three Bemis-Murcko scaffold series under the “full-edge” protocol and evaluated at K=10. All presented models were re-evaluated using the same single-hidden-layer multilayer perceptron (MLP) readout function.
[0076] Figure 5 This figure shows external validation of the structured matrix 36 hemipeptide macrocyclic permeability dataset under a unified “frozen features” protocol. This cohort is a matched series of medicinal chemistry macrocycles covering peptide-to-peptide-like compound substitutions on the peptide side and methylation states on the linker side. All ncAA-RepDistill values in this figure are derived from the same Phase2 frozen checkpoint and the same initial summary global representation across all endpoints and retention schemes. The top row reports the overall Spearman correlation coefficient, overall pairwise accuracy, and overall Top-1 accuracy; the middle row reports the panel summary results for Caco-2, PAMPA, and excluding efflux effects; the bottom row shows the stratified means for Caco-2, PAMPA, and efflux effects under the peptide_state and tether_state retention strategies. Hollow circles represent the “peptide_state” retention set, and solid circles represent the “tether_state” retention set. Error bars represent the bootstrap 95% confidence intervals. Detailed Implementation
[0077] Through extensive and in-depth research, the inventors have developed for the first time a graph-aware SMILES representation model, ncAA-RepDistill, applicable to non-standard amino acid cyclic peptides. Specifically, this model receives SMILES molecular representations and transforms them into lexicalized SMILES representations. After pairing the lexicalized SMILES with paired graph bias matrices, the model is input into a graph-aware Transformer student model, ultimately outputting lexical-level features and aggregated global features. In the construction and training of the ncAA-RepDistill model, lexical-level features are used for self-supervised constraints, and aggregated global features are aligned with protein space teacher models and molecular space teacher models, further constrained using a chemi-aware auxiliary head. Specifically, in the first training stage, training is performed using standard amino acids as the corpus; in the second training stage, training is performed using non-standard amino acids as the corpus, and the protein space teacher model is removed, with adaptive training conducted using the molecular space teacher model and the chemi-aware auxiliary head; in the third training stage, the encoder parameters in the model are frozen, and downstream evaluation training is performed, ultimately obtaining the ncAA-RepDistill model. Based on the ncAA-RepDistill model, a targeted evaluation framework was established to examine whether a deployable, chemically intrinsic SMILES representation can simultaneously support permeability prediction and local editing sensitivity analysis of non-standard cyclic peptides. This invention is based on this framework.
[0078] It should be understood that the specific methods and experimental conditions of the invention described below in varying degrees of detail are intended to provide a substantive understanding of the invention. Definitions of certain terms used in this specification are provided below. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.
[0079] the term As used herein, the terms “containing” or “including (comprise)” can be open-ended, semi-closed, or closed-ended. In other words, the terms also include “consistently made of” or “made of”.
[0080] As used herein, the term “and / or” refers to and covers any and all possible combinations of one or more of the related listed items.
[0081] As used in this article, the term "significant" means that, in a hypothesis test, the observed effect (such as the difference between the experimental and control groups) is unlikely to be caused solely by random error. A hypothesis test includes: the null hypothesis (H0), which assumes that the observed effect does not exist (such as no difference between the experimental and control groups); the p-value, which is the probability of observing the current or more extreme effect when H0 is true; and the significance threshold (α). The significance threshold is typically used to determine whether a hypothesis test is significant. Generally, the significance threshold is 0.05. If the p-value ≤ α, then H0 is rejected, meaning the observed effect exists, and the result is called "significant."
[0082] As used in this article, the term "distillation" refers to knowledge distillation, a model compression technique. Its core idea is to transfer the "knowledge" contained in a large, complex model (called the "teacher model") to a smaller, more efficient model (called the "student model"), so that the student model can significantly reduce computation and storage costs while maintaining high performance.
[0083] As used in this paper, the terms "student model" and "student" are used interchangeably. It refers to a lightweight model with a small number of parameters and a simple structure in knowledge distillation.
[0084] As used in this paper, the terms "teacher model," "teacher encoder," and "teacher" are used interchangeably. The latter refers to a complex model with a large number of parameters and excellent performance in knowledge distillation.
[0085] As used herein, the term "intrinsic single-window length" refers to the maximum length of the input sequence that the student model can process in a single forward propagation. In some embodiments of the invention, the intrinsic single-window length is 768 tokens.
[0086] As used in this article, the terms "Rotated Position Encoding" and "RoPE" are used interchangeably. RoPE is a positional encoding method that directly incorporates positional information into token vectors using a rotation matrix. This method rotates the Query and Key vectors generated from the token, thus naturally introducing relative positional information into attention calculations.
[0087] As used herein, the terms “graph-aware self-attention” and “graph bias self-attention layer” are used interchangeably, referring to the transformation of the bias matrix in the Transformer network of this invention, so that it can be aligned with the lexicalized SMILES representation after rotational position encoding to form global features.
[0088] As used herein, the terms "aggregated global feature," "aggregated global representation," "aggregated feature," "global representation," and "global feature" are used interchangeably. It refers to the 768-dimensional state at the first specific lexical position in the output of the last Transformer layer of the student model of this invention, represented as... .
[0089] In paired comparisons, for reports categorized by task, the primary metric for task t is "the higher the value, the better". The signified improvement is defined as: Among them, the regression task uses As a primary directional summary indicator, the categorized tasks are used As the primary directional summary indicator, indicators such as RMSE or MAE, which are considered "lower is better," are reported separately as absolute values. This matching comparison is interpreted on a task-by-task basis, rather than being combined into a single overall average.
[0090] The computer system is equipped with at least one processor and a memory. The processor invokes a sequence of computer-executable instructions stored in the memory to implement the evaluation process defined in the claims. Although the flowchart describes the operation steps in a specific logical order, in actual execution, the steps may be processed in parallel, their order adjusted, or partially omitted in some cases. As long as such adjustments do not deviate from the core features of the technical solution described in the claims and achieve the same technical effect, they all fall within the scope of protection of this invention. This flexibility in execution order is determined by the programmable nature of computer instructions.
[0091] ncAA and ncAA cyclic peptide As used herein, the terms "ncAA" and "non-standard amino acids" are used interchangeably, referring to all amino acids not belonging to the standard 20 protein amino acids (i.e., amino acids directly encoded by the universal genetic codon). Non-standard amino acids can be divided into two categories: naturally occurring non-standard amino acids and artificially synthesized non-standard amino acids. ncAA cyclic peptides are cyclic peptide compounds whose molecular structure contains at least one non-standard amino acid residue. Cyclic peptides are a class of compounds formed by amino acids linked by peptide bonds to form a cyclic structure (distinguished from linear peptides). Their cyclic backbone and the special modifications carried by non-standard amino acids endow them with higher conformational stability and stronger enzymatic resistance. Modifications such as D-residue substitution, N-methylation, halogenation, and backbone reconstruction can optimize molecular conformation, enzymatic stability, and membrane permeability, and even increase oral exposure levels. Therefore, ncAA and ncAA have promising applications in the field of drug development.
[0092] ncAA-RepDistill As used in this paper, the term "ncAA-RepDistill" is a graph-aware SMILES representation model that distills complementary protein and molecular space prior information into a deployable Transformer student model. This model combines dual-teacher alignment, masked SMILES modeling, randomized SMILES consistency, and a phased training strategy consisting of pre-training with standard peptides followed by adaptive training with unlabeled ncAAs.
[0093] Specifically, during training, the input student model extracts complementary prior information from both the protein teacher model and the molecular teacher model; during inference, the model remains a single encoder processing only SMILES. This study further separates the standard peptide pre-training from the label-free ncAA adaptation process, subsequently testing the frozen representation on publicly available downstream tasks, including scaffold-constrained local editing sequencing based on ncAA-Scan, external validation in medicinal chemistry, and an external ncAA-sensing antimicrobial agent migration task. In this way, this study establishes a targeted evaluation framework to examine whether a deployable, chemically intrinsic SMILES representation can simultaneously support permeability prediction and local editing sensitivity analysis of non-standard cyclic peptides.
[0094] Under a unified “frozen signature” evaluation protocol, ncAA-RepDistill performed best in major non-standard cyclic peptide permeability benchmarks, maintained backbone-constrained local edit ordering via ncAA-Scan, and retained useful migration capabilities in external medicinal chemistry validation and external ncAA-sensing antimicrobial benchmarks. In summary, ncAA-RepDistill is a practical and chemically intrinsic SMILES representation suitable for non-standard amino acid cyclic peptides, particularly in scenarios where endpoint prediction and local edit sensitivity depend on preserving local chemical edits while retaining peptide context.
[0095] In practical applications, the frozen ncAA-RepDistill after training can be used for permeability prediction and local editing sensitivity analysis. The permeability prediction process is as follows: the normalized molecular SMILES are encoded to obtain a global representation g, which is then input into a lightweight regression head trained with labeled data to obtain the predicted value. Local edit sensitivity analysis involves generating single-step edit candidates within the same scaffold, calculating the predicted values before and after editing, and then...
[0096] The system of the present invention This invention provides a system. The system can be embodied as an electronic device, including but not limited to: smartphones, tablets, personal computers, servers, terminals, or other intelligent terminals with data processing capabilities. The system can also be embodied as a computer-readable storage medium, such as a disk, optical disk, solid-state drive, read-only memory, or flash memory, on which a computer program is stored. When the program is executed by one or more processors, it can implement the data processing flow. Furthermore, the system can also be embodied as a computer program product containing computer instructions that, when executed by a computer, cause the computer to perform all or part of the steps of the data processing flow.
[0097] The data processing flow can be loaded, deployed, or run on any of the aforementioned electronic devices. Through the collaboration of hardware resources and the software logic defined in the flow, the system can complete specific data acquisition, transmission, calculation, analysis, storage, or presentation tasks.
[0098] The main advantages of this invention include: (1) This invention is the first to develop a graph-aware SMILES representation model, ncAA-RepDistill, which can be used for endpoint prediction and permeability assessment of non-standard amino acid cyclic peptides. Under the “frozen characteristics” assessment protocol, ncAA-RepDistill performs best in major non-standard cyclic peptide permeability benchmark tests and has good transferability in external validation of medicinal chemistry and external ncAA-aware antimicrobial benchmark tests.
[0099] (2) When constructing ncAA-RepDistill, three training stages are adopted: standard peptide pre-training, label-free ncAA adaptation training, and frozen feature downstream transfer. For the same model configuration, the performance of the model can be significantly improved by introducing a second stage training that is paired with the first stage training.
[0100] (3) The ncAA-RepDistill of the present invention adopts a dual-teacher model during training, which can significantly improve the transferability of the model.
[0101] (4) The ncAA-RepDistill of the present invention can support local editing priority ranking and is used to evaluate the passive permeability, cell permeability and efflux-related behavior of drugs.
[0102] (5) The ncAA-RepDistill of the present invention can handle ultra-long inputs based on long molecules and can be applied to long input pressure analysis scenarios.
[0103] The present invention will be further illustrated below with reference to specific embodiments. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. It should be understood that these embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. It is also understood that the purpose of describing the present invention in conjunction with the embodiments is to cover other options or modifications that may be derived based on the claims of the present invention. To provide a deep understanding of the invention, many specific details will be included in the following description. The invention may also be practiced without using these details. Furthermore, to avoid confusion or obscuring the focus of the invention, some specific details will be omitted in the description.
[0104] Assessment methods and tasks: 1. Evaluation Tasks: All downstream analyses employed five benchmark tasks derived from the PepLand task set: non-standard cyclic peptide permeability (ncaa_cpp), standard peptide binding (aa_binding), non-standard peptide binding (ncaa_binding), standard cell-penetrating peptide classification (aa_cpp), and standard peptide solubility (aa_sol). In this study, these tasks were reprocessed under a unified evaluation workflow: task-related duplicates were removed before splitting, training / validation / test partitions considering scaffold structure were constructed for controlled frozen feature comparisons, and the resulting benchmark files were consistently applied to all compatible encoders listed in Table 1. In this study, ncaa_cpp was considered the primary benchmark task because it most directly examines ncAA permeability within the chemical intrinsic molecular domain. ncaa_binding, as a secondary ncAA sensitivity task, was retained, but its sample size is significantly smaller, and the corresponding estimation results are less stable. These three standard amino acid tasks were primarily used as controls to compare the representations with settings that emphasize sequences more; among them, aa_sol also served as a long input stress test case for this encoder. The downstream benchmark tasks used in this study are shown in Table 1.
[0105] Table 1. Downstream Benchmark Tasks 2. Evaluation Metrics and Comparative Analysis: The evaluation includes a broad cross-model benchmark covering five tasks, as well as several controlled analyses. The broad benchmark compares model families against five downstream endpoints, while the limited-to-phase-1 and phase-2 adaptive model matching, phase-1 ablation experiments, ncAA-Scan, external validation of medicinal chemistry, and XuAMP validation more directly explore issues such as ncAA adaptability, characterization mechanisms, editing priority ranking, and external migration.
[0106] For a comprehensive benchmark test covering five tasks, the evaluation metric for the regression task is R0. 2In summary, evaluation metrics for classification tasks are summarized using AUROC. Unless otherwise stated, all compatible encoders were re-evaluated from scratch under the same partitioning protocol that has had deduplication data removed and scaffold structure considered, and under the same downstream lightweight single-hidden-layer multilayer perceptron (MLP) readout mechanism. Therefore, this benchmark should be viewed as a uniform “frozen feature” comparison rather than a paired significance analysis. Sequence-only protein language models were evaluated only on standard amino acid tasks, while ncAA tasks were benchmarked using only models capable of handling chemically intrinsic molecular inputs. Furthermore, for tasks containing a large number of molecules with standard SMILES lengths exceeding the intrinsic single-window load capacity, frozen feature extraction may also depend on long-input approximation algorithms for ultra-long inputs.
[0107] The standard downstream metric follows its conventional definition: the coefficient of determination (R²) in regression analysis. 2 The parameters include: root mean square error (RMSE), mean absolute error (MAE), and Spearman correlation coefficient; area under the receiver operating characteristic curve (AUROC), area under the precision-recall curve (AUPRC), Matthews correlation coefficient (MCC), F1 score, accuracy, sensitivity (SN), and specificity (SP).
[0108] Unless otherwise stated, the bootstrap confidence intervals are percentile intervals obtained by resampling n=1000 test instances (seed number=42), and the results of multi-subcontrol analysis are summarized as the test set mean ± standard deviation.
[0109] 3. Comparison of Single Edit Priority Ranking Based on ncAA-Scan: ncAA-Scan was designed as a constrained use case benchmark to test whether the model can prioritize experimentally observed single edit variants within a skeleton family. Based on a fixed ncaa_cpp test dataset, molecules were grouped by the Bemis-Murcko skeleton (murcko14), and the three largest skeleton families were retained because they contain enough observed local edit relationships to support a stable skeleton-level summary.
[0110] Within each retained scaffold series, two molecules are connected as candidate undirected edges only if the two segmented circular sequences are of equal length and can be aligned to a minimum segmentation-level Hamming distance of 1 under circular rotation (optionally inverted). Therefore, this benchmark uses only one-step edits observed in the retained test set, rather than a replacement of the enumeration hypothesis. In the main analysis, the complete "all edges" candidate set was used for each retained scaffold sequence. Subsequently, each undirected edge was expanded into two directed edits: A→B and B→A, thus symmetrically characterizing beneficial and harmful local changes. For each directed edit e, its predicted and observed edit effects are defined as follows: ,in, This indicates the predictive effect of local editing. The corresponding observed effects are represented. Three complementary summary measures are reported. First, the global test R-value from the corresponding ncaa_cpp prediction model is retained. 2 Values. Secondly, by calculating the value within each retained stent sequence. and The Spearman correlation coefficient between the three scaffold sequences was used, and the arithmetic mean of the three scaffold sequences was taken to quantify the local ranking fidelity. Third, the top K hit rate and enrichment were used to quantify budget priority ranking.
[0111] For decision-oriented summaries, only consider And it's a non-drawing directed edit. Let's assume... Let K represent the K directed edits with the largest predicted increments. The hit rate of the top K edits is defined as the edits that truly improved the endpoint (i.e., ...) among these recommended edits. The ratio of the observed hits to the random level. To calibrate this metric to the random level, a matched random baseline was generated by performing 1000 random samplings from the same candidate pool using the same effective K value. Subsequently, enrichment@K was defined as the observed hits@K divided by the mean of the random hits@K; this metric is abbreviated as E@K in the text, such as E@10.
[0112] To ensure that the comparison results reflect the quality of representation rather than the heterogeneity of downstream regression variables, all baselines were re-evaluated on the same fixed partition, using the same training partition normalization, the same single-hidden-layer lightweight multilayer perceptron (MLP) readout method, and the same K = 10 evaluation budget.
[0113] 4. External Validation for the Permeability Dataset of Semipeptide Macrocyclic Compounds in Medicinal Chemistry: To conduct external validation in the field of medicinal chemistry, a structured 4×9 matrix of semipeptide macrocyclic compounds reported by Comeau et al. was selected. This matrix contains 36 closely related macrocyclic compounds arranged along two design axes. In the original series design, "peptide_state" corresponds to peptide-side modifications within the macrocycle, primarily peptide-to-peptide-like compound substitution patterns; while "tether_state" corresponds to the methylation state and methyl position on non-peptide linkers. For the main analysis in this paper, all external metrics were calculated solely based on this matrix dataset. PAMPA, Caco-2 A→B, and Caco-2 B→A labels were normalized on a log10 scale, and efflux rates were calculated from paired oriented Caco-2 readings.
[0114] This study employed two retention schemes that considered series characteristics. In the `peptide_state_holdout` scheme, one set of peptide states was retained at a time, resulting in four folds, with n_test=9 for each fold. In the `tether_state_holdout` scheme, one set of anchor states was retained at a time, resulting in nine folds, with n_test=4 for each fold. The encoder remained frozen throughout the external evaluation process. For each fold, the embedding vector was first normalized using a StandardScaler fitted only to the training partition, followed by analysis using a Ridge regressor with alpha=100. No encoder fine-tuning or SMILES randomization was performed in the external evaluation environment.
[0115] The main report focuses on ranking-based summaries. Spearman correlation coefficients were used to quantify the consistency between predicted and observed rankings within each retention unit; pairwise accuracy quantified the proportion of correctly ranked sample pairs; and Top-1 and Top-3 recorded whether the best-performing compound in the test fold was restored to the first position or ranked among the top three predicted candidates. Because the two retention schemes contributed different numbers of folds, this study did not directly average all original folds. Instead, each endpoint was first summarized under "peptide_state_holdout" and "tether_state_holdout," and then these summaries were averaged with equal weights. This ensured that the final overall metric reflected both retention views in a balanced way, rather than being dominated by the scheme with more folds. According to this definition, "Overall Spearman" averages all endpoint-level summaries, while "Caco-2 Panel Spearman" and "PAMPA Panel Spearman" average only the corresponding specific subsets of detections.
[0116] 5. Transfer Evaluation of XuAMP-based Independent ncAA Benchmarks: The XuAMP evaluation follows the independent ncAA benchmarks published in the AmpHGT22 study. The published joint training dataset is used, and transfer evaluation is performed on its retained ncAA test partitions. The Phase2-adapted model remains frozen, and its main aggregated global representations are normalized and passed to the same single-hidden-layer lightweight classifier used in the evaluation of other matched frozen features. Phase2 does not use any XuAMP labels, and overlap audits confirm that there is neither exact structural overlap nor exact sequence overlap between the Phase2 corpus and the XuAMP test partitions.
[0117] In this analysis, the encoder itself was not fine-tuned. Instead, the same primary aggregate global representation used throughout this paper was extracted, and a predefined long input multi-window path was applied when single-window SMILES encoding was not applicable.
[0118] The published ESM2 and AmpHGT values are taken from data in the original AmpHGT paper using the same publication partition. Furthermore, the frozen PepLand embedding vectors are evaluated using the same lightweight downstream head as ncAA-RepDistill, thus external comparisons are performed to achieve a migration test that is as close as possible to the matching frozen representation. A complete list of hyperparameters and seeds is provided in the supplementary information.
[0119] Example 1: Model Architecture and Research Overview like Figure 1As shown, ncAA-RepDistill is built around a deployable, graph-supporting SMILES encoder. Teacher models for protein and molecular spaces, along with auxiliary objective functions supporting chemical structures, are used only during the training phase to shape this shared representation and are not part of the inference interface.
[0120] The framework comprises four functional layers: tokenized SMILES input paired with an RDKit-based graph bias matrix; a Transformer student model that generates token-level states and a primary aggregated global representation; a training-only branch for masked SMILES modeling, dual-teacher alignment, and chemically assisted supervision; and a phased scheduling mechanism to determine when each supervisory path is activated. Phase 1 uses residue-ordered SMILES strings derived from standard peptides, while Phase 2 and all downstream evaluations use normalized molecular SMILES. A key point later in this paper is that all downstream evaluations are performed on the same deployment object—this single, pure SMILES student representation.
[0121] (A) Student model architecture (ncAA-RepDistill): ncAA-RepDistill is configured as a deployable SMILES encoder with dedicated inputs for each stage. The complete system includes input preprocessing, a Transformer backbone supporting a graph structure, a supervised branch used only for training, and a three-stage training plan. During the inference stage, the deployed model receives normalized molecular SMILES sequences and outputs a primary aggregated global representation for downstream transfer; teacher alignment and chemi-aware auxiliary objectives are used only for training.
[0122] The same word segmenter and Transformer backbone network are used in each stage, but the specific SMILES representation varies depending on the data scope. Phase 1 uses residue-SMILES sequences, where standard residues are represented as monomeric SMILES assembled in sequence. Phase 2 and all downstream analyses use SMILES of the normalized isoforms of the entire peptide molecule. This design allows the model to first learn broad peptide sequence regularities and then adapt to chemically realistic ncAA molecule inputs without introducing a second-deployed encoder.
[0123] (A1) Input Processing: The student model segments the input SMILES using a fixed vocabulary of 37 words. After considering the first and last special words, each sequence is padded or truncated to 768 words, which corresponds to 766 effective intrinsic single-window payloads of SMILES characters.
[0124] In the intrinsic single-window path, the downstream attention mechanism receives a shortest path distance bias derived from RDKit, which is in max = 8 points were truncated, therefore explicit graph items are used as local topological priors, rather than reconstructing the full graph topology in an arbitrary range. This bias is generated by the corresponding input object used at each stage. As long as the input meets the intrinsic length constraint, this single-window, graph-biased path is the preferred encoding path.
[0125] Specifically, to incorporate local topological structures into linearized SMILES, the shortest path distance between lexical-aligned atoms is first calculated on the corresponding RDKit graph and then truncated to... max = 8, and convert it to a bias term specific to the head node: , in, Indicates the distance after truncation; Indicates the maximum truncation value; h represents the shortest path distance between atoms i and j in the RDKit molecular diagram; h represents the attention head. This represents the learnable distance embedding corresponding to the h-th attention head; This represents the graph distance offset between word positions i and j in the attention head.
[0126] In the linearized SMILES sequence, there is a correspondence between each term and an atom in the RDKit graph. For a term that corresponds to an atom, its atom index in the molecular graph is determined based on this correspondence, and then the shortest path distance is calculated. For terms that do not correspond to an atom or are filler terms, their corresponding positions are set to predetermined values or masked. In this invention, special terms or filler terms that are not aligned with atoms use a reserved index of 0.
[0127] The bias matrix is expressed as: , Where N is the length of the current word sequence, and each attention head h has a corresponding distance embedding; Add it to the attention score of this attention head before softmax.
[0128] For Phase 2 / Phase 3 molecules where the standard smiles exceed the intrinsic single-window load, a pragmatic long-input approximation method is used to extract frozen features. Each sequence is divided into overlapping 256-character windows with a stride of 128 characters. Each window is wrapped with special markers before and after, independently padded to a length of 768 characters, encoded separately, and converted into a window-level aggregated representation. The derived features are then obtained using an "effective window length weighted average" method. Since these window substrings cannot be guaranteed to be effective independent smiles, explicit graph distance biases are not reconstructed in this long-input mode.
[0129] (A2) Backbone Network: The backbone network is a 13-layer Transformer with a hidden layer size of 768, 12 attention heads, and a dropout rate of 0.12. Rotation Position Encoding (RoPE) is applied to Q and K during self-attention computation. Specifically, word embeddings are obtained first, and then rotation transformations are applied to Q and K during each layer of self-attention computation to encode relative positional information. During training, random SMILES enumeration is performed with a 30% mixing rate, generating two enumeration variants for each molecule. Masked SMILES modeling provides word-level self-supervised learning.
[0130] Image-aware self-attention in head h can be represented as: , in, Q represents the output of the h-th attention head; h represents the index of the attention head; Q (h) K (h) and V (h) These represent the query, key, and value of the attention header, respectively; Indicates softmax activation, which is normalized along the bond positions; B (h) This represents the N×N graph bias term of the attention head; This indicates the key / query dimension for each head, which is 64 dimensions in this implementation.
[0131] The final-level lexical state H can be obtained through the backbone network: , where N represents the number of valid input tokens.
[0132] The main global features are defined by the state of the first special lexical unit: This refers to the 768-dimensional state at the first special term position in the output of the last Transformer layer.
[0133] In addition, the model also retains an auxiliary mean-pooled global aggregation representation: , in, Indicates whether the term i is valid. Unless otherwise stated, downstream migration and external validation use the primary pooling representation g, denoted as... , rather than an auxiliary mean pooling aggregation representation.
[0134] For very long inputs, the long input approximation uses a 256-character window and a 128-character stride; the output is a weighted average of the window-level aggregated representation, not a reconstructed word sequence, and the path does not reconstruct the explicit graph distance bias. Specifically, the derived representation is approximated by a weighted aggregation of overlapping windows, i.e., a weighted average of the effective window length: , in, This represents the m-th window; It is its effective character length, excluding the number of effective characters excluding special words at the beginning and end and padding positions; It is an aggregate representation extracted from the window.
[0135] Therefore, tasks involving a high proportion of very long molecules should be understood as a comprehensive evaluation of the learned representations and the approximate path of the long input, rather than solely based on intrinsic single-window graph perceptual encoders. This consideration is particularly important when interpreting auxiliary long sequence tasks such as aa_sol.
[0136] In Phase 1, the main aggregated features are projected onto a 1280-dimensional protein spatial target and a 2048-dimensional molecular spatial target for teacher model alignment: , in, This represents the spatial weighting term of the protein. This represents the molecular space weighting term.
[0137] To use the chemical space as an anchor point while allowing for correction in the protein space, the features during coupled training are represented in residual form as follows: , in, This indicates that the gradient method has stopped. This is the learned residual correction term. This fusion quantity is only used to couple the teacher objective during training and is not derived for downstream inference or external validation.
[0138] (B) Teacher encoder for knowledge distillation: The use of the teacher model varies by stage. In the first stage, the student model is aligned to both the protein teacher distillation and the molecular teacher distillation, while in the second stage, only the molecular teacher distillation is retained. This separation is because, while the protein teacher has informational value in classical peptide domains, it becomes unsuitable once the adaptation process involves ncAA chemistry, requiring a forced and lossy mapping to a non-standard 20-letter alphabet. There is no teacher branch in the inference stage. The disclosed molecular encoders in Example 2 are baseline models, not teacher models, unless their embedding space is directly used to supervise the student model during training.
[0139] The protein teacher model used was ESM2. A variant with 650 million parameters was employed to extract the hidden states of layer 30, which were then transformed into 1280-dimensional protein spatial targets using mean pooling. This frozen branch was used only in the first stage to provide biologically aligned targets for standard peptides.
[0140] The molecular teacher model, GeminiMol, is used as the frozen molecular encoder in the final model configuration. This model provides a 2048-dimensional molecular space target for the alignment task of the student models. Since this branch deals with intrinsic chemical molecular structures rather than residue alphabets, it is applicable in both the first and second stages.
[0141] In each target space, a mixed Euclidean-cosine objective function is used to train the student projection u, which is then aligned with and frozen to the teacher objective t. , Where u represents the vector of student global features projected onto the teacher space, and for the protein teacher model and the molecular teacher model, it can be taken as follows: or ;t indicates the corresponding frozen teacher target or , The square of the Euclidean distance; Cosine similarity; These are the weighting coefficients for the Euclidean distance and cosine distance terms. This objective function is applied to targets in protein space and molecular space, respectively. Preferably, .
[0142] (C) Three-stage training: These three training phases further differentiate between broad peptide regularity learning and ncAA-specific learning. Phase 1 aligns the student model with prior knowledge of standard peptides in both protein and molecular spaces. Phase 2 removes the protein teacher path and continues adaptive training for unlabeled ncAAs under supervision in both molecular and chemiluminescent spaces. Phase 3 freezes the learned model and tests whether this single representation can be transferred through a lightweight downstream head.
[0143] (1) Phase1 Standard Amino Acid Pre-training Corpus: Based on standard short peptide general pre-training, it adopts protein spatial distillation, molecular spatial distillation, SMILES mask language modeling (MLM), randomized SMILES consistency and chemical perception-assisted supervision.
[0144] Specifically, Phase 1 established extensive peptide prior knowledge for subsequent ncAA adaptation. This corpus was constructed based on short peptide sequences derived from UniRef. Only entries with fewer than 30 residues were retained, and the alphabet was limited to the standard 20 amino acids. Near-homology redundancy was reduced by using MMseqs2 clustering at a 30% sequence homology level; here, c30 is the MMseqs2 redundancy threshold set in this study, not the official UniRef classification. Subsequently, a 9:1 training / validation set split was adopted. The final Phase 1 corpus contains 219,150 training sequences and 24,350 validation sequences.
[0145] At this stage, the SMILES field processed by the student model is not a fully covalently linked complete peptide SMILES. Instead, each standard residue is converted into residue-level monomeric SMILES, which are linked sequentially by "." delimiters to form a residue-SMILES sequence. This representation preserves both residue order and local chemical identity while remaining compatible with the same tokenizer and Transformer backbone used subsequently. The original amino acid sequences are preserved in parallel for protein teacher alignment. In this corpus, all generated inputs are within the intrinsic single-window length limit; therefore, a long input approximation algorithm was not used in the first stage.
[0146] In the randomized SMILES consistency assessment, the residue sequence SMILES corresponding to the standard short peptide is input. Two chemically equivalent randomized SMILES views are generated for the same molecule. These views are then segmented and input into the same student encoder to obtain the global representation. and Then calculate the cosine consistency loss. The randomization SMILES mixture rate for the first stage is 0.3, and is added to the total loss with corresponding weights.
[0147] (2) Phase 2 Non-standard amino acid adaptation corpus: Unlabeled ncAA adaptation training is performed on a multi-source non-standard corpus, protein teacher branches are disabled, while molecular spatial supervision is retained.
[0148] Specifically, Phase2 adapts the model to chemical properties associated with non-standard amino acids using unlabeled in-domain molecular data. The Phase2 unlabeled non-standard amino acid adaptation corpus contains 7716 molecules, with 5133 from the downstream ncaa_cpp training partition and 2583 non-standard amino acids or peptides collected from publicly available resources. These resources cover therapeutic, cell-penetrating, cyclic, antibacterial, toxic, and hormonal peptides, including SATPdb, THPdb, CPPsite, CyclicPepedia, DRAMP, DBAASP, ConoServer, Hemolytik, Hmrbase2, and AHTPDB. This design was chosen because ncAA data remains limited, and Phase2 aims to perform in-domain chemical adaptation before freezing downstream evaluation, rather than building a pre-training corpus completely exclusive to the benchmark dataset.
[0149] All entries were standardized after structure transformation, derived into SMILES representations of isomer standard molecules, and deduplication was performed at the standardized structure level; mapping failures were explicitly logged. In the final training configuration used in this study, no downstream endpoint labels were used in the second stage, including any permeability values that might exist in the source table. Therefore, molecules inherited from the downstream ncaa_cpp training partition entered the second stage only as unlabeled intra-domain chemical examples. The optimization process was driven by molecular spatial distillation, masked SMILES modeling, randomized SMILES consistency, and a structure-derived surrogate head based on RDKit descriptors and fingerprints. A 10% internal validation set (seed=42) was reserved from the adaptation pool, used only for monitoring and model selection in the second stage.
[0150] Since the objective of the second stage is unlabeled for the downstream task, this stage should be considered as unlabeled intra-domain adaptation rather than task-specific supervised pre-training. Neither the validation nor the test sets in the downstream benchmarks are included in the backpropagation and model selection process of the second stage.
[0151] During training, the chemi-aware agent heads also regress the RDKit descriptor and the target derived from the Morgan formula from the initial aggregate representation. These auxiliary heads regularize the main chain during optimization but are removed during inference.
[0152] Phase 1 uses the full objective function, while Phase 2 sets the protein distillation weights to 0.0 and continues adaptation through molecular spatial distillation, masked SMILES modeling, randomized SMILES consistency, and chemi-aware assisted supervision. In the final training pipeline used in this study, no downstream endpoint labels contribute to the loss in Phase 2. Subsequently, the frozen student representations are transferred to the downstream head in Phase 3.
[0153] The stage-specific functions can be summarized as follows: , , , in, The distillation loss of the protein teacher model is specifically expressed as: ; The distillation loss of the molecular teacher model is specifically expressed as: ; It is the masked word cross-entropy, specifically represented as: ; It is the cosine consistency loss of two randomized SMILES global representations, specifically expressed as: ; This is the first stage of molecular space InfoNCE, specifically represented as: ; and These are the primary and secondary descriptor losses for chemical sensing, specifically expressed as follows: , ; It is the residual fusion MSE used only for training, specifically expressed as: , in, It is the main aggregation feature. and It is the global representation obtained by passing two randomized SMILES inputs of the same molecule through the same encoder. It is a student's molecular space projection. Indicates the lexical state of the last layer. The cross-entropy is represented by the symbol, and the mask is the position to be masked. yes temperature coefficient, It is a frozen molecular teacher target; and These are the primary and secondary chemical proxy regression targets, respectively. This represents the regression loss between the surrogate prediction and the corresponding target. and It is a proxy head; It is a student's molecular space projection; This represents the residual correction term after the protein spatial projection is mapped to molecular space. Indicates stopping the gradient method; This indicates the weight of each loss.
[0154] This phase transition preserves the biological prior information obtained in the first phase while avoiding lossy remapping after the adaptation process enters the ncAA chemical reaction.
[0155] (3) Phase 3 Feature Freeze Downstream Transfer: The downstream evaluation is frozen on a retained benchmark dataset that has been cleaned of duplicate data and takes into account the scaffold structure, and the XuAMP external validation setting is used.
[0156] (D) Benchmark boundary checks: Benchmark boundary checks are defined around the reserved evaluation partitions actually used in this study. Each downstream task is first cleaned according to applicable task deduplication rules, then partitioned using a scaffold-aware training / validation / testing protocol, and finally re-verified after partitioning.
[0157] Since Phase2 includes molecules from downstream training partitions, the arguments regarding critical independence in this paper concern the protection of the downstream validation / test set, not complete benchmark exclusivity at the training level. After completing the Phase2 dataset construction and molecule normalization, the adaptive corpus underwent explicit review for downstream validation and test partitions. Molecules with exact structural overlap were removed before training, and identifier matching was performed using normalized InChIKey strings. This review confirmed that there is no overlap between the Phase2 adaptive corpus and the downstream validation / test set, and that there is no exact overlap between the downstream training / validation / test partitions themselves.
[0158] Additional skeleton or connectivity similarities may still exist between the training and retention sets, particularly in the ncAA benchmark dataset for chemical property matching, but such nearest neighbor relationships are not considered exact leaks. Therefore, downstream analysis should be viewed as a “frozen feature” evaluation under conditions of unlabeled intra-domain adaptation (with the protected retention validation / test set preserved). This setting is more rigorous than simple adaptation on the entire benchmark dataset, but it is not equivalent to training on a corpus that is completely mutually exclusive with the benchmark dataset at the training level.
[0159] This distinction is particularly important for the `ncaa_binding` task. Even after reprocessing, the size of this task is still much smaller than `ncaa_cpp`, therefore R... 2 The confidence intervals of ncaa_binding are inherently less stable, and lightweight multilayer perceptron (MLP) heads based on high-dimensional frozen features are more sensitive to seed selection and early stopping. Therefore, ncaa_binding is mainly used as a sensitivity analysis that still preserves the sorting signal, rather than as a regression test set to determine the quality of the model.
[0160] Example 2: Cross-model benchmarking for five downstream tasks For five downstream tasks derived from the PepLand task suite, ncAA-RepDistill was benchmarked against descriptor baseline models, frozen peptide encoders, and frozen molecular encoders. To make the comparisons model-centric rather than pipeline-centric, all compatible entries in Table 2 were re-evaluated using the same single-hidden-layer multilayer perceptron (MLP) readout protocol under the same, deduplicated, and scaffold-supported frozen feature protocol. For the external molecular encoders, this table includes variants of ChemBERTa, GeminiMol, MoLFormer, and Uni-Mol2 as complementary chemical intrinsic baseline models.
[0161] Table 2. Cross-model benchmarks for five downstream tasks - This indicates that the model's endpoint combination is not applicable under the current chemical structure-based natural molecular input protocol.
[0162] Of the five tasks, ncaa_cpp is the most valuable comparative case in this study because it directly investigates the permeability changes of non-standard cyclic peptides, and even after reprocessing, it still has the richer data volume of the two regression datasets for ncAA. In this task, ncAA-RepDistill achieved the best performance (R²). 2 =0.4448±0.0081), surpassing the strongest hand-designed baseline model RDKit (0.4087±0.0057) and the strongest general molecular baseline model GeminiMol (0.3849±0.0156). This result indicates that the learned characterization captures information directly related to changes in the permeability of chemically modified cyclic peptides.
[0163] The remaining tasks can be understood as control conditions defining the operational boundaries of the method. On the three standard amino acid endpoint tasks, protein language models still perform the strongest, with ESM2 outperforming aa_binding, aa_cpp, and aa_sol. This result is consistent with the fact that these tasks are still closer to the residue-level domain represented by protein models. aa_sol requires particular caution in interpretation because it contains a large proportion of molecules whose standard SMILES exceed the intrinsic single-window loading capacity of the model; therefore, frozen feature extraction relies more on long-input approximation methods for very long inputs rather than direct single-window graph-partial encoding. In contrast, under the current benchmark protocol, ncaa_binding is mainly retained as supplementary evidence supporting ncAA because its effective sample size is still much smaller, and the corresponding regression estimates are less stable.
[0164] Example 3: Comparison between the matched "Phase 1 only" and "Phase 2 adjustment" groups To isolate the impact of ncAA adaptation, a matched “Phase 1 only” versus “Phase 2 adaptation” comparison was performed while keeping the downstream process unchanged. This study assembled eight archived, variant-matched Phase 1 / Phase 2 checkpoint pairs. Each checkpoint pair came from the same model configuration, first used for Phase 1 and then for Phase 2; therefore, this pairwise comparison reflects phased adaptation within a fixed architecture / training variant, rather than differences between different models. In each data pair, both the Phase 1 checkpoint and its corresponding Phase 2 adaptation checkpoint were evaluated using the same feature extraction, normalization, hyperparameter search range, early stopping mechanism, and repeated random seeds. The only expected difference in each data pair was whether the model underwent an unlabeled ncAA adaptation phase.
[0165] Figure 2 Eight archived checkpoint pairs evaluated on the ncaa_cpp dataset are summarized. In all eight pairs, the second-stage path improved R performance on the test set. 2 The value increased the mean from 0.389 to 0.426 and brought an average pairing gain of +0.0371.
[0166] The consistency of this directional change is more informative than the results of any single pairing. Since feature extraction, downstream model families, and evaluation workflows remain unchanged, the observed improvements can be attributed to the adaptation phase, rather than changes in downstream fitting conditions. Therefore, extensive standard peptide pre-training provides a useful initial representation, but explicit ncAA adaptation further improves the transfer performance in this matching evaluation.
[0167] Example 4: Structured Phase 1 Ablation Experiment To investigate which components in Phase 1 constitute the transferable representation before any ncAA-specific adaptations are introduced, this study employed a structured Phase 1 ablation experiment. This ablation panel included 11 variants covering five perturbation types: main-chain architecture (standard SMILES Transformer, i.e., no graph bias), training regularization (no randomized SMILES consistency), chemi-aware proxy supervision (no descriptor proxy head), teacher-side removal (no protein language model, no small molecule language model, and no language model teacher), and alternative weight scheduling schemes. All variants used the same first-stage data source, main-chain family, and downstream evaluation workflow. In the results, weight triples are labeled in the order of "protein / small molecule / mixed language model," with 50 / 30 / 20 being the default full-model allocation, and 30 / 50 / 20, 30 / 30 / 40, 35 / 35 / 30, 60 / 30 / 10, and 50 / 50 / 0 as demonstrated alternatives. All variants used the same downstream workflow in all five baseline tasks.
[0168] Will Figure 3 A contains the most information when viewed as a ranking rather than independent point differences; and Figure 3 B shows whether each perturbation is limited to the primary endpoint or causes broader shifts across the five tasks. Teacher-side perturbations remain the dominant signal. Removing the small molecule teacher or both teacher branches results in the model being furthest from the top of the NCAA_CPP rankings, indicating that molecular-level supervision is crucial for the final NCAA-guided representation, and that the two teacher views are complementary rather than redundant.
[0169] Architecture, regularization, and surrogate supervision perturbations further clarify this pattern. Removing graph bias only leads to a slight decrease in the primary endpoint metric, indicating that while topology injection improves model performance, it is not the sole determinant of transferability. In contrast, removing randomization SMILES consistency or descriptor surrogate supervision, while resulting in relatively small losses on the primary endpoint, incurs stronger penalties on selected secondary tasks (especially aa_binding and ncaa_binding). Therefore, its effect should be understood as stabilizing multi-task transferability and robustness to local chemistry, rather than simply increasing single-task gains on ncaa_cpp.
[0170] The results of the weight scan also clearly illustrate this point. Some scheduling schemes that simultaneously preserve all three objectives remain within a relatively narrow performance range, but the "50 / 50 / 0" scheme generally performs worse across all five tasks and ranks significantly lower than the full model in the main rankings. This indicates that the key issue is not the weight redistribution itself; rather, once the MLM is removed, teacher alignment loses its lexical-level structural anchors, and knowledge distillation alone is insufficient to generate stable, transferable representations. This is why the final design combines self-supervised reconstruction with teacher guidance, and why Phase2 continues to retain auxiliary supervision that considers chemical bonds, rather than relying solely on teacher transfer.
[0171] Example 5: Single-Edit Priority Comparison of ncAA-Scan This study also investigated whether the learned features, within a fixed scaffold context, can support local editing priority ranking. To this end, ncAA-Scan, a constrained benchmark built upon single-step edits observed in experiments on the retained test set ncaa_cpp, was used. Unlike the global prediction perspective in Table 1, the focus here is local and decision-oriented: by limiting the comparison to neighboring similarities within the same scaffold family, the study explored whether the model can maintain the ranking of editing effects and enrich beneficial edits within a limited candidate list. Accordingly, Figure 4 The models were summarized using three complementary metrics: the corresponding ncaa_cpp test set R. 2 The mean Spearman correlation coefficients at the scaffold level between the values, predicted editing effects, and observed editing effects, and the 10 editing enrichments (E@10) relative to the matched randomized strategy.
[0172] To ensure that this comparison focuses on downstream regression factors that represent quality rather than heterogeneity, Figure 4 All baseline models presented were re-evaluated on the same fixed partition using the same lightweight single-hidden-layer multilayer perceptron (MLP) head. Under this matching setting, the Phase2-adapted ncAA-RepDistill model occupies the optimal position in the overall optimization space, demonstrating a good balance in global fitting, scaffold-level editing order, and candidate set enrichment (R²). 2 = 0.448, the mean Spearman correlation coefficient for stent-based stents is 0.621, E@10 = 1.959). For example... Figure 4 As shown, this advantage is not limited to a single dimension: the model performs strongly on both the regression axis and the edit priority axis, indicating that the learned representations retain signals that directly contribute to the selection of beneficial local replacements in the stent series.
[0173] Among traditional descriptor baseline models, RDKit2D performed best, while Morgan-PCA128 was significantly weaker. This indicates that in this benchmark, the practical signals captured by hand-constructed features are primarily carried by a richer set of physicochemical descriptors, rather than solely determined by the Morgan fingerprint. Among external molecular encoders, ChemBERTa-MTR demonstrated the strongest global fitting ability, while ChemBERTa-ZINC and Uni-Mol2 remained competitive in skeleton-level editing and ordering. Nevertheless, in Figure 4 In the three decision-related evaluation metrics summarized, no model can achieve the same comprehensive balance as ncAA-RepDistill.
[0174] In summary, these results indicate that the learned representations not only help in making predictions across the entire dataset, but also, after identifying scaffold families, prioritize which experimentally observed local ncAA edits should be tested first.
[0175] Example 6: External validation based on a medicinal chemistry semipeptide macrocyclic compound permeability dataset To verify the usefulness of the learned characterizations beyond in-domain benchmarks, the same frozen Phase2 ncAA-RepDistill checkpoint was evaluated using the structured hemipeptide macrocyclic permeability dataset reported by Comeau et al. In the primary external analysis of this study, a 4×9 matrix consisting of 36 closely related macrocycles was focused on, as this matrix supports two chemically interpretable retention views: peptide_state and tether_state. In this structured series, “peptide_state” represents peptide-side modifications within the macrocycle, primarily peptide-to-peptide-like substitution patterns; while “tether_state” represents the methylation state and methyl position on non-peptide linkers. The matrix reports PAMPA measurements along with paired Caco-2 A→B and B→A readings, enabling the examination of passive permeability, cellular permeability, and efflux-related behaviors within a matched medicinal chemistry series.
[0176] like Figure 5As shown, among the tested baseline models, ncAA-RepDistill exhibits the strongest and most balanced external ranking performance. At the global level, this model achieves the highest overall Spearman correlation coefficient (0.426). This advantage is maintained even when analyzing individual detection results separately, achieving the best overall values on both the Caco-2 detection panel (0.383) and the PAMPA detection panel (0.482). The same trend extends to pairwise comparisons and shortlist-oriented metrics, indicating that the learned representations capture not only broad penetration tendencies but also the relative rankings among closely related analogues.
[0177] The structure of this dataset also makes external results easier to interpret at the mechanistic level. Retaining the "peptide_state" variable aims to test whether this representation can generalize to main-chain modification patterns on the peptide side, while retaining the "tether_state" variable aims to test whether it remains sensitive to methylation changes on the linker side. These changes, though subtle, are still property-related. The fact that the ordering signal is preserved in both perspectives confirms the expected role of ncAA-RepDistill as a "chemically intrinsic" representation for prioritizing analogs in modified macrocyclic peptide series.
[0178] Example 7: Evaluation of transfer performance on the XuAMP ncAA independent benchmark test This study further utilized the published retention test protocol to evaluate the frozen ncAA-RepDistill model on the published XuAMP ncAA independent benchmark. Under this setting, the learned representations still exhibited strong transferability when transferred to external, ncAA-considered antimicrobial agent classification tasks.
[0179] Table 3 shows that the frozen ncAA-RepDistill has reached the performance range of the dedicated AmpHGT pipeline and numerically surpasses it on all reported metrics. Its performance is also significantly higher than the results of the matched frozen PepLand rerun. Therefore, the key point is not architectural imitation, but transfer: a large portion of the downstream discriminative power already exists within the learned representation itself.
[0180] Table 3. Evaluation of transfer performance on the XuAMP ncAA independent benchmark. The results above demonstrate that the framework used in this study is not specifically designed for antimicrobial activity prediction; it is a general, intrinsically chemical peptide characterization method that can be reused with minimal downstream calibration. Therefore, the XuAMP results extend the evidence for transfer learning beyond major benchmarks: once the chemical properties of ncAA are distilled into a reusable feature space, this feature space remains effective in external classification tasks.
[0181] discuss The ncAA-RepDistill framework in this study is a chemically intrinsic characterization learning framework for ncAA peptides, rather than a task-specific predictive model. Its core principle is not that a single model should dominate the evaluation metrics for all modified peptides, but rather that preserving local chemical modifications and incorporating peptide-level context provides a more valuable foundation for prediction and local modification sequencing in non-standard amino acid peptide design. Within this scope, the strongest evidence in this study comes from the primary non-standard amino acid permeability experimental design, in which the experimental objective is most directly determined by local chemical modifications and scaffold context.
[0182] The findings also demonstrate that the value of this representation extends beyond global endpoint fitting. In the ncAA-Scan task, the same feature space remains valid when the task is defined from sample-level prediction to intra-series modification priority ranking. This distinction is crucial. In practical ncAA analogue evaluation, researchers typically compare closely related analogues and explore which local substituents should be tested next. Therefore, the representation that still provides useful information under the redefined approach in this invention is more suitable for practical design workflows than representations that only perform well on aggregated benchmark averages.
[0183] Existing research indicates that the cohort of hemipeptide macrocyclic compounds is not a random external library, but rather a structured series of analogs whose chemical variations are organized along two well-defined design axes: peptide-side backbone editing and linker-side methylation states. This makes external testing valuable at the mechanistic level. This invention retains "peptide_state" to examine whether this representation can migrate between peptide-side editing modes, while retaining "tether_state" to examine whether it remains sensitive to more localized linker modifications. Therefore, the ordering signal is preserved from both perspectives, supporting the idea that ncAA-RepDistill can capture chemically meaningful local variations that can be repeatedly used for backbone-constrained local editing ordering, rather than simply reproducing statistical regularities of intra-domain benchmarks.
[0184] Meanwhile, the XuAMP experiments expand the scope of the transfer discourse from an orthogonal direction. The hemipeptide macrocyclic compound series evaluates ranking transfer in the context of pharmaceutical chemical analogs, while XuAMP evaluates representation reuse in a preserved ncAA-perceptual classification setting. Taken together, these two transfer tests demonstrate that the learned representations are not tied to a single downstream scheme. Instead, these representations are valuable in both chemically driven analog ranking and external biological classification, further demonstrating that the training objective of this invention is to distill peptide- and molecular-level prior information into a deployable SMILES-based encoder. Therefore, the results of XuAMP should be interpreted as preserved-set benchmark transfer, rather than pre-training tests completely unrelated to the benchmark.
[0185] All documents mentioned in this invention are incorporated herein by reference as if each document were individually incorporated by reference. Furthermore, it should be understood that after reading the foregoing teachings of this invention, those skilled in the art can make various alterations or modifications to this invention, and these equivalent forms also fall within the scope defined by the appended claims.
Claims
1. A method for constructing a representation model of non-standard cyclic peptides SMILES, characterized in that, The method includes the following steps: (S1) Provide standard amino acid and non-standard amino acid datasets, optionally with structural diagrams corresponding to peptides in the datasets; the datasets include SMILES representations of peptides; (S2) Preprocess the SMILES representation and the optional structure graph to form the lexicalized SMILES representation and the optional bias matrix, respectively; (S3) Input the lexicalized SMILES representation and the optional bias matrix into a student model, and use the training module to perform knowledge distillation training on the student model; The student model includes a Transformer network; wherein the lexicalized SMILES representation is input into the Transformer network; the Transformer network includes a self-attention layer, a feedforward network layer, and an addition and normalization layer; wherein, in the self-attention layer, rotation position encoding and an optional bias matrix are applied to obtain the self-attention layer output; the self-attention layer output is processed sequentially through a first addition and normalization layer, the feedforward network layer, and a second addition and normalization layer to finally obtain the final layer lexical state, and the first special lexical state is used as a global feature; The training module includes a masked SMILES modeling module, a randomized SMILES consistency module, a dual-teacher projection alignment module, and a chemi-aware auxiliary supervision module; wherein, the masked SMILES modeling module includes a masked language header; the dual-teacher projection alignment module includes a protein teacher model and a molecular teacher model; the chemi-aware auxiliary supervision module includes a descriptor header; the knowledge distillation training includes: The first stage of training, which is conducted on the standard amino acid dataset, involves using the masked SMILES modeling module to impose lexical-level constraints on the final-layer lexical states; and using the randomized SMILES consistency module, the dual-teacher projection alignment module, the chemical perception-assisted supervision module, and molecular space contrastive learning to impose global representation-level constraints on the global features, thereby aligning the student model with the prior knowledge of the standard amino acids. The second stage of training is unlabeled training performed on the non-standard amino acid dataset; wherein, the mask SMILES modeling module is used to perform word-level constraints on the final layer word states; and the randomized SMILES consistency module, the molecular teacher model, and the chemical perception auxiliary supervision module are used to perform global representation-level constraints on the global features. (S4) When the training results reach the predetermined termination condition, the model training is terminated, and the obtained student model is the non-standard cyclic peptide SMILES representation model, namely ncAA-RepDistill.
2. The method as described in claim 1, characterized in that, The method further includes: (S5) freezing the downstream transfer and evaluation of the representation, wherein the student model is frozen and a lightweight downstream task head is trained on a labeled dataset to evaluate the transferability of the student model.
3. The method as described in claim 1, characterized in that, If the peptide segment does not exceed the intrinsic single window length, rotational position encoding is applied in the graph bias self-attention layer of the Transformer network, and the bias matrix is introduced into the attention score to obtain the self-attention layer output containing the bias matrix; finally, global features are obtained. If the peptide exceeds the intrinsic single window length, rotational position encoding is applied in the graph bias self-attention layer of the Transformer network to encode each character window separately, thereby obtaining a window-level aggregated representation; based on each window-level aggregated representation, a window-level global feature is obtained using the effective window length weighted average method.
4. The method as described in claim 1, characterized in that, The protein teacher model includes ESM2; the molecular teacher model includes GeminiMol. The loss function of the SMILES mask modeling module is: , in, This is a cross-entropy calculation term; This is the state of the final-level word element; For masking language header; The actual word tag at the masked position; The loss function of the randomized SMILES consensus module is: , in, For cosine consistency calculation terms; and Global features obtained by encoding two randomized SMILES views of the same peptide using the same student model; The loss function for each space in the dual-teacher projection alignment module is: , in, The vector formed by projecting the global features into the corresponding space; To freeze the target output in the corresponding space; The square of the Euclidean distance; Cosine similarity; These are weighting coefficients; When using the dual-teacher projection alignment module for constraint, the molecular spatial target is used as the anchor point to correct the protein spatial target, thereby obtaining residual features; the residual features are: , in, This is the term used to stop the gradient method calculation; The residual correction term obtained from learning; This is a projection of molecular space. For protein spatial projection; The loss function for correcting the protein spatial target using the molecular spatial target as the anchor point is: , in, This is the term for calculating the mean square error; To stop the gradient operator; To freeze the target output of the molecular teacher model; This is a projection of molecular space. This is the residual correction term after the protein spatial projection is mapped to molecular space; The loss function of the chemical sensing supervision module includes the main chemical sensing proxy regression loss and the auxiliary chemical sensing proxy regression loss. The loss function for the main chemical sensing proxy regression loss is: , Where g represents the global feature; The primary chemical agent is the regression target; The regression loss between the surrogate prediction and the corresponding target; For the agent head; The loss function for the auxiliary chemical sensing proxy regression loss is: , in, To assist in mean pooling representation; To assist chemical agents in regressing to the target; The regression loss between the surrogate prediction and the corresponding target; To assist the proxy head; The loss function for the molecular spatial contrastive learning is: , in, To compare and learn computational items; This is a projection of molecular space. To freeze the target output of the molecular teacher model; for Temperature coefficient; The loss function for the first stage of training is: , in, Distillation loss for the protein teacher model; Distillation loss for the molecular teacher model; For masked word cross-entropy loss; For molecular spatial contrast learning loss; This is the cosine consistency loss; The primary chemical sensing proxy regression loss; To assist in the regression loss of chemical sensing agents; This is the residual loss; Indicates the weight of each loss; The loss function for the second stage of training is: , in, Distillation loss for the molecular teacher model; For masked word cross-entropy loss; This is the cosine consistency loss; The primary chemical sensing proxy regression loss; To assist in the regression loss of chemical sensing agents; This is the residual loss; This indicates the weight of each loss.
5. A prediction system adapted for non-standard cyclic peptides, characterized in that, include: (M1) Input module, the input module being configured to input data, the data including a representation of the compound SMILES; (M2) A prediction module configured to perform the following operations: inputting the SMILES representation of the compound into a non-standard cyclic peptide SMILES representation model to obtain a global representation; inputting the global representation into a lightweight downstream task head to obtain a prediction result; the non-standard cyclic peptide SMILES representation model is constructed using the method of claim 1; (M3) Output module, which is configured to output the prediction results of the prediction module.
6. A method for permeability prediction using a non-standard cyclic peptide SMILES representation model, characterized in that, The method includes the following steps: (x1) Provides a compound SMILES; (x2) Input the SMILES representation of the compound into a non-standard cyclic peptide SMILES representation model to obtain a global representation; input the global representation into a lightweight regression head trained on labeled downstream data to obtain a permeability prediction value; The non-standard cyclic peptide SMILES representation model is constructed using the method described in claim 1.
7. A method for local editing sensitivity analysis using a non-standard cyclic peptide SMILES representation model, characterized in that, The method includes the following steps: (y1) Provide a SMILES representation of a compound, and a SMILES representation of one or more of the same molecules after local editing; (y2) Input the SMILES representation of the compound and all locally edited SMILES representations into a non-standard cyclic peptide SMILES representation model to obtain the corresponding global representation; input the global representation into a lightweight regression head trained with labeled downstream data to obtain the original predicted value and the predicted value corresponding to each local edit; wherein, the non-standard cyclic peptide SMILES representation model is constructed using the method described in claim 1; (y3) Evaluate the difference between the predicted value corresponding to each local edit and the original predicted value, and sort them according to the size of the difference to perform local edit sensitivity analysis.
8. An electronic device comprising a processor and a memory, characterized in that, The memory contains multiple executable instructions, and the processor is configured to read the instructions and execute the steps of the method as described in any one of claims 1, 6, or 7.
9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions that, when read and executed by a processor, implement the steps of the method as described in any one of claims 1, 6, or 7.
10. A computer program product comprising computer-executable instructions, characterized in that, When the computer-executable instructions are executed by a processor, they implement the steps of the method as described in any one of claims 1, 6, or 7.
