Multi-modal molecular representation learning method for predicting permeability of cyclic peptide

Abstract

The invention relates to the field of computer-aided drug design (CADD) and molecular informatics, in particular to a multi-modal representation learning method based on cyclopeptide molecules, which is used for predicting cell membrane permeability of cyclopeptide. The method mainly comprises the following steps: (1) data collection: integrating cyclic peptide permeability data from a ChEMBL database, a CycPeptMPDB database and a CyclicPepeda database and patent literatures; (2) multi-modal learning: for different modal data, a deep learning model is adopted to extract feature representations of the data; the method comprises the following steps of: encoding an SMILES sequence by using ChemBERTa (ChemBERTa); using Vision Transform to extract molecular image features, and learning a molecular image structure and 3D coordinate information based on GNN; (3) multi-modal feature fusion: adopting a self-adaptive extensible fusion mechanism, integrating SMILES feature information into image, graph and 3D coordinate features through a cross-modal feature fusion mechanism, and splicing all modal features to obtain multi-modal molecular representation; and (4) permeability prediction: sending the multi-modal molecular representation into a full connection layer for regression prediction so as to evaluate the permeability of the cyclopeptide.

Description

Technical Field

[0001] The present invention relates to the fields of computer-aided drug design (CADD) and molecular informatics, and in particular to a multimodal representation learning method based on cyclic peptide molecules for predicting the cell membrane permeability of cyclic peptides. Background Art

[0002] In recent years, cyclic peptides have shown excellent stability, heat resistance and biodegradability due to their unique cyclic structure and disulfide bonds, becoming potential molecules in drug development and filling the gap between biopharmaceuticals and small molecule drugs [1], [2]. They have been widely used in the fields of antiviral, antibacterial, anticancer and immunomodulatory. Compared with linear peptides, they have higher stability, stronger enzyme resistance and excellent biological activity, making them drug candidate molecules with great potential.

[0003] However, the prediction of its membrane permeability has always been a difficult problem in drug development. Its complex spatial conformation and chemical properties make it difficult for traditional methods to accurately capture transmembrane behavior. Permeability is affected by multiple factors such as molecular weight, polarity, hydrophobicity and cyclic structure. Traditional physical and chemical models are limited by their inability to fully characterize these complex interactions. For example, experiments such as PAMPA ("Ottaviani G, Martel S, Carrupt P A. Parallel artificial membrane permeability assay: a new membrane for the fast prediction of passive human skin permeability [J]. Journal of medicinal chemistry, 2006, 49 (13): 3948-3954.") are time-consuming and labor-intensive, making the development of accurate computational models particularly important. In recent years, machine learning has gradually become mainstream. Early studies used statistical methods to reveal the relationship between parameters such as hydrogen bonding characteristics and permeability. Some studies have attempted to construct QSPR models to predict the permeability of cyclic peptides. For example, Digiesi et al. (“Digiesi V, de la Oliva Roque V, Vallaro M, et al. Permeability prediction in the beyond-Rule-of 5 chemical space: Focus on cyclic hexapeptides[J]. European Journal of Pharmaceutics and Biopharmaceutics, 2021, 165:259-270.”) used a variety of computational tools to predict permeability, but due to data scarcity (only a few dozen samples), the scope of application was limited. The rise of deep learning has promoted the development of data-driven models. Molecular representation learning effectively captures the nonlinear relationship between molecular structure and properties by automatically extracting molecular features. It is particularly suitable for molecules with special structures such as cyclic peptides. Multi_Cycgt ("Cao L, Xu Z, Shang T, et al. Multi_cycgt: a deep learning-based multimodal model for predicting the membrane permeability of cyclic peptides[J]. Journal of Medicinal Chemistry, 2024, 67(3): 1888-1899.") uses a deep learning approach to predict the membrane permeability of cyclic peptides by fusing multimodal information such as graphs, sequences, and various molecular properties.

[0004] Existing multimodal cyclic peptide permeability prediction methods have two major drawbacks: First, while attempts have been made to combine multiple features such as graph structure, sequence information, and molecular descriptors, the feature fusion process often lacks effective mechanisms to capture the complex interactions between different modalities. Second, cyclic peptide permeability data is relatively scarce, and existing experimental datasets are small, which leads to overfitting during model training and poor generalization ability.

[0005] This patent addresses the aforementioned shortcomings of existing multimodal cyclic peptide permeability prediction methods by proposing an innovative molecular representation learning framework. This framework integrates multimodal information, including SMILES sequences, molecular images, molecular graphs, and 3D coordinates, aiming to address the issue of insufficient feature fusion and improve the accuracy and generalization of cyclic peptide permeability prediction. To address data scarcity, a cyclic peptide permeability dataset was constructed. Summary of the Invention

[0006] The present invention proposes a novel method for multimodal molecular representation learning for cyclic peptide permeability prediction, which mainly includes the following steps:

[0007] (1) Data collection: integrating cyclic peptide permeability data from the ChEMBL database, CycPeptMPDB database, CyclicPepedia database, and patent literature and performing data processing;

[0008] (2) Multimodal learning: Deep learning models are used to extract feature representations of different modal data. ChemBERTa is used to encode SMILES sequences; ViT (Vision Transformer) is used to extract molecular image features, and the molecular graph structure and 3D coordinate information are learned based on the graph neural network (GNN);

[0009] (3) Multimodal feature fusion: Adopting an adaptive and scalable fusion mechanism, SMILES feature information is integrated into image, graph, and 3D coordinate features through a cross-modal feature fusion mechanism, and the features of each modality are spliced to obtain a multimodal molecular representation;

[0010] (4) Permeability prediction: The multimodal molecular representation is fed into the fully connected layer for regression prediction to evaluate the permeability of the cyclic peptide.

[0011] 1. Data Collection Phase

[0012] We integrated data from three primary sources to construct a high-quality cyclopeptide permeability dataset. First, we extracted 20,490 cyclopeptide molecules from the ChEMBL database. Second, we collected 7,268 cyclopeptide sequences from the CycPeptMPDB database and 6,601 cyclopeptide sequences from the CyclicPepedia database. Finally, we screened 226 cyclopeptide sequences from patent literature. The resulting dataset contained 34,585 data points.

[0013] For each cyclic peptide molecule, RDKit was used to normalize the SMILES sequence into a string representation suitable for deep learning models. RDKit also generated a standardized 224×224 pixel resolution RGB image of the molecule from the SMILES sequence for visual feature extraction. The 2D structure and 3D coordinates of the molecule were also generated.

[0014] 2. Multimodal learning stage

[0015] SMILES (Simplified Molecular Input Line Notation) is a string representation that describes the structure of a molecule. When processing SMILES through the ChemBERTa encoder, the SMILES is first converted into a token sequence and then mapped to a high-dimensional continuous vector space through the embedding layer. The embedded vector is then input into the Transformer encoder with the positional encoding to obtain the hidden state vector of the molecule. Capturing global information as a holistic representation of the molecule.

[0016] ViT (Vision Transformer) is a visual model based on the Transformer architecture, which is used to process the image information of cyclic peptide molecules. First, the image is cut into patches of fixed size 16×16×3 and flatten each patch into 16 2 ×3 one-dimensional vector, which is mapped to a fixed-length vector through linear transformation. Then, a learnable token embedding is inserted before the first patch. Aggregate global visual information and preserve the relative position information between patches through position encoding:

[0017]

[0018] p v Visual features obtained through Transformer processing

[0019] Next, through the unified GNN framework, the input molecular graph G = (V, E) and three-dimensional coordinates R∈R N×3Encoded into a high-dimensional feature vector. For 3D coordinates, we use a multi-layer perceptron (MLP) to encode the coordinates of the atoms and enhance the atomic features χ i :

[0020]

[0021] For edge features, the Euclidean distance between nodes i and j is encoded by MLP to generate edge features containing 3D information ∈ ij :

[0022]

[0023] Enhanced node and edge features are passed through message m i Pass for update,

[0024]

[0025] The method to update a node is as follows:

[0026]

[0027]

[0028] Finally, we get the node features containing 3D coordinate information

[0029] 3. Multimodal feature fusion stage

[0030] We designed a cross-modal feature fusion mechanism to efficiently integrate the four modal information to construct a unified molecular representation. We designed a cross-modal feature mapping mechanism to ensure that each modality is aligned in the same high-dimensional space to eliminate the heterogeneity between modalities. Through the cross-modal feature fusion mechanism, the information of different modalities is integrated into a unified molecular representation. v 、h p With h s This module consists of two cross-layers, each of which captures the interaction between different modalities through self-attention, cross-attention and feed-forward sub-layers.

[0031] The self-attention mechanism calculates the attention weight through the Q, K, and V matrices of the same modality, d k is the dimension of the attention head:

[0032]

[0033]

[0034]

[0035]

[0036] The cross attention uses the Q matrix of the current modality to interact with the K and V matrices of the other modality:

[0037]

[0038]

[0039]

[0040] The result of the cross attention is then passed through the feedforward sublayer to form a unified molecular representation that fuses the information of each modality:

[0041]

[0042]

[0043]

[0044] The fused features of each modality are spliced into a unified multimodal molecular representation Z, which integrates SMILES sequence, molecular image, 2D graph structure and 3D coordinate information, and can fully characterize the permeability-related characteristics of cyclic peptide molecules:

[0045] Z=concat(Z s ,Z v ,Z p )#(17)

[0046] 4. Permeability prediction stage

[0047] After completing the multimodal feature fusion, a unified high-dimensional multimodal molecular representation Z is obtained, which is directly input into the fully connected layer to calculate the predicted permeability value:

[0048] y p =W out Z+b out #(18)

[0049] We use the mean squared error (MSE) to measure the difference between the predicted value and the actual permeability value y p The error between:

[0050]

[0051] y i is the true permeability value of the i-th sample, and N is the total number of samples. To prevent the model from overfitting, we introduce L2 regularization (weight decay). The final optimization goal is as follows:

[0052] L=L MSE +λ||Wout || 2 #(20)

[0053] λ is the regularization coefficient. Finally, a 10-fold cross validation method is used to objectively evaluate the performance of the method. DETAILED DESCRIPTION

[0054] The present invention relates to the fields of computer-aided drug design (CADD) and molecular informatics, and more specifically to a multimodal representation learning method based on cyclic peptide molecules. This method comprehensively utilizes SMILES sequences, molecular images, molecular graph structures, and 3D coordinate information to construct an efficient molecular characterization model to improve the accuracy and generalization ability of permeability prediction. The present invention is described in detail below with reference to the following examples, but those skilled in the art should understand that these embodiments are merely for explaining the technical principles of the present invention and do not limit the scope of protection of the present invention.

[0055] Step 1: Collect cyclic peptide permeability-related data from ChEMBL, CycPeptMPDB, CyclicPepedia databases and patent literature, which contain 20,490, 7,268, 6601 and 226 cyclic peptide sequence data respectively. All data were screened and integrated, and finally 27,984 sample data were obtained, covering molecular sequence information and permeability experimental measurement values. In order to construct a multimodal feature representation, RDKit was used to perform standardization processing for each cyclic peptide molecule, and SMILES was converted into a string representation that can be used by deep learning models. RDKit was used to obtain a standardized molecular RGB image from the SMILES sequence, and the image resolution was set to 224×224. At the same time, the 2D graph structure and 3D coordinates of the molecule were calculated and generated based on RDKit.

[0056] Step 2: For the four modal data (SMILES, molecular image, 2D molecular graph, 3D coordinates) obtained in step 1, different deep learning models are used for feature extraction. SMILES is encoded through the ChemBERTa encoder to obtain the feature representation h of the molecular sequence. s ; Use the ViT model to process the molecular image, divide the image into patches, and obtain the high-dimensional visual feature h through formula (1) and Transformer processing v ; The 2D molecular graph G = (V, E) and 3D coordinates R are encoded through a unified GNN framework to capture the topological information of the molecule. The 3D coordinates are encoded using MLP (Multi-layer Perceptron) and fused with the 2D structure to capture the spatial conformational information. The node feature h containing the 3D coordinate information is obtained by processing formulas (2), (3), ... (6) p .

[0057] Step 3: In order to effectively integrate the information of each modality, the four modalities are aligned to the same high-dimensional space to eliminate the heterogeneity between the modalities. The feature vectors of each modality obtained in step 2 are combined with the SMILES feature h through the cross-modal feature fusion mechanism. s Integrated into the visual feature h v and node features h containing 3D coordinate information p To build a unified multimodal molecular representation. s Z is obtained by formulas (8), (11), and (14): s , h v Z is obtained by formulas (9), (12), and (15): v , h p Z obtained by processing formulas (10), (13), and (16) p The features of each modality are then combined into a unified multimodal molecular representation Z using formula (17). This representation integrates SMILES sequence, molecular image, 2D graph structure and 3D coordinate information, and can fully characterize the permeability-related features of cyclic peptide molecules.

[0058] Step 4: Through the cross-modal feature fusion mechanism of step 3, we obtain Z, which integrates the four modal information, and use formula (18) to obtain the predicted permeability value. In order to evaluate the performance of the model, we use formula (19) to calculate the mean square error and optimize the regression model. 10-fold cross validation is used to further verify the generalization ability of the model. Specifically, we randomly divide the dataset into 10 subsets of equal size, where one subset is selected as the test set each time, and the remaining 9 subsets are used as the training set. This process is repeated 10 times to ensure that each sample participates in the test. Finally, the mean MSE of the 10 tests is taken as the model performance indicator.

[0059] Through the above steps, this study constructed a complete multimodal deep learning framework, which effectively improved the accuracy and generalization ability of cyclic peptide permeability prediction.

[0060] Those skilled in the art will understand that the scope of protection of the present invention is not limited to the specific embodiments described. Without departing from the principles of the present invention, those skilled in the art may make equivalent changes or substitutions to the relevant technical features. It should be noted that the technical solutions after such changes or substitutions will fall within the scope of protection of the present invention.

Claims

1. A multimodal molecular representation learning method for cyclic peptide permeability prediction, characterized in that The implementation steps are: (1) Data collection: integrating cyclic peptide permeability data from the ChEMBL database, CycPeptMPDB database, CyclicPepedia database, and patent literature; (2) Multimodal learning: For different modal data, a deep learning model is used to extract their feature representations. ChemBERTa is used to encode SMILES sequences, ViT (Vision Transformer) is used to extract molecular image features, and the molecular graph structure and 3D coordinate information are learned based on the graph neural network (GNN); (3) Multimodal feature fusion: Adopting an adaptive and scalable fusion mechanism, SMILES feature information is integrated into image, graph, and 3D coordinate features through a cross-modal feature fusion mechanism, and the features of each modality are spliced to obtain a multimodal molecular representation; (4) Permeability prediction: The multimodal molecular representation is fed into the fully connected layer for regression prediction to evaluate the permeability of the cyclic peptide.

2. A method for multimodal molecular representation learning for cyclic peptide permeability prediction according to claim 1, characterized in that Data collection phase: (1) Data related to cyclic peptide permeability were collected from ChEMBL, CycPeptMPDB, CyclicPepedia databases, and patent literature, which contained 20,490, 7,268, 6,601, and 226 cyclic peptide sequence data, respectively; (2) Data screening, deduplication, and format standardization were performed to construct a comprehensive dataset containing 27,984 samples, which included SMILES molecular sequence information and experimentally determined permeability values; (3) SMILES sequences are preprocessed using RDKit and converted into string representations that can be processed by deep learning models; (4) SMILES was converted into standardized RGB molecular images using RDKit. The image resolution was set to 224 × 224, and the 2D structure and 3D coordinates of the molecules were generated.

3. A method for multimodal molecular representation learning for cyclic peptide permeability prediction according to claim 1, characterized in that Multimodal learning stage: (1) ChemBERTa was used as the sequence encoder to process the SMILES representation of cyclic peptides; (2) Using Vision Transformer (ViT) as a visual encoder to extract deep features of molecular images; (3) Graph neural network (GNN) is used to uniformly process the molecular graph structure and 3D coordinates, and construct a high-dimensional feature vector - node features containing 3D coordinate information.

4. A method for multimodal molecular representation learning for cyclic peptide permeability prediction according to claim 1, characterized in that Multimodal feature fusion stage: (1) Through the cross-modal feature fusion mechanism, SMILES features are effectively integrated with visual features and node features containing 3D coordinate information. Self-attention mechanism, cross-attention mechanism and feedforward neural network are used to ensure that information between different modalities can be fully transmitted and shared, thereby achieving a close fusion of features from each modality; (2) The fused features of each modality are spliced into a unified multimodal molecular representation, which integrates SMILES sequence, molecular image, 2D graph structure and 3D coordinate information, and can comprehensively characterize the permeability-related characteristics of cyclic peptide molecules.

5. A method for multimodal molecular representation learning for cyclic peptide permeability prediction according to claim 1, characterized in that Permeability prediction stage: (1) The prediction of cyclic peptide permeability is regarded as a regression problem, and the fused feature vector is input into the regression model to predict the permeability value; (2) Calculate the prediction mean square error and optimize the regression model to improve the prediction accuracy; (3) Using the 10-fold cross-validation method, the data set is randomly divided into 10 subsets. Each time, one subset is selected as the test set, and the remaining subsets are used as the training set. This process is repeated 10 times to ensure that each sample participates in the test. Finally, the mean square error (MSE) of 10 tests was taken as the performance indicator of the model.

Images