Contrastive learning molecular property prediction method based on counterfactual and large language model

Abstract

This invention discloses a contrastive learning method for predicting molecular properties based on counterfactual and large language models, belonging to the field of molecular representation learning technology that combines large language models and graph neural networks. The method includes the following steps: generating an original molecular graph from the molecular SMILES string; generating a skeleton perturbation graph and a functional group perturbation graph; ensuring that the generated perturbation graphs are counterfactual hard negative samples; generating a natural language description of the molecule using a large language model and converting it into a molecular semantic embedding vector using a small language model; performing contrastive learning among the original molecular graph, positive samples, and counterfactual hard negative samples; and fusing the original molecular graph embedding obtained after contrastive learning with the molecular semantic embedding for downstream molecular property prediction. This invention not only generates hard negative samples through a counterfactual mechanism but also ensures the diversity of molecular representation patterns through a large language model, significantly improving the accuracy of contrastive learning in predicting molecular properties.

Description

Technical Field

[0001] This invention belongs to the field of molecular representation learning technology that combines large language models and graph neural networks, specifically involving a contrastive learning method for predicting molecular properties based on counterfactual and large language models. Background Technology

[0002] In molecular property prediction tasks, contrastive learning enhances the robustness and generalization of models by augmenting molecular graph structures from different perspectives. However, traditional random augmentation methods often generate simple negative samples that lack chemical meaning, hindering the model from learning the fine-grained dependencies between structure and properties.

[0003] On the other hand, large language models possess powerful semantic understanding and knowledge reasoning capabilities, enabling them to uncover implicit chemical patterns from molecular descriptions. Combining counterfactual mechanisms with large language models can improve the diversity and accuracy of molecular representations while generating semantically rich and chemically plausible enhanced samples. Summary of the Invention

[0004] To address the aforementioned issues, this invention proposes a contrastive learning method for predicting molecular properties based on counterfactual and large language models. This method generates chemically constrained counterfactual hard negative samples through a counterfactual mechanism and introduces linguistic semantic positive samples, thereby improving the accuracy and interpretability of molecular property prediction tasks.

[0005] The technical solution of the present invention is as follows:

[0006] A contrastive learning method for predicting molecular properties based on counterfactual and large language models includes the following steps:

[0007] Step 1: Generate the original molecular diagram using the molecular SMILES string;

[0008] Step 2: Based on the original molecular diagram, generate the skeleton perturbation diagram and the functional group perturbation diagram by using skeleton perturbation and functional group perturbation respectively;

[0009] Step 3: Ensure that the generated perturbation map is a counterfactual hard negative sample by using the Frobenius norm and KL divergence function;

[0010] Step 4: Generate a natural language description of the molecule using the large language model ChatGPT, and convert it into a molecular semantic embedding vector using the small language model RoBERTa, which serves as a positive sample in the contrastive learning.

[0011] Step 5: Perform comparative learning between the original molecular graph, positive samples, and counterfactual hard negative samples to maximize the similarity of positive samples and minimize the similarity of counterfactual hard negative samples;

[0012] Step 6: Fuse the original molecular graph embedding obtained after contrastive learning with the molecular semantic embedding to predict downstream molecular properties.

[0013] Furthermore, the specific process of step 1 is as follows:

[0014] Step 1.1: Obtain the SMILES string of the molecule and input it into the chemical information parsing module for molecular topological reconstruction. The chemical information parsing module parses the string character by character according to the syntax rules of SMILES, identifies the atomic symbols, branched structures, ring markers, aromaticity symbols and bond type symbols, and reconstructs the two-dimensional topological structure of the molecule.

[0015] Step 1.2: After completing the molecular topological reconstruction, traverse all atoms and construct a set of nodes in the graph based on their chemical properties. Extract the following descriptive information for each atom as node features: atom type, atomic number, formal charge, valence state, hybridization mode, aromaticity, and chiral label. Each atom is a graph node, and the above features are used to construct a node feature vector, thereby forming the node feature matrix of the graph.

[0016] Step 1.3: In the molecular structure, if there is a chemical bond between any two atoms, there will be a corresponding edge in the molecular diagram;

[0017] Step 1.4: Extract the structural and property information of each chemical bond to form edge features, including: bond type, bond direction, whether the bond belongs to a ring structure, and whether it is a conjugated bond; the edge features are organized in the form of feature vectors to form an edge feature matrix; by traversing all chemical bonds, construct an edge set and record the numbers of the two atoms connected by the bond to form the topological structure of the graph;

[0018] Step 1.5: Construct a complete original molecular graph using the node set, edge set, and corresponding node and edge features. :

[0019] ;

[0020] in, It is a set of nodes consisting of all atoms; It is the set of edges consisting of all chemical bonds; The node feature matrix; Let be the edge feature matrix.

[0021] Furthermore, the specific process of step 2 is as follows:

[0022] Step 2.1: First, generate a skeleton perturbation map using skeleton perturbation; the specific process is as follows:

[0023] Step 2.1.1: Skeletal perturbation from the original molecular map Extract the scaffold diagram and convert it into a line graph; specifically, use the open-source toolkit RDKit to extract the original molecular diagram. bracket diagram And turn it into a line graph ;in, , , These are the adjacency matrix, node feature matrix, and edge weight matrix of the line graph; the original molecular graph. Each edge in Online chart A node corresponds to a middle node. , , These represent the indices of the two atomic nodes connected by a chemical bond; if the original molecular diagram... The edge in and edge If there is a common node between them, then the line graph Edges are generated between corresponding nodes. ; Indicates the index of the atomic nodes connected by a chemical bond;

[0024] Step 2.1.2: Select two nodes from the line graph and map them to two edges of the support graph;

[0025] Step 2.1.3: Perform the edge swapping operation to generate a new support structure and thus obtain the skeleton perturbation diagram;

[0026] Step 2.2: Then, functional group perturbation is used to generate a functional group perturbation map.

[0027] Furthermore, the specific process of step 2.1.2 is as follows:

[0028] After the line chart is generated, first in Randomly select a node Then, to reduce the search space, we introduce... The adjacency matrix, Each element in the adjacency matrix is ​​represented as:

[0029] ;

[0030] in, For nodes and nodes The edges established between them; , For indexes of different nodes;

[0031] The operation of reducing the search space is divided into the following steps:

[0032] First, perform a masking operation on the nodes using the adjacency matrix of the line graph; select For the first node used for edge swapping, introduce a first mask vector. Set it up as follows Each element:

[0033] ;

[0034] in, for The first in 1 element, element index Index of the corresponding node ; for The number of nodes in;

[0035] Then, for the adjacency matrix of the line graph... Perform a logical AND operation between each row and every other row; row index. Index of the corresponding node Each element of the adjacency matrix of the line graph is re-represented as follows:

[0036] ;

[0037] in, Based on nodes Under the constraints, the line graph undergoes a secondary filtering of adjacency relationships, retaining those with... Edge pairs with the same adjacent nodes Set all others to 0; For nodes and nodes The edges established between them; For nodes and nodes The edges established between them; For the index of the node; if the first The first line If the nth element is 1, then the nth element is 1. Soon to meet with the first row and number Perform a logical AND operation on rows; row indexes. Index of the corresponding node ; element index Index of the corresponding node If any element in a row is 1, the corresponding node will be masked. In ;

[0038] Finally, the corresponding edges with different valence key values ​​are masked; a second mask vector is constructed. ,set up Each element is as follows:

[0039] ;

[0040] in, for The first in One element; , These are nodes in the line graph. ,node The corresponding valence bond values ​​at the edges of the original molecular diagram;

[0041] After all the above masking processes are completed, randomly select and Index of all elements in the middle is 1 As the second node .

[0042] Furthermore, the specific process of step 2.1.3 is as follows:

[0043] Will and These two nodes are converted into corresponding edges in the scaffold graph, which are the first edges to be swapped. The second side to be exchanged ; the bracket diagram , Corresponding to the original molecular diagram The edges in the middle, that is, the edges in the middle, are obtained , ,in, , , , Selected edges respectively , The four nodes on; This refers to the properties of chemical bonds; then, they are broken. , Then reconnect, and obtain the first swapped edge after reconnection. The second swapped edge after reconnection This leads to the skeleton perturbation map. .

[0044] Furthermore, the specific process of step 2.2 is as follows:

[0045] Step 2.2.1: From the original molecular diagram Randomly select an edge And interrupt it; among them , Selected edges respectively The two nodes on the graph; update the number of hydrogen atoms at the nodes where chemical bonds are broken:

[0046] ;

[0047] in, for The updated value; The number of hydrogen atoms at the nodes where chemical bonds are broken; It is the valence state of the fracture edge;

[0048] Step 2.2.2: Randomly select two disconnected nodes and connect them to obtain a new edge. ,in , They are the edges The two nodes on the graph; update the number of hydrogen atoms in the nodes connected by chemical bonds:

[0049] ;

[0050] in, for The updated value; The number of hydrogen atoms at the nodes that form chemical bonds;

[0051] Step 2.2.3, when When the value is less than 0, the number of electrons in the nodes connecting chemical bonds is updated:

[0052] ;

[0053] in, for The updated value; The number of electrons at the nodes that connect chemical bonds;

[0054] Step 2.2.4, let This leads to the functional group perturbation diagram. .

[0055] Furthermore, the specific process of step 3 is as follows:

[0056] Step 3.1: Construct a first objective function to maximize the similarity between the original molecular graph and the two perturbation graphs; the first objective function... The design is as follows:

[0057] ;

[0058] in, It is the Frobenius norm;

[0059] Step 3.2: Construct a second objective function to minimize the similarity between the probability distributions of the original molecular graph and the perturbation graph; the second objective function... The design is as follows:

[0060] ;

[0061] in, is the KL divergence function, used to measure the similarity between two probability distributions; This indicates a predictor that outputs the probability of the current graph belonging to a specific category.

[0062] Step 3.3: Overall objective function for generating counterfactual facts of hard negative samples as follows;

[0063] .

[0064] Furthermore, the specific process of step 4 is as follows:

[0065] The SMILES string is input into the ChatGPT large language model; a natural language prompt template is written to guide ChatGPT to output in a specified style. ChatGPT generates a natural language description based on the input SMILES string and the prompt template. :

[0066] ;

[0067] in, ChatGPT is a large language model. The string is for the molecule SMILES; For natural language prompt word templates;

[0068] Natural language descriptions generated by the large language model ChatGPT Input into the small language model RoBERTa to generate molecular semantic embedding vectors. As positive samples in contrastive learning:

[0069] ;

[0070] in, For sequence aggregation functions; This is a Transformer-based context representation encoder in the small language model RoBERTa; This is the text segmentation module.

[0071] Furthermore, the specific process of step 5 is as follows:

[0072] Comparative learning is performed using a dictionary lookup method, given the original molecular diagram. Skeleton perturbation diagram and functional group perturbation diagram Molecular semantic embedding vector and the first encoder With the second encoder Define four embedded formulas and in the dictionary Searching for original molecular maps Embedded vector Best matching key The four embedded formulas are as follows:

[0073] ;

[0074] ;

[0075] ;

[0076] ;

[0077] in, Original molecular diagram The embedding vector; This is a molecular semantic embedding vector; skeleton perturbation diagram The embedding vector; Functional group perturbation diagram The embedding vector;

[0078] The objective function for contrastive learning employs InfoNCE loss to maximize the similarity between the original molecular graph and positive samples, and minimize the similarity between the original molecular graph and counterfactual hard negative samples. The InfoNCE loss is as follows:

[0079] ;

[0080] in, Loss due to InfoNCE; It is an exponential function; Similarity; This refers to temperature hyperparameters. For the first A counterfactual hard negative sample.

[0081] Furthermore, the specific process of step 6 is as follows:

[0082] After comparative learning is completed, and The similarity reaches its maximum; first, and The fusion process is performed to generate the fused molecular representation:

[0083] ;

[0084] ;

[0085] ;

[0086] in, To compare the original molecular diagram after learning The embedding vector; To compare the learned molecular semantic embedding vectors; The fused molecules are represented as follows; , These are the weighting coefficients of the molecular semantic embedding vector and the original molecular graph embedding vector, respectively, satisfying... ;

[0087] The fused molecular representation is then input into a randomly initialized multilayer perceptron network to perform a molecular property prediction task.

[0088] ;

[0089] ;

[0090] ;

[0091] in, As the first intermediate variable; It is the second intermediate variable; This indicates the predicted results of molecular properties; It is the ReLU activation function; It is a linear mapping function;

[0092] Finally, the predicted molecular properties are used as training supervision signals for fine-tuning through supervised learning; the total loss function for supervised learning is defined. for:

[0093] ;

[0094] in, As weight; To predict the loss based on properties, the formula is:

[0095] ;

[0096] in, For the first The true molecular properties of each data point; Indicates the number of data points in the dataset; For the first The molecular property prediction results for the data.

[0097] The beneficial technical effects brought about by this invention are as follows.

[0098] 1. Generating chemically constrained counterfactual hard negative samples to improve model robustness: This invention constructs counterfactual samples through skeleton perturbation and functional group perturbation, and uses perturbation measurements to ensure that the perturbed molecules maintain chemically interpretable similarity differences, thereby generating "hard negative samples" that are difficult to obtain using traditional random augmentation methods. Compared with simple random pruning and random replacement, the generated counterfactual hard negative samples have greater structural discriminative power and can effectively improve the model's sensitivity and robustness to key structural features.

[0099] 2. Introducing positive semantic samples to enhance the semantic richness of molecular representation: This invention utilizes the large language model ChatGPT to generate natural language descriptions of molecular structures, which are then transformed into text embeddings by smaller language models such as RoBERTa, thereby obtaining high-level features containing semantic information such as functional groups, reactivity, and toxicological characteristics. These positive semantic samples can supplement molecular chemical knowledge that is difficult to express using graph structures, making molecular representation more comprehensive and refined.

[0100] 3. Integrating graph structure features and linguistic semantic features to improve the accuracy of molecular property prediction: This invention integrates graph structure features extracted by GNN with RoBERTa's linguistic embedding vectors to construct a unified molecular representation across modalities. This hybrid representation can simultaneously capture the molecular topology, functional group distribution, and chemical semantic patterns, significantly improving the accuracy and generalization ability of downstream prediction models. Attached Figure Description

[0101] Figure 1 This is the overall flowchart of the contrastive learning molecular property prediction method based on counterfactual and large language models of the present invention.

[0102] Figure 2 This is a schematic diagram of the perturbation process of a certain original molecular skeleton in this invention.

[0103] Figure 3 This is a schematic diagram of the perturbation process of functional groups in a certain original molecular diagram of the present invention.

[0104] Figure 4 This is a flowchart illustrating the generation of molecular semantic embedding vectors using the ChatGPT large language model of this invention.

[0105] Figure 5 This is a schematic diagram of the comparative learning structure of the present invention. Detailed Implementation

[0106] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments:

[0107] like Figure 1 As shown, this invention proposes a contrastive learning method for predicting molecular properties based on counterfactual and large language models, specifically including the following steps:

[0108] Step 1: Generate the original molecular diagram from the molecular SMILES string. The specific process is as follows:

[0109] Step 1.1: Obtain the SMILES string for the molecule and input it into the chemical information parsing module for molecular topological reconstruction. This module parses the string character by character according to the SMILES syntax rules, identifying atomic symbols, branched structures, ring markers, aromaticity symbols, and bond type symbols to reconstruct the molecule's two-dimensional topological structure. The resulting molecular topological structure includes: a list of atoms and their unique numbers, the connections between atoms, information on the type and directionality of chemical bonds, and descriptions of aromaticity, ring structure, and chirality.

[0110] Step 1.2: After completing the molecular topological reconstruction, traverse all atoms and construct a set of nodes in the graph based on their chemical properties. Extract the following descriptive information as node features for each atom: atom type (e.g., carbon, nitrogen, oxygen), atomic number, formal charge, valence state, hybridization mode, aromaticity, chiral label, etc. Each atom is a graph node, and its above features constitute a node feature vector, thereby forming the node feature matrix of the graph.

[0111] Step 1.3: In the molecular structure, if there is a chemical bond between any two atoms, there will be a corresponding edge in the molecular diagram;

[0112] Step 1.4: To further preserve the chemical semantics of chemical bonds, extract the structural and property information of each chemical bond to form edge features, including: bond type (single, double, triple, or aromatic), bond direction (if stereochemical directionality exists), whether the bond belongs to a ring structure, and whether it is a conjugated bond, etc.; the edge features are organized in the form of feature vectors to form an edge feature matrix. By traversing all chemical bonds, an edge set is constructed, recording the atom numbers of the two atoms connected by the bond, thus forming the topological structure of the graph;

[0113] Step 1.5: In summary, by using the set of nodes, the set of edges, and the corresponding node and edge features, a complete original molecular graph is constructed. Its form can be expressed as:

[0114] ;

[0115] in, It is a set of nodes consisting of all atoms; It is the set of edges consisting of all chemical bonds; The node feature matrix; Let be the edge feature matrix.

[0116] Step 2: Based on the original molecular map Skeleton perturbation maps were generated using skeleton perturbation and functional group perturbation, respectively. and functional group perturbation diagram The specific process is as follows:

[0117] Step 2.1: First, generate a skeleton perturbation map using skeleton perturbation. Skeletal perturbation is a method of generating isomers by exchanging the edges of the molecular scaffold without changing the functional groups of the molecules, generating a skeletal perturbation diagram. The specific process is as follows:

[0118] Step 2.1.1: First, the skeletal perturbation is performed from the original molecular map. Extract the scaffold diagram and convert it into a line graph;

[0119] This invention specifically uses the open-source toolkit RDKit to extract the original molecular map. bracket diagram And turn it into a line graph ;in, , , These are the adjacency matrix, node feature matrix, and edge weight matrix of the line graph, respectively.

[0120] Original molecular diagram Each edge in Online chart A node corresponds to a middle node. , , These represent the indices of the two atomic nodes connected by a chemical bond; if the original molecular diagram... The edge in and edge If there is a common node between them, then the line graph Edges are generated between corresponding nodes. ; This represents the index of the atomic nodes connected by a chemical bond. In this way, selecting all four nodes can be simplified to selecting only two edges when swapping them.

[0121] Step 2.1.2: Then, select two nodes from the line graph and map them to two edges of the support graph;

[0122] After the line chart is generated, first in Randomly select a node Then, to reduce the search space, we introduce... Adjacency matrix, line graph Each element in the adjacency matrix is ​​represented as:

[0123] ;

[0124] in, For nodes and nodes The edges established between them; , For indexes of different nodes;

[0125] The operation of reducing the search space is divided into the following steps:

[0126] First, to avoid selecting edges with shared nodes or generating edges already existing in the original molecular graph during edge swapping, a node masking operation is performed using the adjacency matrix of the line graph. Selected For the first node used in edge swapping, this invention introduces a first mask vector. To ensure that duplicate nodes are not selected, configure as follows: Each element:

[0127] ;

[0128] in, for The first in 1 element, element index Index of the corresponding node ; for The number of nodes in;

[0129] Then, to avoid reconnecting to existing edges, the adjacency matrix of the line graph is modified accordingly. Perform a logical AND operation between each row and every other row; row index. Index of the corresponding node Each element of the adjacency matrix of the line graph is re-represented as follows:

[0130] ;

[0131] in, Based on nodes Under the constraints, the line graph undergoes a secondary filtering of adjacency relationships, retaining those with... Edge pairs with the same adjacent nodes Set all others to 0. For nodes and nodes The edges established between them; For nodes and nodes The edges established between them; This is the index of the node. After this operation, if the first... The first line If the nth element is 1, then the nth element is 1. Soon to meet with the first row and number Perform a logical AND operation on rows; row indexes. Index of the corresponding node ; element index Index of the corresponding node If any row contains an element that is 1, the corresponding node will be masked. In .

[0132] Furthermore, to prevent the generated line graph from deviating too much from the original graph due to significant changes in the skeleton, this invention also masks the nodes in the line graph whose entire row of elements becomes zero after the first logical AND operation. In this way, edge swap operations can be restricted to nodes within their 2-hop neighborhood.

[0133] Finally, to further reduce the probability of edge swapping failures, corresponding edges with different value keys are masked. To this end, the present invention also constructs a second mask vector. And set it as follows Each element:

[0134] ;

[0135] in, for The first in One element; , These are nodes in the line graph. ,node The corresponding valence bond values ​​at the edges of the original molecular diagram;

[0136] After all the above masking processes are completed, randomly select and Index of all elements in the middle is 1 As the second node .

[0137] Step 2.1.3: Finally, perform the edge swapping operation to generate a new support structure, thus obtaining the skeleton perturbation diagram. The specific process is as follows:

[0138] In the steps above, two nodes have been selected in the line graph. and Prepare to swap the edges. Then, convert these two nodes into corresponding edges in the support graph, which will be the first edges to be swapped. The second side to be exchanged The ultimate goal of the edge swapping operation is to swap edges on the original molecular graph, so here we need to swap the edges in the scaffold graph. , Corresponding to the original molecular diagram The edges in can be obtained , ,in, , , , Selected edges respectively , The four nodes on; This refers to the properties of chemical bonds. Then, they are broken. , Then reconnect, and obtain the first swapped edge after reconnection. The second swapped edge after reconnection This led to the acquisition of the skeleton perturbation map. .

[0139] Figure 2 An example of a perturbation process of an original molecular diagram framework is given. The original molecular diagram contains carbon, nitrogen, and oxygen atoms, and the different atoms are connected by single and double bonds. Figure 2 During the skeletal perturbation process, the skeletal structure is first extracted (i.e., atoms not extracted in the original molecular diagram are deleted; in the diagram, these unextracted atoms are highlighted and their colors are lightened accordingly). Then, the skeletal structure is converted into a line graph, and a node is randomly selected from the line graph (corresponding to...). Figure 2 The selected atom), then select another node in its two adjacent neighborhoods (corresponding to...). Figure 2 The process of perturbing the skeleton (by selecting two edges separated by one side and reconnecting them as if they were connected by a line) is reflected in the original molecular diagram.

[0140] Step 2.2: Then, functional group perturbation is used to generate a functional group perturbation map. The design goal of functional group perturbation is to reconnect the edges of functional groups in a molecule while maintaining the core molecular skeleton structure. By adjusting the connection patterns of functional groups, new isomers are generated, thereby altering the semantic features of the molecule without destroying the chemical core structure. Functional groups are the parts of a molecule that have specific chemical functions. This method allows for flexible modification of the edge connections of functional groups while preserving core skeleton information, generating new molecular structures. This helps to better describe and compare the similarities between molecules. (Generation of functional group perturbation diagram) The specific process is as follows:

[0141] Step 2.2.1: From the original molecular diagram Randomly select an edge And interrupt it; among them , Selected edges respectively The two nodes on the graph. In a molecular graph, modifying edges leads to changes in the molecular topology, which in turn affects the molecular properties, including the number of hydrogen atoms and the amount of charge. Therefore, the number of hydrogen atoms in the nodes is updated as follows:

[0142] ;

[0143] in, for The updated value; The number of hydrogen atoms at the nodes where chemical bonds are broken; It is the valence state of the fracture edge.

[0144] Step 2.2.2: Then, randomly select two disconnected nodes and connect them to obtain a new edge. ,in , They are the edges The two nodes on the [node name]. Similarly, the number of hydrogen atoms in these nodes should also be updated here:

[0145] ;

[0146] in, for The updated value; The number of hydrogen atoms at the nodes that form chemical bonds;

[0147] Step 2.2.3, Additionally, when When the number is less than 0, the electron count needs to be updated:

[0148] ;

[0149] in, for The updated value; The number of electrons at the nodes that connect chemical bonds;

[0150] Step 2.2.4, let This leads to the functional group perturbation diagram. In real chemistry, the number of hydrogen atoms on an atom must be greater than or equal to zero. It is not permissible for the number of hydrogen atoms to be less than zero when reconnecting edges. In order to maintain the chemical validity of the molecular diagram, it must be "reset to zero".

[0151] Figure 3An example of functional group perturbation process is given for a primitive molecular diagram containing carbon, nitrogen, and oxygen atoms, connected by single and double bonds; Figure 3 In the process of perturbation of functional groups, the functional groups are first extracted (that is, the part remaining except for the atoms that have not been extracted; in the figure, the atoms that have not been extracted are highlighted and the color is lightened accordingly), disconnected from the skeleton, and reconnected in the manner of connecting lines, thus completing the perturbation of functional groups.

[0152] Step 3: Once the two perturbation graphs are obtained, the next problem is how to ensure that the generated graph is a hard negative sample. Therefore, this invention uses a counterfactual mechanism to solve this problem, as this method naturally satisfies the requirements for generating hard negative samples. Both aim to output things that are different at the semantic level but similar at the structural level. This invention ensures that the generated perturbation graph is a counterfactual hard negative sample through the Frobenius norm and the KL divergence function; the specific process is as follows:

[0153] Step 3.1: First, construct a first objective function to maximize the similarity between the original molecular graph and the two perturbation graphs; this objective function attempts to ensure that the perturbations are minimized. Specifically, it utilizes... and , The Frobenius norm of the difference. The smaller the Frobenius norm, the more similar they are. The Frobenius norm of the masking matrix can measure the similarity between features. A relatively large norm indicates that a small portion of feature entries are masked. First objective function Specifically, the Frobenius norm is used to measure the difference in the feature matrices between the original molecular graph and the perturbation graph, and the specific design is as follows:

[0154] ;

[0155] in, It is the Frobenius norm;

[0156] When the Frobenius norm is minimized and the semantic difference is significant, the current perturbation graph is determined to be a counterfactual hard negative sample.

[0157] Step 3.2: After generating counterfactual hard negative samples, it is also necessary to ensure that the generated graph is semantically different from the original graph. Therefore, this invention considers the classification problem and constructs a second objective function. The principle of the second objective function is to minimize the similarity between the probability distributions of the original molecular graph and the perturbation graph. Therefore, the second objective function is designed as follows:

[0158] ;

[0159] in, is the KL divergence function, used to measure the similarity between two probability distributions; This represents a predictor that outputs the probability of the current graph belonging to a certain category.

[0160] Step 3.3: Finally, the overall objective function for generating counterfactual facts from hard negative samples. as follows;

[0161] ;

[0162] Step 4: Generate a natural language description of the molecule using the large language model ChatGPT, and then convert it into a molecular semantic embedding vector using the small language model RoBERTa, which serves as a positive sample in contrastive learning; the specific process is as follows:

[0163] The string "SMILES" is input into the large language model ChatGPT. To enable ChatGPT to generate meaningful natural language descriptions, natural language prompt word templates are developed. , The prompt words guide the ChatGPT large language model to output in a specified style. Based on the input SMILES string and the prompt word template, ChatGPT generates a natural language description containing functional groups, chemical properties, etc. :

[0164] ;

[0165] in, ChatGPT is a large language model. The string is for the molecule SMILES; For natural language prompt word templates;

[0166] Natural language descriptions generated by the large language model ChatGPT Input into the small language model RoBERTa to generate molecular semantic embedding vectors. As positive samples in contrastive learning:

[0167] ;

[0168] in, This is a sequence aggregation function used to map the vector sequence output by the small language model RoBERTa into sentence vectors of fixed dimensions; It is a Transformer-based context representation encoder in the small language model RoBERTa, used to perform multi-layer self-attention computation on the input token sequence and output the context semantic vector representation of each token; This is a text segmentation module used to convert natural language text into a sequence of tokens, build a model vocabulary index corresponding to each token, and output an integer sequence that can be processed by the small language model RoBERTa encoder.

[0169] Figure 4 An example of generating molecular semantic embedding vectors for the large language model ChatGPT is provided. Natural language prompt templates (including instruction and response example templates) are written, and a series of molecular SMILES string representations are given. The SMILES strings and prompt templates are then simultaneously input into the large language model ChatGPT to obtain a response. The response is a natural language description containing functional groups and chemical properties (chemical characteristics). The instruction is: "Suppose you are an expert in the intersection of chemistry and artificial intelligence. Given a series of molecular SMILES representations, your job is..."; the response example template is: "Your response should follow the following format. Functional group: ... Chemical property: ...". The input molecular SMILES string representation is: "(1) CN(C)C(=O)c1 ccc(cc1)OC (2)CS(=O) (=O)C1 ...".

[0170] Step 5: Perform comparative learning among the original molecular graph, positive samples, and counterfactual hard negative samples to maximize the similarity of positive samples and minimize the similarity of counterfactual hard negative samples; for example... Figure 5 As shown, the specific process is as follows:

[0171] This invention specifically employs a dictionary lookup method for comparative learning, given the original molecular diagram. Skeleton perturbation diagram and functional group perturbation diagram Molecular semantic embedding vector and the first encoder With the second encoder Define four embedding formulas and set them in a dictionary (a set of key vectors). Searching for original molecular maps Embedded vector Best matching key The four embedded formulas are as follows:

[0172] ;

[0173] ;

[0174] ;

[0175] ;

[0176] in, Original molecular diagram The embedding vector; This is a molecular semantic embedding vector; skeleton perturbation diagram The embedding vector; Functional group perturbation diagram The embedding vector;

[0177] The objective function for contrastive learning employs InfoNCE loss to maximize the similarity between the original molecular graph and positive samples, and minimize the similarity between the original molecular graph and counterfactual hard negative samples, thereby improving the discriminativeness of the graph embeddings learned during contrastive learning. In this function design, Corresponding to the original molecular diagram, bonds Corresponding to positive samples, and Corresponding to counterfactual hard negative samples. The InfoNCE loss is as follows:

[0178] ;

[0179] in, Loss due to InfoNCE; It is an exponential function; Similarity; This refers to temperature hyperparameters. It is a dictionary containing positive samples and all counterfactual hard negative samples, used to calculate the InfoNCE loss; For the first A counterfactual hard negative sample.

[0180] Step 6: Fuse the original molecular map embedding obtained after contrastive learning with the molecular semantic embedding to predict downstream molecular properties. The specific process is as follows: After contrastive learning is completed, and The similarity is maximized. First, and The molecules are fused to generate a fused molecular representation. Then, a randomly initialized multilayer perceptron (MLP) network maps the fused molecular representation to a specific molecular property prediction space. Finally, its labels are used as training supervision signals for fine-tuning through supervised learning.

[0181] The formula for generating the fused molecular representation is:

[0182] ;

[0183] ;

[0184] ;

[0185] in, To compare the original molecular diagram after learning The embedding vector; To compare the learned molecular semantic embedding vectors; The fused molecules are represented as follows; , These are the weighting coefficients of the molecular semantic embedding vector and the original molecular graph embedding vector, respectively, satisfying... ;

[0186] The fused molecular representation is then input into the MLP network to perform the molecular property prediction task.

[0187] ;

[0188] ;

[0189] ;

[0190] in, As the first intermediate variable; It is the second intermediate variable; This indicates the predicted results of molecular properties; It is the ReLU activation function; It is a linear mapping function;

[0191] Define the total loss function for supervised learning. for:

[0192] ;

[0193] in, As weight; To predict the loss based on properties, the formula is:

[0194] ;

[0195] in, For the first The true molecular properties of each data point; Indicates the number of data points in the dataset; For the first The molecular property prediction results for the data.

[0196] To demonstrate the feasibility and superiority of this invention, the following comparative experiments are presented.

[0197] The experimental datasets selected were BBBP and BACE. BBBP is the blood-brain barrier permeability dataset. In developing drugs acting on the central nervous system, compounds must be able to penetrate this barrier; however, for non-central nervous system drugs, penetration can lead to side effects. Therefore, the BBBP dataset is used to predict compound permeability and is a key indicator for evaluating the model's early drug screening capabilities. BACE is the β-secretase 1 inhibitor binding affinity dataset. This dataset provides labels indicating whether a compound has inhibitory activity against the target (i.e., whether it can bind to it), and is often used to evaluate the model's accuracy in drug target identification.

[0198] The evaluation metric selected is ROC-AUC for classification tasks; a higher ROC-AUC value indicates more accurate prediction results. Twelve existing models were compared with the method of this invention (the prediction model formed by the method of this invention is named CounterMol-LLM), and the specific comparison results are shown in Table 1. The 12 existing models are as follows:

[0199] SSLP: A self-supervised learning pre-training paradigm designed to learn general molecular representations using unlabeled data;

[0200] GraphConv: Generalizes convolution operations from regular grids (such as images) to graph-structured data, updating node representations by aggregating neighbor information;

[0201] Weave: Learnable pairwise atom aggregation on molecular diagrams, rather than just single-atom neighbor aggregation, to simulate interatomic interactions in greater detail.

[0202] SchNet: Specifically designed for atomic systems (molecules / materials), it directly processes three-dimensional atomic coordinates through continuous filtering convolutional layers, maintaining invariance to translation, rotation, etc.

[0203] MPNN: Proposes a unified message-passing neural network framework that summarizes many GNNs as an iterative process of "message passing-aggregation-update";

[0204] MGCN: Extends Graph Convolutional Networks (GCN) to multi-layer graphs, handling node and edge relationships both within and between layers;

[0205] GROVER: Combining the Transformer architecture with message passing networks, it learns rich information from massive unlabeled molecules through self-supervised tasks at the node, edge, and graph levels.

[0206] MolCLR: A self-supervised representation learning framework for molecular graphs based on contrastive learning, which allows the model to learn invariant features by applying different enhancements to the same molecule.

[0207] GraphCL: A general graph contrastive learning framework that generates enhanced views by randomly perturbing the local subgraphs of nodes and maximizes the consistency between different views of the same node.

[0208] MoMu: A molecular-text multimodal model designed to align the representations of chemical molecular structures (as shown in the figure) with natural language descriptions in a shared semantic space;

[0209] CaR: Molecular representation learning based on causal inference, which aims to identify substructures that are causally related to molecular properties from data, rather than just related features;

[0210] Graph2Token: A method to directly convert graph structures (such as molecules) into sequence tokens for processing using powerful sequence models (such as Transformer).

[0211] The first nine models mentioned above are methods that do not incorporate a large language model, while the last three models and the model of this invention are methods that incorporate a large language model.

[0212] Table 1. Comparison of ROC-AUC results between the present invention and different models on classification tasks.

[0213] .

[0214] As can be seen from Table 1, the CounterMol-LLM model of this invention performs best on both datasets.

[0215] Of course, the above description is not intended to limit the present invention, and the present invention is not limited to the examples given above. Any changes, modifications, additions or substitutions made by those skilled in the art within the scope of the present invention should also fall within the protection scope of the present invention.

Claims

1. A counterfactual and large language model-based contrastive learning molecular property prediction method, characterized in that, Includes the following steps: Step 1: Generate the original molecular diagram using the molecular SMILES string; Step 2: Based on the original molecular map, generate the skeleton perturbation map and the functional group perturbation map respectively using skeleton perturbation and functional group perturbation; the specific process is as follows: Step 2.1: First, generate a skeleton perturbation map using skeleton perturbation; the specific process is as follows: Step 2.1.1: Skeletal perturbation from the original molecular map Extract the scaffold diagram and convert it into a line graph; specifically, use the open-source toolkit RDKit to extract the original molecular diagram. bracket diagram And turn it into a line graph ;in, , , These are the adjacency matrix, node feature matrix, and edge weight matrix of the line graph; the original molecular graph. Each edge in Online chart A node corresponds to a middle node. , , These represent the indices of the two atomic nodes connected by a chemical bond; if the original molecular diagram... The edge in and edge If there is a common node between them, then the line graph Edges are generated between corresponding nodes. ; Indicates the index of the atomic nodes connected by a chemical bond; Step 2.1.2: Select two nodes from the line graph and map them to two edges of the support graph; Step 2.1.3: Perform the edge swapping operation to generate a new support structure and thus obtain the skeleton perturbation diagram; Step 2.2: Next, functional group perturbation is used to generate a functional group perturbation map; the specific process is as follows: Step 2.2.1: From the original molecular diagram Randomly select an edge And interrupt it; among them , Selected edges respectively Two nodes on; This relates to the properties of chemical bonds; it updates the number of hydrogen atoms at nodes where chemical bonds are broken. ； in, for The updated value; The number of hydrogen atoms at the nodes where chemical bonds are broken; It is the valence state of the fracture edge; Step 2.2.2: Randomly select two disconnected nodes and connect them to obtain a new edge. ,in , They are the edges The two nodes on the graph; update the number of hydrogen atoms in the nodes connected by chemical bonds: ； in, for The updated value; The number of hydrogen atoms at the nodes that form chemical bonds; Step 2.2.3, when When the value is less than 0, the number of electrons in the nodes connecting chemical bonds is updated: ； in, for The updated value; The number of electrons at the nodes that connect chemical bonds; Step 2.2.4, let This leads to the functional group perturbation diagram. ; Step 3: Ensure that the generated perturbation map is a counterfactual hard negative sample by using the Frobenius norm and KL divergence function; Step 4: Generate a natural language description of the molecule using the large language model ChatGPT, and convert it into a molecular semantic embedding vector using the small language model RoBERTa, which serves as a positive sample in the contrastive learning. Step 5: Perform comparative learning between the original molecular graph, positive samples, and counterfactual hard negative samples to maximize the similarity of positive samples and minimize the similarity of counterfactual hard negative samples; Step 6: Fuse the original molecular graph embedding obtained after contrastive learning with the molecular semantic embedding to predict downstream molecular properties.

2. The method for predicting molecular properties based on counterfactual and large language models according to claim 1, characterized in that, The specific process of step 1 is as follows: Step 1.1: Obtain the SMILES string of the molecule and input it into the chemical information parsing module for molecular topological reconstruction; the chemical information parsing module parses the string character by character according to the syntax rules of SMILES, identifies the atomic symbols, branched structures, ring markers, aromaticity symbols and bond type symbols, and reconstructs the two-dimensional topological structure of the molecule. Step 1.2: After completing the molecular topological reconstruction, traverse all atoms and construct a set of nodes in the graph based on their chemical properties. Extract the following descriptive information for each atom as node features: atom type, atomic number, formal charge, valence state, hybridization mode, aromaticity, and chiral label. Each atom is a graph node, and the above features are used to construct a node feature vector, thereby forming the node feature matrix of the graph. Step 1.3: In the molecular structure, if there is a chemical bond between any two atoms, there will be a corresponding edge in the molecular diagram; Step 1.4: Extract the structural and property information of each chemical bond to form edge features, including: bond type, bond direction, whether the bond belongs to a ring structure, and whether it is a conjugated bond; the edge features are organized in the form of feature vectors to form an edge feature matrix; by traversing all chemical bonds, construct an edge set and record the numbers of the two atoms connected by the bond to form the topological structure of the graph; Step 1.5: Construct a complete original molecular graph using the node set, edge set, and corresponding node and edge features. : ； in, It is a set of nodes consisting of all atoms; It is the set of edges consisting of all chemical bonds; The node feature matrix; Let be the edge feature matrix.

3. The method for predicting molecular properties based on counterfactual and large language models according to claim 1, characterized in that, The specific process of step 2.1.2 is as follows: After the line chart is generated, first in Randomly select a node Then, to reduce the search space, we introduce... The adjacency matrix, Each element in the adjacency matrix is ​​represented as: ； in, For nodes and nodes The edges established between them; , For indexes of different nodes; The operation of reducing the search space is divided into the following steps: First, perform a masking operation on the nodes using the adjacency matrix of the line graph; select For the first node used for edge swapping, introduce a first mask vector. Set it up as follows Each element: ； in, for The first in 1 element, element index Index of the corresponding node ; for The number of nodes in; Then, for the adjacency matrix of the line graph... Perform a logical AND operation between each row and every other row; row index. Index of the corresponding node Each element of the adjacency matrix of the line graph is re-represented as follows: ； in, Based on nodes Under the constraints, the line graph undergoes a secondary filtering of adjacency relationships, retaining those with... Edge pairs with the same adjacent nodes Set all others to 0; For nodes and nodes The edges established between them; For nodes and nodes The edges established between them; For the index of the node; if the first The first line If the nth element is 1, then the nth element is 1. Soon to meet with the first row and number Perform a logical AND operation on rows; row indexes. Index of the corresponding node ; element index Index of the corresponding node If any row contains an element that is 1, the corresponding node will be masked. In ; Finally, the corresponding edges with different valence key values ​​are masked; a second mask vector is constructed. ,set up Each element is as follows: ； in, for The first in One element; , These are nodes in the line graph. ,node The corresponding valence bond values ​​at the edges of the original molecular diagram; After all the above masking processes are completed, randomly select and Index of all elements in the middle As the second node .

4. The contrastive learning molecular property prediction method based on counterfactual and large language models according to claim 3, characterized in that, The specific process of step 2.1.3 is as follows: Will and These two nodes are converted into corresponding edges in the scaffold graph, which are the first edges to be swapped. The second side to be exchanged ; the bracket diagram , Corresponding to the original molecular diagram The edges in the middle, that is, the edges in the middle, are obtained , ,in, , , , Selected edges respectively , The four nodes on; then, disconnect. , Then reconnect, and obtain the first swapped edge after reconnection. The second swapped edge after reconnection This leads to the skeleton perturbation map. .

5. The contrastive learning molecular property prediction method based on counterfactual and large language models according to claim 4, characterized in that, The specific process of step 3 is as follows: Step 3.1: Construct a first objective function to maximize the similarity between the original molecular graph and the two perturbation graphs; the first objective function... The design is as follows: ； in, It is the Frobenius norm; Step 3.2: Construct a second objective function to minimize the similarity between the probability distributions of the original molecular graph and the perturbation graph; the second objective function... The design is as follows: ； in, is the KL divergence function, used to measure the similarity between two probability distributions; This indicates a predictor that outputs the probability of the current graph belonging to a specific category. Step 3.3: Overall objective function for generating counterfactual facts of hard negative samples as follows; 。 6. The contrastive learning molecular property prediction method based on counterfactual and large language models according to claim 5, characterized in that, The specific process of step 4 is as follows: The SMILES string is input into the ChatGPT large language model; a natural language prompt template is written to guide ChatGPT to output in a specified style. ChatGPT generates a natural language description based on the input SMILES string and the prompt template. : ； in, ChatGPT is a large language model. The string is for the molecule SMILES; For natural language prompt word templates; Natural language descriptions generated by the large language model ChatGPT Input into the small language model RoBERTa to generate molecular semantic embedding vectors. As positive samples in contrastive learning: ； in, For sequence aggregation functions; This is a Transformer-based context representation encoder in the small language model RoBERTa; This is the text segmentation module.

7. The contrastive learning molecular property prediction method based on counterfactual and large language models according to claim 6, characterized in that, The specific process of step 5 is as follows: Comparative learning is performed using a dictionary lookup method, given the original molecular diagram. Skeleton perturbation diagram and functional group perturbation diagram Molecular semantic embedding vector and the first encoder With the second encoder Define four embedded formulas and in the dictionary Searching for original molecular maps Embedded vector Best matching key The four embedded formulas are as follows: ； ； ； ； in, Original molecular diagram The embedding vector; For molecular semantic embedding vectors; skeleton perturbation diagram The embedding vector; Functional group perturbation diagram The embedding vector; The objective function for contrastive learning employs InfoNCE loss to maximize the similarity between the original molecular graph and positive samples, and minimize the similarity between the original molecular graph and counterfactual hard negative samples. The InfoNCE loss is as follows: ； in, Loss due to InfoNCE; It is an exponential function; Similarity; This refers to temperature hyperparameters. For the first A counterfactual hard negative sample.

8. The contrastive learning molecular property prediction method based on counterfactual and large language models according to claim 7, characterized in that, The specific process of step 6 is as follows: After comparative learning is completed, and The similarity reaches its maximum; first, and The fusion process is performed to generate the fused molecular representation: ； ； ； in, To compare the original molecular diagram after learning The embedding vector; To compare the learned molecular semantic embedding vectors; The fused molecules are represented as follows; , These are the weighting coefficients of the molecular semantic embedding vector and the original molecular graph embedding vector, respectively, satisfying... ; The fused molecular representation is then input into a randomly initialized multilayer perceptron network to perform a molecular property prediction task. ； ； ； in, As the first intermediate variable; It is the second intermediate variable; This indicates the predicted results of molecular properties; It is the ReLU activation function; It is a linear mapping function; Finally, the predicted molecular properties are used as training supervision signals for fine-tuning through supervised learning; the total loss function for supervised learning is defined. for: ； in, As weight; To predict the loss based on properties, the formula is: ； in, For the first The true molecular properties of each data point; Indicates the number of data points in the dataset; For the first The molecular property prediction results for the data.

Images