Molecular toxicity detection method combined with chemical language model
By combining chemical language models and graph isomorphic networks, the problem of structural and semantic separation in molecular toxicity detection models is solved, achieving cross-modal feature alignment and semantic fusion, thereby improving the accuracy and stability of molecular toxicity detection.
Patent Information
- Application Number
- CN202511778767.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-28
- Publication Date
- 2026-02-27
AI Technical Summary
Existing molecular toxicity detection models suffer from problems such as structural and semantic separation, weak knowledge transferability, and poor model generalization ability.
By combining a chemical language model, a corpus is formed by acquiring the SMILES sequences of molecules and chemical text descriptions. This corpus is then transformed into a molecular graph using RDKit. Embedding learning is performed through a graph isomorphic network and a chemical language model to calculate similarity scores and KL divergence to share knowledge, optimize model parameters, and achieve cross-modal feature alignment and semantic fusion.
It significantly improves the quality of molecular representation, enhances the accuracy and stability of molecular toxicity detection, and strengthens its performance in drug screening and chemical property prediction.
Smart Images
Figure CN121583398A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the field of molecular characterization learning and toxicity detection, and particularly relates to a molecular toxicity detection method combined with a chemical language model. BACKGROUND
[0002] Data-driven solutions have been one of the main research directions in many biomedical and chemical tasks, such as new drug discovery, molecular toxicity detection, and new molecule design. With the exponential growth of chemical data and the rapid development of deep learning technology, researchers have begun to use deep neural networks to automatically model the association between molecular structure, properties, and biological activity, providing a new approach to molecular toxicity evaluation.
[0003] Unlike traditional machine learning methods, deep learning can automatically learn the potential vector representation of data through a multi-layer nonlinear network structure, thereby capturing complex molecular semantics and structural features. Molecular representation learning mainly focuses on two methods: one is based on molecular sequence data representation, such as Simplified Molecular Input Line Entry System (SMILES) or International Chemical Identifier (InChI); the other is to regard molecules as two-dimensional graph structure data and use Graph Neural Network (GNN) to extract their topological structure features.
[0004] SMILES-based sequence models can learn the symbolic patterns of molecules, but lack an understanding of real chemical structures and physical properties; while graph structure models capture topological relationships, they cannot express semantic knowledge at the chemical language level. The information of the two modalities is independent of each other, making it difficult to achieve unified modeling. In addition, the grammatical order of molecular SMILES sequences does not completely match the actual chemical bond connection rules, making it difficult for sequence models to accurately reflect the local chemical structure of molecules. Especially in the context of scarce labeled data, these deep models are prone to overfitting and poor robustness. Therefore, there is an urgent need for a new molecular toxicity detection method that can simultaneously integrate chemical language semantics and molecular structure topological information. By using a chemical language model to learn general chemical rules and semantic associations from a large-scale chemical corpus, and integrating this knowledge to improve the quality of molecular representation, better toxicity detection performance can be achieved. SUMMARY
[0005] The present application aims to provide a molecular toxicity detection method combined with a chemical language model to solve the problems of structural and semantic fragmentation, weak knowledge transfer, and poor model generalization in existing molecular toxicity detection models.
[0006] To achieve the above purpose, the present application provides the following method:
[0007] The molecular toxicity detection method combined with a chemical language model provided by the present application is:
[0008] S1: Obtain a SMILES sequence of a molecule and a corresponding chemical text description to form a chemical corpus, and convert the SMILES sequence into a molecular graph by using an analysis tool RDKit;
[0009] S2: Select a general T5 language model for initialization, train the T5 language model on the chemical corpus by using a mask recombination and structure denoising algorithm to learn grammar rules and chemical dependency relationships, and obtain a chemical language model;
[0010] S3: Calculate the similarity of a target molecule with samples in a database by using Morgan fingerprints to obtain context prompt information;
[0011] S4: Use the context information to perform molecular annotation to generate a corresponding chemical text description for the molecular graph;
[0012] S5: Perform embedding learning on the chemical text description and the molecular graph by using a graph isomorphism network and the chemical language model, respectively, to obtain text feature representation and structure feature representation of the molecule;
[0013] S6: Align the two cross-modal features of the molecular text and the molecular structure, calculate the similarity score between the text feature representation and the structure feature representation of the molecule, and obtain a contrastive alignment loss according to a target distribution;
[0014] S7: Use the class-level representation of the text to guide structure feature learning, calculate the KL divergence between the text feature representation and the structure feature representation of the molecule to share knowledge and semantics;
[0015] S8: Jointly train the graph isomorphism neural network and the chemical language model by using the similarity score, the contrastive alignment loss, and the KL divergence to share knowledge and semantics, obtain an overall model, and optimize the parameters of the overall model;
[0016] S9: Detect the toxicity of a biological molecule by using the trained overall model.
[0017] Preferably, a general T5 language model is selected for initialization, and the formula for mask recombination on the chemical corpus is:
[0018] ;
[0019] wherein, represents the T5 language model after initialization, represents a molecular SMILES data set in one training, and are mask set residual components, respectively, represents a weight factor, represents the model in the parameter the probability value.
[0020] Preferably, structured noise is introduced into the chemical text description, including synonym replacement, order disturbance and character-level spelling noise, and the T5 language model is further trained using a structure denoising algorithm, as follows:
[0021] ;
[0022] wherein, denotes the T5 language model after the structure denoising algorithm, denotes the chemical text corpus, denotes the noise text, is a noise vector, is a prediction function of the continuous feature layer, is a discretization reconstruction function, denotes the cross-entropy loss, is a trade-off parameter;
[0023] Adjusting the T5 language model to adapt to the chemical field is completed by the mask reorganization and the structure denoising algorithm, as follows:
[0024] .
[0025] wherein, denotes the adjusted T5 language model, and denote the loss trade-off parameter, is a regularization constraint term for model parameters.
[0026] Preferably, the similarity of the target molecule with the samples in the database is calculated using the Morgan fingerprint to obtain the context prompt information, as follows:
[0027] .
[0028] wherein, is a similarity calculation function, denotes the binary Morgan fingerprint of a molecule, and denote the training molecule set and the query molecule, respectively.
[0029] Preferably, the context information is used for molecular annotation to generate the corresponding chemical text for the molecule, and the overall optimization loss is calculated as follows:
[0030] ;
[0031] wherein L ICMC () denotes the overall optimization loss, parameters of a language model, a binary Morgan fingerprint of a molecule, context information of retrieved similar instances, aggregated context information loss, probability value of the model under parameters .
[0032] Preferably, the chemical text description and the molecular graph are respectively embedded by a graph isomorphism network and the chemical language model, and the formula for obtaining the text feature representation and the structure feature representation of the molecule is:
[0033] ;
[0034] ;
[0035] ;
[0036] wherein, is a learnable vector, which is projected by a linear projection to calculate the importance score of each token ,() represents summing by row, represents a broadcast mechanism. Preferably, the graph isomorphism network is composed of k layers, adopts ReLU(.) as an activation function, and the final output of the structure feature representation of the molecule
[0037] is 256-dimensional, and the calculation formula is as follows:
[0038] ;
[0039] ;
[0040] wherein, and represent a concatenation operation and a readout function, respectively, represents the feature representation of the node in the i-th layer, represents the adjacent node of , is a learnable parameter, represents a multi-layer perceptron, and the initial node feature is given by .
[0041] Preferably, the similarity score between the text feature representation and the structure feature representation of the molecule is calculated by aligning the two cross-modal features of the molecular text and the molecular structure, The final contrast alignment loss formula is obtained is:
[0042]
[0043]
[0044] wherein Softmax() is used for calculating the similarity probability value, and respectively, are the loss weighting parameters and the temperature parameters of contrast learning.
[0045] Preferably, the class-level representation of the text is used to guide the structural features learning, by calculating the KL divergence between the probability distribution of the molecular text features and the probability distribution of the structural features The sharing of knowledge and semantics can be realized:
[0046]
[0047]
[0048] .
[0049] wherein, denotes the temperature hyperparameter, denotes vector splicing.
[0050] Preferably, the formula for optimizing the parameters of the overall model is:
[0051] ;
[0052] wherein respectively, are the generation task loss and the mutual information loss, and is the hyperparameter for balancing the two losses.
[0053] The beneficial effects of the present application are embodied in: by introducing the text knowledge of the language model and combining with the molecular structure information; first, the structural rules and chemical semantic information of the molecule are learned by using multi-objective self-supervised training; second, in order to strengthen the association between the molecular structure and the semantics, a context-driven molecular description strategy based on similarity retrieval is proposed; cross-modal migration and fusion of semantic knowledge are realized through contrast feature alignment and knowledge distillation; by comprehensively utilizing chemical language semantic information and molecular structure features, the quality of the molecular representation learned by the model is significantly improved, which can obtain higher accuracy and stability in molecular toxicity detection, drug screening and chemical property prediction and other various downstream tasks. BRIEF DESCRIPTION OF DRAWINGS
[0054] In order to more clearly illustrate the technical solutions in the specific embodiments or the prior art of the present application, the drawings required to be used in the specific embodiments or the prior art description will be briefly introduced below. In all the drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, each element or part is not necessarily drawn according to the actual proportion.
[0055] Figure 1 A flowchart of a molecular toxicity detection method combined with a chemical language model provided for an embodiment of the present application is shown in the figure.
[0056] Figure 2 A chemical language model training schematic diagram used for an embodiment of the present application is shown in the figure.
[0057] Figure 3 A modal alignment schematic diagram used for an embodiment of the present application is shown in the figure. DETAILED DESCRIPTION
[0058] In order to make the person skilled in the art better understand the technical solutions of the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by the person skilled in the art without creative labor fall within the scope of protection of the present application.
[0059] The terms "first", "second", and the like in the specification and claims of the present application and the above-mentioned drawings are used to distinguish different objects, not to describe a specific order. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, device, product, or end including a series of steps or units is not limited to the listed steps or units, but can optionally include steps or units not listed, or can optionally include other steps or units inherent to the process, method, product, or end.
[0060] In this document, the term "embodiment" means that the specific features, structures, or characteristics described in connection with the embodiment can be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily mean the same embodiment, nor is it independent or alternative to other embodiments. The person skilled in the art explicitly and implicitly understands that the embodiments described herein can be combined with other embodiments.
[0061] Deep learning can automatically learn the latent vector representation of data through multi-layer nonlinear network structure, so as to capture complex molecular semantic and structural features. Molecular representation learning mainly focuses on two methods: one is based on the representation of molecular sequence data, such as simplified molecular input line entry system (SMILES) or international chemical identifier (InChI); the other is to regard the molecule as two-dimensional graph structure data, and use graph neural network (GNN) to extract its topological structure features.
[0062] The sequence model based on SMILES can learn the symbolic pattern of the molecule, but lacks the understanding of the real chemical structure and physical properties; while the graph structure model based on SMILES can capture the topological relationship, but cannot express the semantic knowledge of the chemical language. The information of the two modalities is independent of each other, and it is difficult to realize unified modeling. In addition, the grammatical order of the molecular SMILES sequence is not completely consistent with the actual chemical bond connection rule, which leads to the difficulty of accurately reflecting the local chemical structure of the molecule by the latent representation generated by the sequence model. Especially in the scene of scarce labeled data, these deep models are prone to overfitting and poor robustness. Therefore, a new molecular toxicity detection method is needed, which can simultaneously fuse the chemical language semantics and the molecular structure topological information. The chemical language model is used to learn the general chemical rules and semantic associations from a large-scale chemical corpus, and these knowledge is fused to improve the quality of molecular representation and obtain better toxicity detection performance.
[0063] The present application aims to provide a molecular toxicity detection method combined with a chemical language model, to solve the problems of structural and semantic split, weak knowledge transfer, and poor model generalization ability in the prior art molecular toxicity detection model.
[0064] As shown in Figure 1 The embodiment of the present application provides a molecular toxicity detection method combined with a chemical language model, comprising the following steps:
[0065] S1: Obtain the SMILES sequence of the molecule and the corresponding chemical text description to form a chemical corpus, and convert the SMILES sequence into a molecular graph by using the analysis tool RDKit.
[0066] S2, select a general T5 language model for initialization, and train the T5 language model on the chemical corpus by using the mask recombination and structure denoising algorithm to learn the grammar rules and chemical dependency relationship, to obtain a chemical language model.
[0067] In the embodiment of the present application, the formula for mask recombination on the chemical corpus is:
[0068] ;
[0069] Wherein, represents the initialized T5 language model, denotes the SMILES dataset of molecules in the first training, and are the mask set residual components, denotes the weight factor, denotes the probability value of the model under the parameters ; structured noise is introduced into the chemical text description, including synonym replacement, order disturbance and character-level spelling noise, and the T5 language model is further trained by using a structure denoising algorithm, and the formula is:
[0070] .
[0071] wherein, denotes the T5 language model after the structure denoising algorithm, denotes a chemical text corpus, denotes a noisy text, is a noise vector, is a prediction function of a continuous feature layer, is a discretization reconstruction function, denotes a cross-entropy loss, is a trade-off parameter; the T5 language model is adjusted to adapt to the chemical field by using a mask reorganization and a structure denoising algorithm, and the formula is:
[0072] .
[0073] wherein, denotes the adjusted T5 language model, and denote loss trade-off parameters, is a regularization constraint term for model parameters.
[0074] S3, calculate the similarity of the Morgan fingerprint of the target molecule with the samples in the database to obtain context prompt information.
[0075] In the embodiments of the present application, the formula for obtaining the context prompt information by calculating the similarity of the Morgan fingerprint of the target molecule with the samples in the database is:
[0076] .
[0077] wherein, is a similarity calculation function, denotes the binary Morgan fingerprint of a molecule, and denote the training molecule set and the query molecule, respectively.
[0078] S4, use the context information to label the molecules, and generate corresponding chemical text descriptions for the molecule graphs.
[0079] In the embodiments of the present application, the molecular annotation is performed by using the context information, and the corresponding chemical text is generated for the molecule, and the overall optimization loss is calculated as follows:
[0080] ;
[0081] Wherein L ICMC () represents the overall optimization loss, represents the parameters of the language model, represents the binary Morgan fingerprint of the molecule, wherein represents the context information of the retrieved similar instance, represents the aggregated context information loss, represents the probability value of the model under the parameters .
[0082] S5, the chemical text description and the molecular graph are embedded and learned by the graph isomorphism network and the chemical language model respectively, and the text feature representation and the structure feature representation of the molecule are obtained.
[0083] In the embodiments of the present application, the chemical text description and the molecular graph are embedded and learned by the graph isomorphism network and the chemical language model respectively, and the formula for obtaining the text feature representation and the structure feature representation of the molecule is as follows:
[0084] ;
[0085] ;
[0086] ;
[0087] Wherein, represents a learnable vector, which is calculated by linear projection to calculate the importance score of each word segmentation , () represents the sum by row, represents the broadcast mechanism; the graph isomorphism network is composed of k layers, adopts ReLU(.) as the activation function, and the structure feature representation of the molecule finally output is 256 dimensions, and the calculation formula is as follows:
[0088] ;
[0089] ;
[0090] Wherein, and represent the splicing operation and the readout function respectively, represents the node of the layer characterized by, denotes a neighboring node of, is a learnable parameter, denotes a multi-layer perceptron, the initial node features are given by .
[0091] S6, aligning the two cross-modal features of molecular text and molecular structure, calculating the similarity score between the text feature representation and the structure feature representation of the molecule and the target distribution to obtain a contrastive alignment loss.
[0092] In the embodiment of the application, the two cross-modal features of molecular text and molecular structure are aligned, the similarity score between the text feature representation and the structure feature representation of the molecule is calculated and the target distribution to obtain the final contrastive alignment loss formula :
[0093]
[0094]
[0095] wherein Softmax() is used to calculate the similarity probability value, and are the loss weighting parameters and the temperature parameters of contrastive learning, respectively;
[0096] The class-level embedding vector output by the chemical language model is used as a teacher signal to guide the learning of the structure feature , and two projection heads are constructed to generate prediction vectors and :
[0097] ;
[0098] ;
[0099] wherein, and denote the corresponding weight vector and bias term, respectively.
[0100] S7, using the class-level representation of the text to guide the learning of the structure feature, calculating the KL divergence between the text feature representation and the structure feature representation of the molecule to share knowledge and semantics.
[0101] In the embodiment of the application, the class-level representation of the text is used to guide the learning of the structure feature , and the probability distribution of the molecular text feature is calculated to guide the learning of the structure feature The KL divergence between the probability distribution of the structure features The KL divergence between the probability distribution of the structure features
[0102]
[0103]
[0104] .
[0105] wherein, denotes a temperature hyperparameter, denotes vector concatenation.
[0106] S8, share knowledge and semantics through similarity score, contrast alignment loss and KL divergence, and jointly train the graph isomorphism neural network and the chemical language model to obtain an overall model, and optimize the parameters of the overall model.
[0107] In the embodiment of the application, the formula for optimizing the parameters of the overall model is:
[0108] ;
[0109] wherein are the generation task loss and the mutual information loss, respectively, and is a hyperparameter for balancing the two losses.
[0110] S9, adopt the trained overall model to detect the toxicity of biomolecules.
[0111] The beneficial effects of the application are embodied in: by introducing the text knowledge of the language model and combining the molecular structure information; first, the structure rules and chemical semantic information of the molecule are learned by using multi-objective self-supervised training; second, in order to strengthen the association between the molecular structure and the semantics, a context-driven molecular description strategy based on similarity retrieval is proposed; cross-modal migration and fusion of semantic knowledge are achieved through contrast feature alignment and knowledge distillation; by comprehensively utilizing the chemical language semantic information and the molecular structure features, the quality of the molecular representation learned by the model is significantly improved, so that it can obtain higher accuracy and stability in various downstream tasks such as molecular toxicity detection, drug screening and chemical property prediction.
[0112] The above-mentioned are only embodiments of the present application, and common technical solutions or characteristics in the scheme are not described in detail here; it should be pointed out that for those skilled in the art, without departing from the scheme of the present application, a number of modifications and improvements can be made, which should also be considered as the protection scope of the present application, which will not affect the effect and practicality of the patent. The protection scope claimed in this application should be subject to the content of its claims, and the specific implementation mode and the like recorded in the specification can be used to explain the content of the claims.
Claims
1. A method for detecting molecular toxicity in combination with a chemical language model, characterized by, The method comprises: S1: obtaining a SMILES sequence of a molecule and a corresponding chemical text description to form a chemical corpus, and converting the SMILES sequence into a molecular graph by using an analysis tool RDKit; S2, selecting a general T5 language model for initialization, and training the T5 language model on the chemical corpus by using a mask reorganization and a structure denoising algorithm to learn grammar rules and chemical dependency relationships, to obtain a chemical language model; S3, calculating the similarity of the Morgan fingerprint of the target molecule with samples in a database to obtain context prompt information; S4, using the context information to perform molecular annotation to generate a corresponding chemical text description for the molecular graph; S5, embedding learning the chemical text description and the molecular graph by using a graph isomorphism network and the chemical language model respectively to obtain text feature representation and structure feature representation of the molecule; S6, aligning the two cross-modal features of the molecular text and the molecular structure, calculating the similarity score and the target distribution between the text feature representation and the structure feature representation of the molecule to obtain a contrast alignment loss; S7, using the class-level representation of the text to guide structure feature learning, calculating the KL divergence between the text feature representation and the structure feature representation of the molecule to share knowledge and semantics; S8, jointly training the graph isomorphism neural network and the chemical language model by using the similarity score, the contrast alignment loss and the KL divergence sharing knowledge and semantics to obtain an overall model, and optimizing the parameters of the overall model; S9, using the trained overall model to detect the toxicity of biological molecules.
2. The molecular toxicity detection method combined with a chemical language model according to claim 1, wherein: The general T5 language model is initialized, and the formula for mask reorganization on the chemical corpus is: ； wherein, denotes the initialized T5 language model, denotes the molecular SMILES dataset in one training, and are mask set residual components, denotes a weight factor, denotes the probability value of the model under the parameters .
3. The molecular toxicity detection method combined with a chemical language model according to claim 2, wherein: Structured noise is introduced into the chemical text description, including synonym replacement, order disturbance and character-level spelling noise, and the T5 language model is further trained by using a structure denoising algorithm, and the formula is: ； wherein, denotes the T5 language model after the structure denoising algorithm, denotes a chemical text corpus, denotes a noisy text, is a noise vector, is a prediction function of a continuous feature layer, is a discretization reconstruction function, denotes a cross-entropy loss, is a trade-off parameter; The T5 language model is adjusted by using the mask reorganization and the structure denoising algorithm to adapt to the chemical field, and the formula is: 。 wherein, denotes the adjusted T5 language model, and denotes a loss trade-off parameter, is a regularization constraint term on the model parameters.
4. The molecular toxicity detection method combined with a chemical language model according to claim 1, wherein: The formula for calculating the similarity of the Morgan fingerprint of the target molecule with samples in a database to obtain context prompt information is: 。 wherein, is a similarity calculation function, denotes a binary Morgan fingerprint of a molecule, and denotes a training set of molecules and a query molecule, respectively.
5. The molecular toxicity detection method combined with a chemical language model according to claim 4, wherein: The context information is used for molecular annotation to generate a corresponding chemical text for the molecule, and the overall optimization loss is calculated as follows: ； where L ICMC represents the overall optimization loss, represents the parameters of the language model, represents the binary Morgan fingerprint of the molecule, where represents the context information of the retrieved similar instances, represents the aggregated context information loss, represents the probability value of the model at parameters .
6. The molecular toxicity detection method combined with a chemical language model according to claim 1, wherein: The formula for embedding learning the chemical text description and the molecular graph by using a graph isomorphism network and the chemical language model respectively to obtain text feature representation and structure feature representation of the molecule is: ； ； ； wherein, denotes a learnable vector that is projected linearly computes the importance score of each token denotes a sum over rows, denotes a broadcast mechanism. 7. The method of claim 1, wherein the method further comprises: The graph isomorphism network consists of k layers, adopts ReLU(.) as an activation function, and finally outputs a structure feature representation of a molecule is 256-dimensional, and the calculation formula is as follows: ； ； where, and denote concatenation operation and readout function, respectively, denotes the layer node feature representation, denotes adjacent node, is a learnable parameter, denotes a multi-layer perceptron with initial node features given by .
8. The method of claim 1, wherein the method further comprises: aligning the two cross-modal features of molecular text and molecular structure, computing a similarity score between the text feature representation and the structure feature representation of the molecule and the target distribution resulting in a final contrastive alignment loss formula is: where Softmax() is used to calculate the similarity probability value, and are the loss trade-off parameter and the temperature parameter for contrastive learning, respectively.
9. The method of claim 8, wherein the method further comprises: Utilizing class-level representations of text Guiding structural features Learning, by computing the KL divergence between the probability distribution of the molecular text features and the probability distribution of the structural features allows for sharing of knowledge and semantics: 。 wherein, denotes a temperature hyperparameter, denotes vector concatenation.
10. The method of claim 9, wherein the method further comprises: The formula for optimizing the parameters of the overall model is: ； where are the generated task loss and mutual information loss, respectively, and is a hyperparameter balancing the two losses.