Molecular property prediction large language model construction method based on knowledge graph

Abstract

The invention discloses a molecular property prediction large language model construction method based on a knowledge graph. The method comprises a plurality of steps: firstly, acquiring pre-training corpora by using a high-throughput data acquisition technology, including related data such as molecular attributes and metabolic pathways; then, a knowledge graph about fourteen end points is constructed based on the data, and the knowledge graph represents the relation between the molecules and the attributes and the end points of the molecules in a triple form; thirdly, pre-training the large language model by using a LoRA technology, optimizing the model by adopting an incremental learning strategy, and forming an expert large model; then, a triple related to the target endpoint is retrieved from the knowledge graph through the RAG technology, and retrieval results are sorted; and filtering the retrieved triple through cue word design and an expert large model, and extracting key information. And finally, converting the filtered triple list into a natural language, inputting the natural language into an expert large model, generating a prediction result of molecular properties, and providing a reasoning basis. According to the method, the accuracy and efficiency of molecule and endpoint chemical reaction prediction are remarkably improved.

Description

Technical Field

[0001] The present invention belongs to the intersection of pharmacy and artificial intelligence, and specifically relates to a method and technology for constructing a large language model for predicting molecular properties based on a knowledge graph. Background Art

[0002] In the prediction of drug molecular properties, information such as the physicochemical properties, pharmacological characteristics and toxicity of drug molecules is a crucial link in the drug development process. Traditional methods for predicting drug molecular properties mainly rely on physical and chemical methods such as quantum chemical calculations and molecular dynamics simulations. Although these methods are accurate, they are computationally complex and time-consuming, making them difficult to adapt to the needs of large-scale drug screening.

[0003] In recent years, the rapid development of machine learning and deep learning technologies has provided new methods for predicting the molecular properties of drugs. However, existing prediction models based on deep learning often rely on a large amount of labeled data for training. Although certain breakthroughs have been made in predictive performance, the following problems still exist: (1) Most existing models rely on large-scale, structured drug molecule datasets, which have high requirements for data acquisition and annotation. Especially in the new drug development stage, the lack of labeled data limits the generalization ability of the model. (2) Models usually make predictions based only on molecular structure information, and it is difficult to effectively utilize external knowledge such as existing drug knowledge bases and literature, thereby limiting the model's predictive ability in small sample conditions. (3) Different models are often required to predict different physical and chemical properties, pharmacokinetic properties, etc. of drugs, and there is a lack of a unified framework to simultaneously handle multiple drug molecular properties.

[0004] Based on this, a method for constructing a molecular property prediction large language model based on knowledge graph can effectively solve the above problems. This method enhances the model's reasoning and understanding ability by integrating various external knowledge such as drug knowledge base and literature, and improves the accuracy of prediction when data is scarce. At the same time, the knowledge-enhanced large model has multi-task learning capabilities and can handle different types of drug molecular property prediction tasks under a unified framework, thereby improving prediction efficiency and reducing the time cost of model training and debugging.

[0005] In summary, a method for constructing a large language model for molecular property prediction based on knowledge graph is designed to solve the above problems. Summary of the invention

[0006] The present invention aims to provide a method for constructing a large language model for molecular property prediction based on a knowledge graph, so as to realize the prediction of androgen receptor (AR), aryl hydrocarbon receptor (AhR), antioxidant response element (ARE), androgen receptor ligand binding domain (AR-LBD), aromatase, ATPase family, AAA domain containing 5 (ATAD5), carcinogen (Carc), estrogen receptor (ER), estrogen receptor ligand binding domain (ER-LBD), glucocorticoid receptor (GR), heat shock element (HSE), matrix metalloproteinase (MMP), cyclin 53 (Tumor protein P53, P53) and thyroid hormone receptor (Thyroid This method improves the accuracy and efficiency of reaction prediction between molecules and biological endpoints.

[0007] To achieve the above object, the present invention provides the following solutions:

[0008] A method for constructing a large language model for molecular property prediction based on a knowledge graph, the method comprising:

[0009] Data acquisition:

[0010] Obtain the pre-training corpus and the relevant data required to build the knowledge graph. The pre-training corpus includes three parts: SMILES sequence, scientific text, and package sequence. The specific steps are:

[0011] Using crawler technology, SMILES sequences were crawled from the PubChem website, and the titles and abstracts of papers were crawled from the PubMed website as scientific texts. Named entity recognition (NER) tools were used to identify compound entities from scientific texts and replace them with their corresponding SMILES sequences to construct package sequences.

[0012] The specific steps for acquiring the corpus required to build a knowledge graph are:

[0013] Using web crawler technology, crawl the molecular properties and related metabolic pathways on the PubChem website. Specifically, for each molecule, crawl the data in the Computed Properties module and Pathways module on the PubChem website;

[0014] Knowledge graph construction:

[0015] Construct a knowledge graph about endpoints. The knowledge graph includes multiple nodes, such as SMILES nodes, endpoint nodes, property nodes, and metabolic pathway nodes. The relationships between nodes are represented by triples:

[0016] <SMILES, has property, property node>;

[0017] <SMILES, undergoes, metabolic pathway>;

[0018] <SMILES, affects, endpoint node>;

[0019] <Endpoint node, has property, property node>;

[0020] Pre-training of large language models:

[0021] Pre-train the LLaMA3.1-8B model using SMILES, scientific texts, and wrapped sequences to obtain an expert large model. The specific implementation methods include:

[0022] Based on the LoRA (Low-Rank Adaptation) technology for incremental learning to reduce the number of parameters and improve the training efficiency. By performing low-rank decomposition on specific layers or parameters, the number of parameters to be updated is effectively reduced, thus improving the training efficiency and resource utilization rate of the large model; LoRA fine-tuning will cover all layers of the model, which ensures that the entire model can obtain more efficient learning effects through low-rank adjustment at all levels;

[0023] Using the RAG technology for knowledge retrieval:

[0024] When the user inputs a molecular property prediction problem, first define a query to identify the subject, extract the triple subject to be retrieved, and then retrieve relevant information about the subject from the knowledge graph, where vectorization technology is used to calculate the similarity;

[0025] Filtering retrieved data:

[0026] Use the trained expert large model to filter the retrieved triple list, identify and screen out relevant attribute or property data that may cause chemical reactions between molecules and endpoints, and obtain a filtered triple;

[0027] Molecular property prediction and reasoning:

[0028] The external generated big model is called to convert the filtered triples into natural language and input them into the expert big model as a context information, so as to realize the knowledge enhancement of the expert big model, and finally predict whether the molecule can react with the endpoint and provide corresponding judgment basis.

[0029] The present invention can effectively improve the accuracy and efficiency of molecular property prediction by comprehensively utilizing knowledge graph, RAG technology and large language model, and promote the development of fields such as drug development and molecular design, which is specifically reflected in the following aspects:

[0030] Improve prediction accuracy by building a rich knowledge graph that combines the complex relationship between molecular structure and biological endpoints to more accurately capture potential reaction mechanisms. This accuracy is critical in drug development, helping to identify new drug candidate molecules and reduce R&D risks;

[0031] Accelerate the R&D process and use RAG technology for rapid knowledge retrieval, significantly shortening the time for data processing and information acquisition. R&D personnel can obtain the required information in a shorter time, thereby accelerating the drug screening and verification process and improving overall R&D efficiency.

[0032] Enhanced decision support: By analyzing a large amount of historical data and real-time information, the system can provide reliable predictions and suggestions for R&D personnel. This intelligent decision support can help scientists make more forward-looking choices in the molecular design stage and optimize the allocation of experimental resources.

[0033] Reduce costs. Efficient prediction of molecular properties can reduce the number of laboratory tests and reduce the resources and capital investment required for physical experiments, thereby effectively controlling R&D costs and improving economic benefits.

[0034] Promoting innovation, the implementation of this invention will bring new research ideas to the fields of drug development and molecular design, promote the discovery and application of new materials and new drugs, and promote the innovative development of the biopharmaceutical industry. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to make the purpose, technical solution and advantages of the invention more clear, the present invention will be further described in detail below with reference to the accompanying drawings, in which:

[0036] Figure 1 It is a flow chart of a method for constructing a large language model for predicting molecular properties based on a knowledge graph according to the present invention. DETAILED DESCRIPTION

[0037] In order to enable those skilled in the art to better understand the solutions of the present invention and to make the above-mentioned objectives, technical solutions and advantages of the present invention more obvious and understandable, the present invention is further described in detail below in conjunction with embodiments and accompanying drawings of the embodiments.

[0038] like Figure 1 As shown, this embodiment provides a method for constructing a large language model for molecular property prediction based on a knowledge graph, which specifically includes:

[0039] S1: Data acquisition: obtain the relevant data required for pre-training corpus and knowledge graph construction through high-throughput data collection;

[0040] S2: Knowledge graph construction: In the data preparation stage, the collected information such as molecular properties and metabolic pathways is used to construct a knowledge graph about the endpoint. The knowledge graph consists of triples.

[0041] S3: Pre-training a large language model: Use the acquired corpus to pre-train the large language model, and use LoRA technology to perform incremental learning on all layers of the model. The trained model is an expert large model.

[0042] S4: Use RAG technology to perform knowledge retrieval, retrieve relevant triple information from the knowledge graph of the endpoint, and organize the results into a triple list;

[0043] S5: Filter the retrieved data; construct a suitable prompt word, and let the expert model filter the retrieved triple list to obtain a filtered triple list;

[0044] S6: Prediction and reasoning of molecular properties: The filtered triple list is converted into a natural language description and input into the expert large model as a new prompt word to obtain the prediction results of the molecular properties.

[0045] This method can predict a total of fourteen endpoints, including AR, AhR, ARE, AR-LBD, Aromatase, ATAD5, Carc, ER, ER-LBD, GR, HSE, MMP, P53 and TR. This method improves the accuracy and efficiency of the reaction prediction between molecules and biological endpoints.

[0046] Furthermore, step S1 data acquisition specifically includes: using crawler technology to obtain SMILES sequences as molecular structure data; obtaining titles and abstracts of scientific papers through crawler technology to build a scientific text corpus; using named entity recognition tools to identify compound entities from scientific texts, and replacing the compound entities with corresponding SMILES sequences to generate package sequences; obtaining the physicochemical properties and metabolic pathway data of molecules for building a knowledge graph.

[0047] Step 101: Use web crawler technology to crawl 30M SMILES sequences from the PubChem website and crawl the titles and abstracts of 30M papers from the PubMed website as scientific texts; use NER tools to identify compound entities from the scientific texts and replace the entities with their corresponding SMILES sequences, thus constructing 8M wrapped sequences; crawl the property characteristics and involved metabolic pathways of molecules from the PubChem website. Specifically, for each molecule, crawl the data in the Computed Properties module and Pathways module on the PubChem website.

[0048] The construction of the knowledge graph in Step S2 specifically includes: defining nodes and relationships. The nodes include SMILES nodes, property nodes, metabolic pathway nodes, and endpoint nodes AR nodes; the relationships include types such as "has property", "undergoes", and "affects". The SMILES nodes are associated with the property nodes through the "has property" relationship to describe the physicochemical properties of the molecules; the SMILES nodes are associated with the metabolic pathway nodes through the "undergoes" relationship to describe the metabolic processes of the molecules; the SMILES nodes are associated with the endpoint nodes through the "affects" relationship to indicate the impact of the molecules on the endpoints; the endpoint nodes are associated with the property nodes through the "has property" relationship to describe the biological properties of the endpoints.

[0049] Step 102: Based on the relevant data for constructing the knowledge graph crawled in Step 101, construct a knowledge graph about the AR endpoints. Specifically, the knowledge graph contains SMILES nodes, AR nodes, property nodes, and metabolic pathway nodes, and the relationships between the nodes are represented by triples, including: <SMILES, has property, property node>, <SMILES, undergoes, metabolic pathway>, <SMILES, affects, AR node>, <AR node, has property, property node>.

[0050] Furthermore, the pre-training of the large language model in Step S3 specifically includes: using the obtained SMILES sequences, scientific texts, and wrapped sequences as pre-training corpora to pre-train the general large language model; adopting the LoRA technology for incremental learning to reduce the number of parameters and improve the training efficiency through low-rank decomposition; the LoRA fine-tuning covers all layers of the model. During the pre-training process, set the per-device batch size to 12, the gradient accumulation steps to 2, the learning rate to 1.0e-4, the number of training epochs to 1, and use the cosine annealing learning rate scheduler and BF16 precision for training.

[0051] Step 103: 30M SMILES sequences, 30M scientific texts, and 8M package sequences are used as pre-training corpora to pre-train the LLaMA3.1-8B model. LoRA technology is used for incremental learning to reduce the number of parameters and improve training efficiency. By performing low-rank decomposition on specific layers or parameters, the number of parameters that need to be updated is effectively reduced, thereby improving the training efficiency and resource utilization of large models. LoRA fine-tuning will cover all layers of the model. This method ensures that the entire model can achieve more efficient learning effects through low-rank adjustment at all levels. On 8 A100 graphics cards, the training batch size per device is 12, which helps to provide the model with enough data to learn while maintaining memory efficiency. Secondly, by setting the number of gradient accumulation steps to 2, a larger batch size can be simulated when hardware resources are limited, thereby improving the stability and effect of training. The learning rate is set to 0.0001, which is a relatively small value, which helps the model to update weights slowly and steadily in the early stages of training, avoiding instability in the training process caused by excessive update steps. The number of training rounds is set to 1, which means that we will traverse the training dataset completely once; the cosine annealing strategy is selected as the learning rate scheduler, which can periodically adjust the learning rate during the training process, which helps the model to adjust the weights more carefully in the later stage of training, thereby improving the final performance. The warm-up ratio is set to 0.1, which means that in the first 10% of the training time, the learning rate will gradually increase from 0 to the set value, which helps the model to be more stable in the early stages of training; in addition, BF16 training is enabled, which means that BFloat16 precision is used for calculations, which can significantly reduce memory usage and speed up the training process while largely maintaining the accuracy of the model.

[0052] Furthermore, the knowledge retrieval of step S4 specifically includes: based on the molecular property prediction question input by the user, calculating the similarity between the query subject and the nodes in the knowledge graph through vectorization technology; retrieving triples related to the query subject from the knowledge graph, and outputting a list of triples related to the query question.

[0053] Step 104: Use RAG technology to perform knowledge retrieval reasoning

[0054] This step introduces RAG (Retrieval-Augmented Generation) technology to match the molecular property prediction problem input by the user with the relevant information in the knowledge base to support the prediction process. The specific process is as follows:

[0055] (1) Retrieval input: The user inputs a property prediction question for a certain molecule. For example, the user inputs the question “Does {SMILES} react with the AR endpoint?” Here, {SMILES} represents the structural information of the molecule, and the AR endpoint is the potential reaction object.

[0056] (2) Define the query: Based on the molecules and reaction endpoints input by the user, the system automatically identifies key entities and attributes. First, the SMILES sequence is extracted as the main entity, and the AR endpoint is identified as the target node of the reaction. During this process, the system determines the focus of the query, namely "{SMILES}" and "AR", through predefined rules or models.

[0057] (3) Similarity retrieval: Based on the query focus determined in the previous step, the system searches the knowledge graph for other entities and attributes related to "{SMILES}" and "AR", especially those that may be related to chemical reactions. During the retrieval, the system uses vectorization technology to calculate the similarity between the SMILES molecular sequence and other molecules and endpoints in the knowledge base to find the most relevant triple data, including reaction influencing factors, molecular structure characteristics and other information.

[0058] (4) Search output: Based on the similarity search results, the system generates a list of multiple triplets, showing the molecules, endpoints, and their properties that are most relevant to the input question. This output will serve as input for subsequent steps to further analyze whether there is potential chemical reaction between the molecule and the endpoint.

[0059] Furthermore, the data filtering in step S5 specifically includes: using the expert big model to filter the triple list retrieved from the knowledge graph; screening out key attributes or property data that may cause chemical reactions between the molecule and the endpoint by analyzing the molecular structure, physicochemical properties and their relationship with the reaction endpoint; filtering out irrelevant triplets to obtain a filtered triple list as input for subsequent reasoning.

[0060] Step 105: Filter the retrieved triples using the expert model

[0061] In this step, a specially trained expert model is used to filter the triple data retrieved from the knowledge graph to further screen out the key attributes or factors that may trigger chemical reactions between molecules and endpoints. The process specifically includes:

[0062] (1) Input triples: Construct a suitable prompt word Prompt and input the triple list generated in step 104 into the expert model. The model will analyze the nodes and relationships in the triples based on the knowledge learned during the training process.

[0063] (2) Property analysis: The expert model analyzes the molecular structure, physicochemical properties and their relationship with the reaction endpoints (such as AR) involved in the triplets to identify the relevant properties that may lead to chemical reactions. For example, the model evaluates factors such as the chemical groups, polarity, and electron cloud distribution of SMILES molecules, and matches them with the reactivity of the endpoints to determine which triplets have the potential for chemical reactions.

[0064] (3) Triple filtering: Based on the above analysis results, the model will filter out triplets that are not likely to react. For example, if the charge distribution or spatial configuration between the molecule and the endpoint is not suitable for reaction, the triple will be excluded. After filtering, only those triplets that are considered likely to trigger chemical reactions are retained, resulting in a streamlined list of triplets that are highly relevant to chemical reactions.

[0065] Furthermore, the molecular property prediction of step S6 specifically includes: converting the filtered triples into natural language prompts and inputting them into the expert big model; the expert big model performs knowledge-enhanced reasoning based on the input prompts to predict whether the molecule reacts with the endpoint; at the same time, the model provides reasoning process and judgment basis, explains the potential mechanism of the reaction between the molecule and the endpoint, and outputs the final prediction result.

[0066] Step 106: Use the filtered triple combination prompt to perform knowledge-enhanced prediction using the expert model

[0067] This step is based on the filtered triples generated in step 105, and through the prompt mechanism, further guides the expert model to perform knowledge-enhanced reasoning, predict whether the molecule and the endpoint will react chemically, and provide an explanation. The specific process is as follows:

[0068] (1) Generate prompt: The filtered triplets are constructed into natural language prompts in the form of “Will the {SMILES} molecule react with the AR endpoint? The relevant chemical properties are X, Y, and Z. Please explain its reaction mechanism.” Here, X, Y, and Z represent the key chemical properties or reaction factors extracted from the filtered triplets.

[0069] (2) Knowledge-enhanced prediction: The generated natural language prompt is input into the expert model, and the model performs knowledge-enhanced reasoning based on the prompt information. The expert model has been pre-trained with a large amount of medicinal chemistry data and chemical reaction mechanisms, so it can combine the attributes in the prompt to deeply reason about the possibility of the molecule reacting with the endpoint.

[0070] (3) Prediction and explanation: Based on the input prompts, the model infers and predicts whether the {SMILES} molecule will react with the AR endpoint. At the same time, the model will also output corresponding explanations to explain the mechanism of the reaction. For example, based on the electronic configuration or structural characteristics of the SMILES molecule, the model may point out that it has a high affinity with the AR endpoint and thus has the possibility of reaction. The explanation includes the type of reaction, the changes in chemical bonds involved, and a detailed analysis of the relevant chemical properties.

[0071] In the above, the specific embodiments of the present invention are described with reference to the accompanying drawings. However, those skilled in the art will appreciate that various changes and substitutions may be made to the specific embodiments of the present invention without departing from the spirit and scope of the present invention. These changes and substitutions are all within the scope defined by the claims of the present invention.

Claims

1. A method for constructing a large language model for molecular property prediction based on knowledge graph, characterized in that: include: S1: Data acquisition: obtain the relevant data required for pre-training corpus and knowledge graph construction through high-throughput data collection; S2: Knowledge graph construction: In the data preparation stage, the collected information such as molecular properties and metabolic pathways is used to construct a knowledge graph about the endpoint. The knowledge graph consists of triples. S3: Pre-training a large language model: Use the acquired corpus to pre-train the large language model, and use LoRA technology to perform incremental learning on all layers of the model. The trained model is an expert large model. S4: Use RAG technology to perform knowledge retrieval, retrieve relevant triple information from the knowledge graph of the endpoint, and organize the results into a triple list; S5: Filter the retrieved data; construct a suitable prompt word, and let the expert model filter the retrieved triple list to obtain a filtered triple list; S6: Prediction and reasoning of molecular properties: The filtered triple list is converted into a natural language description and input into the expert large model as a new prompt word to obtain the prediction results of the molecular properties.

2. The method for constructing a large language model for molecular property prediction based on a knowledge graph according to claim 1, characterized in that: The data acquisition in step S1 specifically includes: using crawler technology to obtain SMILES sequences as molecular structure data; obtaining titles and abstracts of scientific papers through crawler technology to build a scientific text corpus; using named entity recognition tools to identify compound entities from scientific texts, and replacing the compound entities with corresponding SMILES sequences to generate package sequences; obtaining the physicochemical properties and metabolic pathway data of molecules for building a knowledge graph.

3. The method for constructing a large language model for molecular property prediction based on knowledge graph according to claim 1, characterized in that: The knowledge graph construction of step S2 specifically includes: defining nodes and relationships, where nodes include SMILES nodes, attribute nodes, metabolic pathway nodes, and endpoint nodes; relationships include types such as "has attributes", "experiences", and "influences"; associating SMILES nodes with attribute nodes through the "has attributes" relationship to describe the physicochemical properties of molecules; associating SMILES nodes with metabolic pathway nodes through the "experiences" relationship to describe the metabolic process of molecules; associating SMILES nodes with endpoint nodes through the "influence" relationship to represent the influence of molecules on endpoints; associating endpoint nodes with attribute nodes through the "has attributes" relationship to describe the biological properties of endpoints.

4. The method for constructing a large language model for molecular property prediction based on knowledge graph according to claim 1, characterized in that: The pre-training large language model in step S3 specifically includes: using the obtained SMILES sequence, scientific text and package sequence as pre-training corpus to pre-train the general large language model; using LoRA technology for incremental learning, reducing the number of parameters and improving training efficiency through low-rank decomposition; LoRA fine-tuning covers all layers of the model, setting the batch size per device to 12, the number of gradient accumulation steps to 2, the learning rate to 1.0e-4, the number of training rounds to 1, and using the cosine annealing learning rate scheduler and BF16 precision for training during the pre-training process.

5. The method for constructing a large language model for molecular property prediction based on knowledge graph according to claim 1, characterized in that: The knowledge retrieval in step S4 specifically includes: based on the molecular property prediction question input by the user, calculating the similarity between the query subject and the nodes in the knowledge graph through vectorization technology; retrieving triples related to the query subject from the knowledge graph, and outputting a list of triples related to the query question.

6. The method for constructing a large language model for molecular property prediction based on knowledge graph according to claim 1, characterized in that: The data filtering in step S5 specifically includes: using the expert big model to filter the triple list retrieved from the knowledge graph; screening out key attributes or property data that may cause chemical reactions between molecules and endpoints by analyzing the molecular structure, physicochemical properties and their relationship with the reaction endpoints; filtering out irrelevant triples to obtain a filtered triple list as input for subsequent reasoning.

7. The method for constructing a large language model for molecular property prediction based on knowledge graph according to claim 1, characterized in that: The molecular property prediction in step S6 specifically includes: converting the filtered triples into natural language prompts and inputting them into the expert model; the expert model performs knowledge-enhanced reasoning based on the input prompts to predict whether the molecule reacts with the endpoint; at the same time, the model provides reasoning process and judgment basis, explains the potential mechanism of the reaction between the molecule and the endpoint, and outputs the final prediction result.

Images