A method for constructing high-quality fine-tuned chemical data based on knowledge graphs

Abstract

This invention discloses a method for constructing high-quality fine-tuned chemical data based on knowledge graphs. First, multi-source chemical reaction data is collected to construct a chemical reaction knowledge graph. Then, five task subgraph patterns are defined within the chemical reaction knowledge graph. Instances of chemical reaction task subgraphs are retrieved and extracted from the knowledge graph, and these instances are transformed into instruction-based question-and-answer samples suitable for training a large language model, thus forming an instruction dataset. Subsequently, the instruction dataset is used to train the large language model. Finally, a test set is constructed to test the trained large language model for inference, and an evaluation system is used to systematically and quantitatively evaluate the model. The evaluation results are fed back into the construction of the instruction dataset and the extraction strategy for chemical reaction task subgraph instances. This invention provides a scalable, verifiable, and sustainably evolving training data construction mechanism, continuously improving the cognitive ability of large language models in chemical knowledge.

Description

Technical Field

[0001] This invention relates to the field of data construction technology for large language models, specifically a method for constructing high-quality fine-tuned chemical data based on knowledge graphs. Background Technology

[0002] With the continuous advancement of digital transformation and the open data ecosystem, multi-source data, including scientific research literature, patent texts, industry standards, product manuals, database entries, and internet information, has experienced explosive growth in scale, type, and update frequency. Knowledge points often exist in different forms of expression scattered across multi-source data, and with version iterations, terminology evolution, and viewpoint updates, the knowledge system exhibits typical characteristics of "multi-source heterogeneity, continuous evolution, and cross-domain connections."

[0003] Against this backdrop, for applications such as literature analysis, intelligence mining, technology route discovery, scientific discovery, and knowledge question answering, business users generally expect large language models to possess the following capabilities: First, the ability to quickly absorb cutting-edge knowledge contained in recent data, maintaining the timeliness of knowledge; second, the ability to establish relationships between entities, concepts, methods, and conclusions across data sources, enabling multi-hop reasoning and evidence chain tracing; and third, the ability to maintain factual consistency, logical self-consistency, and verifiability in complex reasoning and generation tasks, reducing seemingly plausible but ultimately flawed outputs. How to enable large language models to quickly absorb the latest knowledge, establish relationships across data sources, and maintain factual consistency and reliability in complex reasoning tasks has become a crucial issue in the field of intelligent information processing.

[0004] Model fine-tuning is an important means of improving the domain capabilities and knowledge coverage of large models. However, the construction of high-quality training data is extremely costly, which has become a key bottleneck restricting the improvement of model capabilities.

[0005] Currently, the mainstream data construction methods mainly fall into two categories:

[0006] (1) Manual expert annotation method:

[0007] This approach, where domain experts manually design instructions, question-and-answer samples, or knowledge relationship annotations based on the target task, offers high assurance in terms of factual accuracy, terminology standardization, and task alignment, making it particularly suitable for professional fields with high requirements for security and rigor. However, this approach suffers from high costs, long development cycles, limited scalability, and difficulty in continuous updates, making it challenging to support continuous iteration of large-scale models.

[0008] (2) Automatic data synthesis method for large models:

[0009] This approach typically uses existing textual data as input, with a large language model automatically generating question-and-answer pairs, instruction data, or multi-turn dialogues for retraining or fine-tuning. Its advantages lie in high generation efficiency, relatively controllable costs, and the ability to quickly expand the sample size, forming a usable dataset in a short time. However, in complex and specialized knowledge scenarios, this approach faces significant drawbacks: First, the generation process lacks strong factual constraints, potentially leading to model illusions or confusion between similar concepts, thus introducing factual errors. Second, samples often exist in the form of "isolated question-and-answer pairs," lacking systematic connections across samples and entities, resulting in fragmented data distribution that struggles to support the model's learning of structured knowledge systems. Third, the model's ability to model complex domain knowledge, logical constraints, multi-hop relationships, and cross-entity consistency is insufficient, easily generating seemingly reasonable but unverifiable reasoning chains. Fourth, the generated data is difficult to systematically verify for consistency, especially when the data involves multi-source information fusion, version differences, or the coexistence of multiple schemes; manual sampling alone is insufficient to detect hidden conflicts. This results in automatically synthesized data that, while "large in volume," often suffers from problems such as "insufficient credibility, unstable structure, and difficulty in auditing," thereby limiting the effectiveness of fine-tuning and the usability of the project.

[0010] Therefore, how to achieve a high-quality fine-tuning data construction mechanism that ensures training data is "factually credible, structurally consistent, logically controllable, verifiable, and sustainably scalable" while maintaining the advantages of automation at scale has become a critical technical problem that urgently needs to be solved. The core of this problem lies in the need for a general method that can structure and organize knowledge from multi-source heterogeneous data, explicitly establish entities and their relationship networks, and use structural facts as the basis and constraint when generating training samples, thereby reducing illusions, improving consistency, and enhancing correlation.

[0011] Based on existing data construction and fine-tuning practices, the existing technology has the following main shortcomings.

[0012] (1) Insufficient data credibility:

[0013] Many automated synthesis schemes rely primarily on plain text generation or weakly constrained generation, lacking traceable factual sources and structured constraints. This easily introduces false relationships, incorrect attributions, or invalid inference chains. In specialized fields and high-risk decision-making scenarios, this problem directly impacts the reliability and usability of the model output, and the errors are often subtle and difficult to detect through limited sampling.

[0014] (2) Weak sample correlation:

[0015] Existing methods often construct data using "single samples" as the basic unit, lacking systematic connections between samples and failing to reflect the multi-hop relationships, path dependencies, and multi-source evidence fusion patterns in real knowledge networks. The knowledge learned by models on such data is often distributed in a point-like manner, making it difficult to form structured understanding and generalizable reasoning capabilities, resulting in unstable performance in tasks such as multi-step reasoning, cross-entity querying, and evidence chain construction.

[0016] (3) Poor scalability:

[0017] When data sources are updated, domain terminology evolves, or task types expand, existing data construction processes often require redesigning templates, re-extracting samples, or re-labeling, making low-cost incremental maintenance difficult. Especially in multi-source data fusion scenarios, the integration of new data sources is usually accompanied by issues such as entity disambiguation, relationship alignment, and consistency verification. Without a unified structured framework, the cost of expansion increases significantly.

[0018] (4) Difficulty in forming a closed-loop optimization mechanism:

[0019] In existing processes, data construction, model training, and model evaluation are often disconnected: after data generation, there is a lack of effective mechanisms to correct data distribution, address weaknesses, or rectify conflicting samples based on evaluation results; and there is a lack of interpretable correspondence between evaluation metrics and data construction strategies, making it difficult to form a continuous iterative closed-loop optimization process of "generation-training-evaluation." This ultimately manifests as unstable model capability improvement, delayed absorption of new knowledge, and high long-term maintenance costs. Summary of the Invention

[0020] The technical problem to be solved by this invention is to provide a high-quality fine-tuning chemical data construction method based on knowledge graphs. The method focuses on the construction of knowledge graphs of multi-source chemical reaction data, extraction of subgraph instances, generation of instruction datasets, and a closed loop of fine-tuning and evaluation of large language model parameters. It provides a scalable, verifiable, and sustainably evolving training data construction mechanism to continuously improve the cognitive ability of large language models in chemical knowledge.

[0021] The technical solution of this invention is as follows:

[0022] A method for constructing high-quality fine-tuned chemical data based on knowledge graphs, specifically including the following steps:

[0023] (1) Collect multi-source chemical reaction data and construct a chemical reaction knowledge graph. The entity nodes in the chemical reaction knowledge graph represent chemical molecules, and the relationship edges represent the transformation relationships between chemical molecules through chemical reactions. The relationship edges have a clear directionality.

[0024] (2) Five task subgraph patterns are set in the chemical reaction knowledge graph. Through the automatic matching mechanism, chemical reaction task subgraph instances that meet the rules of various task subgraph patterns and satisfy logical consistency and factual verifiability are retrieved and extracted from the chemical reaction knowledge graph to form a set of chemical reaction task subgraph instances.

[0025] (3) Transform each chemical reaction task subgraph instance in the chemical reaction task subgraph instance set into instruction question and answer samples suitable for training large language models, thereby forming an instruction dataset;

[0026] (4) The instruction dataset is used to train the large language model, so that the large language model learns the rules of the five task subgraph patterns;

[0027] (5) Construct a test set, conduct inference tests on the trained large language model, and use an evaluation system to systematically and quantitatively evaluate the prediction ability of the large language model. Feed the evaluation results back to the construction of the instruction dataset and the extraction strategy of chemical reaction task subgraph instances, adjust the task subgraph pattern extraction distribution in reverse, and make up for the weak chemical reaction task subgraph instances.

[0028] The multi-source chemical reaction data includes document-based chemical reaction data, structured chemical reaction data from patent databases, and reaction examples and substrate-product correspondence data from existing chemical databases.

[0029] The entity nodes are uniquely identified using the standardized molecular SMILES structure representation, and the content on the relation edges includes auxiliary conditions used to describe the chemical reaction.

[0030] After the acquisition of multi-source chemical reaction data and before the construction of the chemical reaction knowledge graph, the multi-source chemical reaction data is cleaned, specifically as follows:

[0031] S11. Use cheminformatics tools to perform SMILES syntax parsing, normalization, and structural consistency verification on all chemical molecule entries;

[0032] S12. The RxnMapper unsupervised atomic mapping tool is used to perform atomic-level mapping of chemical reactants and products, correcting the problem of confusion between reactants and reagents caused by manual annotation or data capture.

[0033] S13. Duplicate records where the reactant and product sets of a chemical reaction are exactly the same are deduplicated, and only one valid record is retained; abnormal entries where the same chemical molecule is both a reactant and a product are judged as data redundancy and deleted.

[0034] S14. Use existing large language models to perform text semantic review, verify the contextual consistency of the descriptive text of chemical reaction data, and filter out two types of problematic data: irrelevant reagent errors and multi-step reaction polymerization errors.

[0035] The five task subgraph patterns include a two-step path reasoning task subgraph pattern, a three-step path reasoning task subgraph pattern, a multi-solution reactant generation task subgraph pattern, a multi-solution product prediction task subgraph pattern, and an alternative co-reactant prediction task subgraph pattern.

[0036] The two-step path reasoning task subgraph pattern described above is retrieved and extracted from the chemical reaction knowledge graph. A two-step reaction chain in the form of a chemical reaction task subgraph example;

[0037] The three-step path reasoning task subgraph pattern described above is retrieved and extracted from the chemical reaction knowledge graph. A three-step reaction chain in the form of a chemical reaction task subgraph example;

[0038] The multi-solution reactant generation task subgraph pattern is to retrieve and extract reaction instances from the chemical reaction knowledge graph that generate the same product from multiple different reactants, and use them as chemical reaction task subgraph instances.

[0039] The multi-solution product prediction task subgraph mode retrieves and extracts reaction instances from the chemical reaction knowledge graph that produce multiple different products from the same reactant, and uses them as chemical reaction task subgraph instances.

[0040] The aforementioned subgraph pattern for predicting alternative co-reactants involves retrieving and extracting reaction instances from the chemical reaction knowledge graph that involve selecting different co-reactants to react with the same reactant and producing the same product, and using these instances as chemical reaction task subgraph instances.

[0041] The process of transforming each chemical reaction task subgraph instance in the chemical reaction task subgraph instance set into instruction question-and-answer samples suitable for training a large language model, thereby forming an instruction dataset, specifically involves: defining text instruction templates corresponding to five types of chemical reaction task subgraph instances; transforming each type of chemical reaction task subgraph instance into instruction question-and-answer samples suitable for training a large language model; and each text instruction template including a task description, output format specification, and output constraints. The task description clarifies the reasoning or generation goals that the large language model needs to achieve; the output format specification standardizes the representation of molecular SMILES structures and the required number of results output by the large language model; and the output constraints limit the output format of the large language model.

[0042] After the command question-and-answer samples are transformed and generated, semantic enhancement processing is performed. Through automatic rewriting, synonym replacement and sentence transformation, semantically equivalent but diverse command question-and-answer samples are generated. Combined with the transformed command question-and-answer samples, a command fine-tuning dataset is formed.

[0043] The training of the large language model using the instruction dataset enables the large language model to learn the rules of five task subgraph patterns. Specifically, LoRA technology is used to train the large language model and to fine-tune the parameters of the large language model.

[0044] The loss function used to train the large language model is the cross-entropy loss function. , see the following formula (1) for details:

[0045] (1);

[0046] In equation (1), Parameters representing a large language model; This represents the total number of tokens in the vocabulary determined during the training phase of the large language model. This vocabulary is a fixed category space for the output layer of the large language model. The actual labels representing sample instructions and questions; Represents the parameters The large language model under the given conditions predicts the instruction question-answer sample as the first... The probability of each category.

[0047] The aforementioned construction of a test set is used to perform inference tests on the trained large language model, and an evaluation system is employed to systematically and quantitatively assess the predictive ability of the large language model. Specifically:

[0048] S51. When constructing the test set, collect multiple real and feasible reference answers for each question to form a reference answer set for that question;

[0049] S52. Construct an evaluation system. The evaluation system systematically and quantitatively evaluates the predictive ability of the large language model from three dimensions: accurate matching rate, structural similarity, and chemical effectiveness.

[0050] The exact match rate measures the proportion of a large language model's output that is completely identical to the reference answers in the reference answer set.

[0051] Structural similarity To characterize the structural similarity between the predicted molecule and the reference molecule, the following steps are taken: First, a Morgan fingerprint is generated for each chemical molecule based on cheminformatics tools, and the structure of the chemical molecule is encoded into a fixed-length binary feature vector. Then, the Tanimoto similarity is used to compare the Morgan fingerprint of the predicted molecule with the Morgan fingerprint of the reference molecule. The calculation formula is shown in the following formula (2):

[0052] (2);

[0053] In equation (2), This represents the Morgan fingerprint vector generated by the predicted structural representation of the SMILES molecule. This represents the Morgan fingerprint vector generated by the structural representation of the reference molecule SMILES; This represents the number of bits that are simultaneously 1 in both the predicted and reference molecule's Morgan fingerprint vectors, corresponding to the number of local structural features shared by the predicted and reference molecules. and These represent the total number of 1 bits in the Morgan fingerprint vectors of the predicted molecule and the reference molecule, respectively, reflecting the number of structural features contained in each molecule. When the predicted molecule and the reference molecule have completely identical structures, the FTS value is 1; when the predicted molecule and the reference molecule do not share any structural features, the FTS value is 0.

[0054] Chemovalidation is used to evaluate the legality and usability of chemical molecules generated by the large language model at the level of chemical rules. Specifically, it involves: first, inputting the SMILES string output by the large language model into a cheminformatics tool for parsing and checking whether it conforms to the SMILES syntax specification; then determining whether the SMILES string can be successfully converted into a valid molecular object. If the SMILES string can be correctly parsed by the cheminformatics tool and generate a valid molecular structure, the prediction result is considered valid; otherwise, it is considered invalid.

[0055] S53. Construct a scoring mechanism based on multiple answer sets: when any two paths output by the large language model appear in the reference answer set, score 1 point; when only one path output by the large language model appears in the reference answer set, score 0.5 points; when the output of the large language model has no matching path with the reference answer set, score 0 points.

[0056] Advantages of this invention:

[0057] (1) In view of the problem that existing automatically synthesized data lacks strong factual constraints and is prone to illusion or conceptual confusion, this invention introduces a chemical reaction knowledge graph as the factual foundation and structural constraint for training data generation, so that multi-source chemical reaction data is mapped into a structured knowledge network of molecular nodes-reaction directed edges. This makes the subsequent training sample generation no longer rely on the free play of pure text, but uses the structural facts in the chemical reaction knowledge graph as the basis for generation and the source of hard constraints, thereby improving the credibility and auditability of training samples from the source and meeting the requirements of chemical reaction scenarios for fact traceability and conclusion verification.

[0058] (2) This invention defines and matches five chemical reaction task subgraph instances. Each chemical reaction task subgraph instance is matched based on path length, number of branches, convergence structure, and node / edge constraints. Available instances are automatically retrieved in the chemical reaction knowledge graph, thereby ensuring that the generated training samples naturally contain multi-hop relationships, path dependencies, and traceable evidence chains. On this basis, the training samples are generated by subgraph-task-driven processes: the same chemical molecule can appear repeatedly in different task structures and form cross-sample associations; the same path can derive multiple questioning methods and task objectives with different focuses; the same target reactant can correspond to multiple incoming edges to form a multi-solution space. This mechanism makes the subsequently constructed instruction dataset change from a point distribution to a network distribution, which is conducive to the large language model learning of structured knowledge systems and cross-entity consistency, thereby obtaining more stable generalization ability in tasks such as complex question answering, chain reasoning, and evidence chain tracing.

[0059] (3) In view of the problems of unstable automatic data structure, difficulty in system verification and difficulty in auditing, the present invention adopts a strategy of generating task templates and structural constraints: each training sample is mapped from a subgraph instance to a standardized task description, and the output format, number of results and prohibition of interpretation are clearly defined. Then, natural language rewriting and expression diversification are carried out within the fact boundary of the chemical reaction knowledge graph.

[0060] (4) This invention addresses the scalability issues of needing to redo templates, resample, and relabel when updating existing process data sources or expanding tasks. It integrates multi-source chemical reaction data access, chemical molecule alignment, unified expression of relationships, and consistency verification into the framework of chemical reaction knowledge graph. After incremental data enters, it is only necessary to execute the same standardization and consistency verification process and update the nodes and edges in the chemical reaction knowledge graph. This will automatically extract and add subgraph instances in five task subgraph modes, generating incremental training samples.

[0061] (5) In view of the shortcomings of the existing technology, which is that the data construction, training and evaluation are separated and it is difficult to form a continuous optimization closed loop, the present invention designs the evaluation system as a closed loop mechanism that can guide the data strategy in reverse: the prediction ability of the large language model is systematically and quantitatively evaluated through the evaluation system, and the evaluation results are fed back to the construction of the instruction dataset and the extraction strategy of chemical reaction task subgraph instances, and the task subgraph pattern extraction distribution is adjusted in reverse to make up for the weak chemical reaction task subgraph instances; specifically, three types of indicators, namely exact matching rate (EM), structural similarity (FTS) and chemical validity, are used for joint evaluation, and a scoring mechanism based on multiple answer sets is constructed for multi-solution tasks, so as to simultaneously measure strict correctness, structural rationality and fault tolerance, and result usability / analyzability, avoiding the bias caused by a single indicator, and can directly guide the subsequent data to be supplemented by incremental extraction and task rebalancing, thereby forming an interpretable and operable closed loop optimization path. Attached Figure Description

[0062] Figure 1 This is a flowchart of the present invention.

[0063] Figure 2 This is a schematic diagram of the five task subgraph modes of the present invention. Detailed Implementation

[0064] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0065] See Figure 1 A method for constructing high-quality fine-tuned chemical data based on knowledge graphs, specifically including the following steps:

[0066] (1) Collect multi-source chemical reaction data and construct a chemical reaction knowledge graph. The entity nodes in the chemical reaction knowledge graph represent chemical molecules and are uniquely identified using the standardized molecular SMILES structure expression method. The relationship edges in the chemical reaction knowledge graph represent the transformation relationship between chemical molecules through chemical reactions. The relationship edges have a clear directionality. The content on the relationship edges includes auxiliary conditions used to describe the chemical reaction, such as co-reactants, catalysts, solvent types, reaction type labels, etc., to support fact traceability and subsequent constraint verification.

[0067] Among them, multi-source chemical reaction data includes literature text-based chemical reaction data (containing unstructured content such as reaction step descriptions, reagent conditions, and product information), structured chemical reaction data in patent databases, and reaction examples and substrate-product correspondence data in existing chemical databases (Reaxys, SciFinder).

[0068] After collecting multi-source chemical reaction data but before constructing a chemical reaction knowledge graph, the multi-source chemical reaction data is cleaned, specifically as follows:

[0069] S11. Use the cheminformatics tool RDKit to perform SMILES syntax parsing, normalization, and structural consistency verification on all chemical molecule entries;

[0070] S12. The RxnMapper unsupervised atomic mapping tool is used to perform atomic-level mapping of chemical reactants and products, correcting the problem of confusion between reactants and reagents caused by manual annotation or data capture.

[0071] S13. Duplicate records where the reactant and product sets of a chemical reaction are exactly the same are deduplicated, and only one valid record is retained; abnormal entries where the same chemical molecule is both a reactant and a product are judged as data redundancy and deleted.

[0072] S14. Use the existing large language model DeepSeek-V3 to perform text semantic review, verify the context consistency of the descriptive text of chemical reaction data, and filter out two types of problematic data: irrelevant reagents incorrectly included (e.g., solvents, extractants or catalysts mentioned but not actually involved in the reaction) and multi-step reaction polymerization errors (e.g., records that mislabel multiple independent reaction steps as a single-step reaction).

[0073] (2) Five task subgraph patterns are set in the chemical reaction knowledge graph. Through the automatic matching mechanism, chemical reaction task subgraph instances that meet the rules of various task subgraph patterns and satisfy logical consistency and factual verifiability are retrieved and extracted from the chemical reaction knowledge graph to form a set of chemical reaction task subgraph instances.

[0074] See Figure 2 The five task subgraph patterns include two-step path reasoning task subgraph pattern, three-step path reasoning task subgraph pattern, multi-solution reactant generation task subgraph pattern, multi-solution product prediction task subgraph pattern, and alternative co-reactant prediction task subgraph pattern.

[0075] The two-step path reasoning task subgraph pattern is retrieved and extracted from the chemical reaction knowledge graph. A two-step reaction chain in the form of a chemical reaction task subgraph example;

[0076] The three-step path reasoning task subgraph pattern is retrieved and extracted from the chemical reaction knowledge graph. A three-step reaction chain in the form of a chemical reaction task subgraph example;

[0077] The multi-solution reactant generation task subgraph pattern retrieves and extracts reaction instances from the chemical reaction knowledge graph that generate the same product from multiple different reactants, and uses them as chemical reaction task subgraph instances.

[0078] The multi-product prediction task subgraph pattern retrieves and extracts reaction instances from the chemical reaction knowledge graph that produce multiple different products from the same reactant, and uses them as chemical reaction task subgraph instances.

[0079] The subgraph pattern for predicting alternative co-reactants is to retrieve and extract reaction instances from the chemical reaction knowledge graph that produce the same product by selecting different co-reactants to react with the same reactant, and use these as examples of chemical reaction task subgraphs.

[0080] (3) Define text instruction templates corresponding to five chemical reaction task subgraph instances, transform each chemical reaction task subgraph instance in the chemical reaction task subgraph instance set into instruction question-and-answer samples suitable for training large language models, and then perform semantic enhancement processing. Through automatic rewriting, synonym replacement and sentence transformation, generate semantically equivalent but diverse instruction question-and-answer samples, and combine the transformed instruction question-and-answer samples to form an instruction fine-tuning dataset.

[0081] The text instruction templates all include a task description, an output format specification, and output constraints. The task description is used to clarify the reasoning or generation goals that the large language model needs to achieve. The output format specification is used to standardize the representation of the molecular SMILES structure and the number of results required by the large language model. The output constraints are used to limit the output format of the large language model.

[0082] (4) The large language model LLaMA3-8B-Instruct is trained using the instruction fine-tuning dataset, so that the large language model LLaMA3-8B-Instruct learns the rules of five task subgraph patterns. Specifically, LoRA technology is used to train the large language model and achieve fine-tuning of the parameters of the large language model.

[0083] The loss function used for training the large language model LLaMA3-8B-Instruct is the cross-entropy loss function. , see the following formula (1) for details:

[0084] (1);

[0085] In equation (1), Parameters representing a large language model; This represents the total number of tokens in the vocabulary determined during the training phase of the large language model. This vocabulary is a fixed category space for the output layer of the large language model. The actual labels representing sample instructions and questions; Represents the parameters The large language model under the given conditions predicts the instruction question-answer sample as the first... The probability of each category;

[0086] The entire training of the large language model was implemented using Python and PyTorch frameworks. The training environment was configured with an NVIDIA A100 (80GB) GPU, a batch size of 32, and a learning rate of [missing information]. The training rounds are 5, and the maximum sequence length is 4096 tokens;

[0087] (5) Construct a test set, conduct inference tests on the trained large language model, and use an evaluation system to systematically and quantitatively evaluate the prediction ability of the large language model. Feed the evaluation results back to the construction of the instruction dataset and the extraction strategy of chemical reaction task subgraph instances, adjust the task subgraph pattern extraction distribution in reverse, and make up for the weak chemical reaction task subgraph instances.

[0088] The systematic and quantitative evaluation of large language models specifically includes:

[0089] S51. When constructing the test set (500 test samples), collect multiple real and feasible reference answers for each question to form a reference answer set for that question; for example, in the generation of multiple-solution reactants, if the target chemical molecule is synthesized... If there are more than two feasible routes in the real database, collect all feasible routes into the reference answer set;

[0090] S52. Construct an evaluation system. The evaluation system systematically and quantitatively evaluates the predictive ability of the large language model from three dimensions: accurate matching rate, structural similarity, and chemical effectiveness.

[0091] Exact Match (EM) measures the proportion of outputs of a large language model that are completely identical to the reference answers in the reference answer set, reflecting the strict correctness of the large language model in the multi-step path structure.

[0092] Fingerprint Tanimoto Similarity (FTS) is used to characterize the degree of similarity between the predicted molecule and the reference molecule at the structural level. Specifically, it involves generating a Morgan fingerprint for each chemical molecule using cheminformatics tools, encoding the structure of the chemical molecule into a fixed-length binary feature vector, and then comparing the Morgan fingerprint of the predicted molecule with that of the reference molecule using Tanimoto similarity. The calculation formula is shown in the following formula (2):

[0093] (2);

[0094] In equation (2), This represents the Morgan fingerprint vector generated by the predicted structural representation of the SMILES molecule. This represents the Morgan fingerprint vector generated by the structural representation of the reference molecule SMILES; This represents the number of bits that are simultaneously 1 in both the predicted and reference molecule's Morgan fingerprint vectors, corresponding to the number of local structural features shared by the predicted and reference molecules. and These represent the total number of 1 bits in the Morgan fingerprint vectors of the predicted and reference molecules, respectively, reflecting the number of structural features contained in each molecule. When the predicted and reference molecules have completely identical structures, the FTS value is 1; when the predicted and reference molecules do not share any structural features, the FTS value is 0. Compared to the exact matching rate metric, the FTS metric can tolerate chemically reasonable local structural differences and is more suitable for evaluating the correctness of structural approximations in multi-solution and multi-path generation tasks.

[0095] Chemovalidity is used to evaluate the legality and usability of chemical molecules generated by the large language model at the chemical rule level. Specifically, it involves first inputting the SMILES string output by the large language model into a cheminformatics tool for parsing and checking whether it conforms to the SMILES syntax specification; then determining whether the SMILES string can be successfully converted into a valid molecular object. If the SMILES string can be correctly parsed by the cheminformatics tool and generate a valid molecular structure, the prediction result is considered valid; otherwise, it is considered invalid. Validity measures the reliability of the large language model's generated results in terms of chemical syntax consistency and molecular structure rationality, thereby avoiding generated results that are valid only at the string level but unusable in a chemical sense.

[0096] S53. Construct a scoring mechanism based on multiple answer sets: 1 point is awarded when any two paths output by the large language model appear in the reference answer set; 0.5 points are awarded when only one path output by the large language model appears in the reference answer set; and 0 points are awarded when the output of the large language model does not match any path in the reference answer set. This scoring mechanism can effectively measure the coverage and diversity of the large language model in terms of multi-path generation capabilities, and can avoid the bias caused by the traditional single-answer evaluation method.

[0097] Example

[0098] Examples of chemical reaction path planning tasks using five task subgraph patterns.

[0099] (1) Task 1, Two-step path reasoning:

[0100] Problem (input to a large language model):

[0101] Task Description: Please design a two-step chemical reaction scheme to achieve the following: <smiles> COC1=CC(CC=C)=C(N)C=C1< / smiles> Synthesis of starting materials <smiles> COC1=CC2=C(C=C1)N(C(=O)C1=CC=C(Cl)C=C1)C(C)=C2< / smiles> The reaction consists of two consecutive steps: in the first step, <smiles> COC1=CC(CC=C)=C(N)C=C1< / smiles> A complete chemical reaction occurs, producing intermediate products. <smiles> x< / smiles> In the second step, with <smiles> x< / smiles> The reactants yield the final product. <smiles> COC1=CC2=C(C=C1)N(C(=O)C1=CC=C(Cl)C=C1)C(C)=C2< / smiles> .

[0102] Output format instructions: Please provide intermediate products. <smiles> x< / smiles> Possible SMILES expressions.

[0103] Output constraints: No explanation or other information is required; only the SMILES of x are returned.

[0104] Answer (output of the large language model):

[0105] <smiles> COC1=CC2=C(C=C1)N(C(=O)C1=CC=C(Cl)C=C1)C(C)=C2< / smiles> .

[0106] (2) Path reasoning for steps two and three of the task:

[0107] Problem (input to a large language model):

[0108] Task Description: Please design a three-step chemical reaction scheme, starting from... <smiles> CN1CC2=C(C=C(Br)C=C2)C1=O< / smiles> Start synthesizing <smiles> CN1CC2=C(C=C(C=C2)C2=CC=C(S2)C2=CN=CC(NC(=O)OC(C)(C)C)=C2)C1=O< / smiles> The reaction consists of three consecutive steps: the first step, <smiles> CN1CC2=C(C=C(Br)C=C2)C1=O< / smiles> A complete chemical reaction occurs, producing intermediate products. <smiles> [x1_SMILES]< / smiles> The second step is to... <smiles> [x1_SMILES]< / smiles> As reactants, intermediate products are generated. <smiles> [x2_SMILES]< / smiles> The third step is to... <smiles> [x2_SMILES]< / smiles> The reactants yield the final product. <smiles> CN1CC2=C(C=C(C=C2)C2=CC=C(S2)C2=CN=CC(NC(=O)OC(C)(C)C)=C2)C1=O< / smiles> .

[0109] Output format instructions: Please provide a set of possible SMILES expressions for intermediate products x1 and x2. The final answer should be presented in the following format: Intermediate product x1: <smiles> [x1_SMILES]< / smiles> Intermediate product x2: <smiles> [x2_SMILES]< / smiles> .

[0110] Output constraints: No explanation or other information is required; simply submit the answer in the specified format.

[0111] Answer (output of the large language model):

[0112] Intermediate product x1: <smiles> CN1CC2=C(C=C(C=C2)C2=CC=CS2)C1=O< / smiles> ;

[0113] Intermediate product x2: <smiles> CN1CC2=C(C=C(C=C2)C2=CC=C(I)S2)C1=O< / smiles> .

[0114] (3) Task 3: Formation of multiple reactants:

[0115] Problem (input to a large language model):

[0116] Task Description: Please craft <smiles> COc1ccc2c(c1)cc(C)n2C(=O)c1ccc(Cl)cc1< / smiles> A single-step chemical reaction provides two sets of reactants. If multiple reactants are involved in each reaction scheme, they must be separated by a period (.). All molecules must be represented using the SMILES notation.

[0117] Output format description: "Reactant group 1: <smiles> SMILES notation for the first group of reactants< / smiles> Reactant group 2: <smiles> SMILES notation for the second group of reactants< / smiles> ”;

[0118] Output constraints: No explanation or other information is required; simply return the answer in the specified format.

[0119] Answer (output of the large language model):

[0120] Reactant group 1: <smiles> C=C(C)c1cc(OC)ccc1NC(=O)c1ccc(Cl)cc1< / smiles> ;

[0121] Reactant group 2: <smiles> C=CCc1cc(OC)ccc1NC(=O)c1ccc(Cl)cc1< / smiles> .

[0122] (4) Task 4: Prediction of multiple solutions:

[0123] Problem (input to a large language model):

[0124] Task Description: In chemical reactions, the same combination of reactants may produce different products depending on the choice of reagents or the reaction conditions. There is a combination of reactants... <smiles> O=C1C=2C=CC=C(OC)C2CCC1.O=CC1=CC(OC)=C(OC)C(OC)=C1Cl< / smiles> Please provide two sets of possible products. If a single reaction involves multiple products, separate them with a period (.). All molecules should be presented using the SMILES notation.

[0125] Output format description: Product group 1: <smiles> SMILES notation for the first group of possible products< / smiles> Product group 2: <smiles> SMILES notation for the second group of possible products< / smiles> .

[0126] Output constraints: No explanation or other information is required; simply return the answer in the specified format.

[0127] Answer (output of the large language model):

[0128] Product group 1: <smiles> O=C1C=2C(OC)=CC=CC2CCC1=CC3=CC(OC)=C(OC)C(OC)=C3Br< / smiles> ;

[0129] Product group 2: <smiles> O=C1C=2C=CC=C(OC)C2CCC1=CC3=CC(OC)=C(OC)C(OC)=C3Br< / smiles> .

[0130] (5) Task 5: Prediction of Alternative Co-reactants:

[0131] Problem (Model Input):

[0132] Task Description: Identify two different reactant pathways, both involving compound A and both capable of synthesizing target molecule B. Given: Initial reactant A: <smiles> c1ccc(P(c2ccccc2)c2ccccc2)cc1< / smiles> Target product B: <smiles> ClP(c1ccccc1)c1ccccc1< / smiles> Requirements: Provide two different sets of reactants, each containing compound A; product B must be generated in both cases; solvents, catalysts, or other non-reactants must be excluded.

[0133] Output format instructions: Multiple reactants in the same set should be separated by periods (.) using the SMILES expression. The final answer should be presented in the following format: Reactant set 1: <smiles> [Valid _SMILES_1]< / smiles> Reactant set 2: <smiles> [Valid _SMILES_2]< / smiles> .

[0134] Output constraints: No explanation or other information is required; simply return the answer in the specified format.

[0135] Answer (output of the large language model):

[0136] Reactant set 1: <smiles> ClP(Cl)Cl.c1ccc(P(c2ccccc2)c2ccccc2)cc1< / smiles> ;

[0137] Reactant set 2: <smiles> ClP(Cl)c1ccccc1.c1ccc(P(c2ccccc2)c2ccccc2)cc1< / smiles> .

[0138] Performance Analysis:

[0139] The large language model trained by this invention is compared with mainstream general-purpose large language models (GPT-4o, GPT-4.1, DeepSeek-V3, DeepSeek-R1, GPT-o3 and Qwen3) and chemical-specific large models (ChemDFM and LlaSMol). The comparison results of evaluation metrics (EM, FTS and Validity) on two-step and three-step path reasoning tasks are shown in Table 1 below, and the comparison results of evaluation metrics (EM, FTS and Validity) on multi-solution generation tasks are shown in Table 2 below.

[0140] Table 1

[0141]

[0142] Table 2

[0143]

[0144] See Table 1 above. For non-chain reasoning models, two methods were used for evaluation: standard question (std) and chain of thought (CoT). Experimental results show that the large language model obtained by the method of this invention has achieved significant improvements in both two-step and three-step reasoning tasks: (1) the accurate matching rate in two-step reasoning tasks reached 26.4%, the structural similarity reached 56.59%, surpassing all models except GPT-o3, and the effectiveness reached over 93%; (2) the accurate matching rate in three-step reasoning tasks reached 15.1%, surpassing models such as GPT-4o and DeepSeek-V3, and the structural similarity and effectiveness were 43.10% and 93.80% respectively, surpassing all benchmark models;

[0145] The results in Table 2 above show that: (1) In the task of generating multiple reactants, the large language model obtained by the method of this invention has an accurate matching rate second only to GPT-o3 and a structural similarity second only to GPT-4.1 under the thought chain prompt, and its effectiveness surpasses all benchmark models; (2) In the prediction of multiple products, its performance is average compared to other tasks, and it is on par with DeepSeek-V3 and Qwen3 in terms of accurate matching rate. The reason is that there is a lack of data corresponding to this task in the source dataset. Only about 9,000 data points are included in the 215,000 data points, accounting for only 4.1%; (3) In the prediction of alternative co-reactants, the large language model trained by the method of this invention has an accurate matching rate and structural similarity second only to GPT-o3 and GPT-4.1 under the thought chain prompt, and its effectiveness reaches 96.6%, surpassing all benchmark models.

[0146] Furthermore, the experimental results not only validate the effectiveness of the method of this invention, but also demonstrate the guiding value of the evaluation system in the data construction stage through its performance on the multi-solution product prediction task. By quantitatively analyzing the performance differences under different task subgraph patterns, we can identify task types with weak data coverage and adjust the extraction ratio and retrieval strategy of the corresponding subgraph patterns in the knowledge graph accordingly. This allows us to supplement the corresponding chemical reaction task subgraph instances and optimize the distribution structure of the instruction dataset. This achieves controllability and optimizability in the data construction process without relying on human experience for data completion, demonstrating scalability and sustainable evolution.

[0147] Furthermore, the large language model trained by the method of this invention outperforms existing large chemical language models of similar scale (ChemDFM and LlaSMol) on almost all metrics, contributing a new solution for the application of large language models in chemical reaction pathway prediction tasks.

[0148] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for constructing high-quality fine-tuned chemical data based on knowledge graphs, characterized in that: Specifically, it includes the following steps: (1) Collect multi-source chemical reaction data and construct a chemical reaction knowledge graph. Entity nodes in the chemical reaction knowledge graph represent chemical molecules, and relation edges represent the transformation relationships between chemical molecules through chemical reactions. The relation edges have a clear directionality. The multi-source chemical reaction data includes document text-type chemical reaction data, structured chemical reaction data in patent databases, and reaction examples and substrate-product correspondence data in existing chemical databases. (2) Five task subgraph patterns are set in the chemical reaction knowledge graph. Through the automatic matching mechanism, chemical reaction task subgraph instances that meet the rules of various task subgraph patterns and satisfy logical consistency and factual verifiability are retrieved and extracted from the chemical reaction knowledge graph to form a set of chemical reaction task subgraph instances. (3) Transform each chemical reaction task subgraph instance in the chemical reaction task subgraph instance set into instruction question and answer samples suitable for training large language models, thereby forming an instruction dataset; (4) The instruction dataset is used to train the large language model, so that the large language model learns the rules of the five task subgraph patterns; (5) Construct a test set, conduct inference tests on the trained large language model, and use an evaluation system to systematically and quantitatively evaluate the prediction ability of the large language model. Feed the evaluation results back to the construction of the instruction dataset and the extraction strategy of chemical reaction task subgraph instances, adjust the task subgraph pattern extraction distribution in reverse, and make up for the weak chemical reaction task subgraph instances.

2. The method for constructing high-quality fine-tuned chemical data based on knowledge graphs according to claim 1, characterized in that: The entity nodes are uniquely identified using the standardized molecular SMILES structure representation, and the content on the relation edges includes auxiliary conditions used to describe the chemical reaction.

3. The method for constructing high-quality fine-tuned chemical data based on knowledge graphs according to claim 1, characterized in that: After the acquisition of multi-source chemical reaction data and before the construction of the chemical reaction knowledge graph, the multi-source chemical reaction data is cleaned, specifically as follows: S11. Use cheminformatics tools to perform SMILES syntax parsing, normalization, and structural consistency verification on all chemical molecule entries; S12. The RxnMapper unsupervised atomic mapping tool is used to perform atomic-level mapping of chemical reactants and products, correcting the problem of confusion between reactants and reagents caused by manual annotation or data capture. S13. Duplicate records where the reactant and product sets of a chemical reaction are exactly the same are deduplicated, and only one valid record is retained; abnormal entries where the same chemical molecule is both a reactant and a product are judged as data redundancy and deleted. S14. Use existing large language models to perform text semantic review, verify the contextual consistency of the descriptive text of chemical reaction data, and filter out two types of problematic data: irrelevant reagent errors and multi-step reaction polymerization errors.

4. The method for constructing high-quality fine-tuned chemical data based on knowledge graphs according to claim 1, characterized in that: The five task subgraph modes include a two-step path reasoning task subgraph mode, a three-step path reasoning task subgraph mode, a multi-solution reactant generation task subgraph mode, a multi-solution product prediction task subgraph mode, and an alternative co-reactant prediction task subgraph mode. The two-step path reasoning task subgraph pattern described above is retrieved and extracted from the chemical reaction knowledge graph. A two-step reaction chain in the form of a chemical reaction task subgraph example; The three-step path reasoning task subgraph pattern described above is retrieved and extracted from the chemical reaction knowledge graph. A three-step reaction chain in the form of a chemical reaction task subgraph example; The multi-solution reactant generation task subgraph pattern is to retrieve and extract reaction instances from the chemical reaction knowledge graph that generate the same product from multiple different reactants, and use them as chemical reaction task subgraph instances. The multi-product prediction task subgraph pattern described above retrieves and extracts reaction instances from the chemical reaction knowledge graph that produce multiple different products from the same reactant, and uses them as chemical reaction task subgraph instances. The aforementioned subgraph pattern for predicting alternative co-reactants involves retrieving and extracting reaction instances from the chemical reaction knowledge graph that involve selecting different co-reactants to react with the same reactant and producing the same product, and using these instances as chemical reaction task subgraph instances.

5. The method for constructing high-quality fine-tuned chemical data based on knowledge graphs according to claim 1, characterized in that: The process of transforming each chemical reaction task subgraph instance in the chemical reaction task subgraph instance set into instruction question-and-answer samples suitable for training a large language model, thereby forming an instruction dataset, specifically involves defining five text instruction templates corresponding to chemical reaction task subgraph instances, and transforming each chemical reaction task subgraph instance into instruction question-and-answer samples suitable for training a large language model. Each text instruction template includes a task description, an output format specification, and output constraints. The task description clarifies the reasoning or generation goals that the large language model needs to complete, the output format specification standardizes the representation of molecular SMILES structures and the required number of results output by the large language model, and the output constraints limit the output format of the large language model.

6. The method for constructing high-quality fine-tuned chemical data based on knowledge graphs according to claim 1, characterized in that: After the command question-and-answer samples are transformed and generated, semantic enhancement processing is performed. Through automatic rewriting, synonym replacement and sentence transformation, semantically equivalent but diverse command question-and-answer samples are generated. Combined with the transformed command question-and-answer samples, a command fine-tuning dataset is formed.

7. The method for constructing high-quality fine-tuned chemical data based on knowledge graphs according to claim 1, characterized in that: The training of the large language model using the instruction dataset enables the large language model to learn the rules of five task subgraph patterns. Specifically, LoRA technology is used to train the large language model and to fine-tune the parameters of the large language model.

8. The method for constructing high-quality fine-tuned chemical data based on knowledge graphs according to claim 1, characterized in that: The loss function used to train the large language model is the cross-entropy loss function. , see the following formula (1) for details: （1）； In equation (1), Parameters representing a large language model; This represents the total number of tokens in the vocabulary determined during the training phase of the large language model. This vocabulary is a fixed category space for the output layer of the large language model. The actual labels representing sample instructions and questions; Represents the parameters The large language model under the given conditions predicts the instruction question-answer sample as the first... The probability of each category.

9. The method for constructing high-quality fine-tuned chemical data based on knowledge graphs according to claim 1, characterized in that: The aforementioned construction of a test set is used to perform inference tests on the trained large language model, and an evaluation system is employed to systematically and quantitatively assess the predictive ability of the large language model. Specifically: S51. When constructing the test set, collect multiple real and feasible reference answers for each question to form a reference answer set for that question; S52. Construct an evaluation system. The evaluation system systematically and quantitatively evaluates the predictive ability of the large language model from three dimensions: accurate matching rate, structural similarity, and chemical effectiveness. The exact match rate measures the proportion of a large language model's output that is completely identical to the reference answers in the reference answer set. Structural similarity To characterize the structural similarity between the predicted molecule and the reference molecule, the following steps are taken: First, a Morgan fingerprint is generated for each chemical molecule based on cheminformatics tools, and the structure of the chemical molecule is encoded into a fixed-length binary feature vector. Then, the Tanimoto similarity is used to compare the Morgan fingerprint of the predicted molecule with the Morgan fingerprint of the reference molecule. The calculation formula is shown in the following formula (2): （2）； In equation (2), This represents the Morgan fingerprint vector generated by the predicted structural representation of the SMILES molecule. This represents the Morgan fingerprint vector generated by the structural representation of the reference molecule SMILES; This represents the number of bits that are simultaneously 1 in both the predicted and reference molecule's Morgan fingerprint vectors, corresponding to the number of local structural features shared by the predicted and reference molecules. and These represent the total number of 1 bits in the Morgan fingerprint vectors of the predicted molecule and the reference molecule, respectively, reflecting the number of structural features contained in each molecule. When the predicted molecule and the reference molecule have completely identical structures, the FTS value is 1; when the predicted molecule and the reference molecule do not share any structural features, the FTS value is 0. Chemovalidation is used to evaluate the legality and usability of chemical molecules generated by the large language model at the level of chemical rules. Specifically, it involves: first, inputting the SMILES string output by the large language model into a cheminformatics tool for parsing and checking whether it conforms to the SMILES syntax specification; then determining whether the SMILES string can be successfully converted into a valid molecular object. If the SMILES string can be correctly parsed by the cheminformatics tool and generate a valid molecular structure, the prediction result is considered valid; otherwise, it is considered invalid. S53. Construct a scoring mechanism based on multiple answer sets: when any two paths output by the large language model appear in the reference answer set, then score 1 point; If the output of the large language model has only one path that appears in the reference answer set, it is scored as 0.5 points; if the output of the large language model has no matching path with the reference answer set, it is scored as 0 points.

Images