Experiment scheme automatic generation system based on knowledge graph and large language model

Abstract

The application discloses an experimental scheme automatic generation system based on a knowledge graph and a large language model, belongs to the field of artificial intelligence and chemical experiment automation, and comprises the following modules: an experimental knowledge graph construction module, which can extract and structure experimental knowledge from scientific literature based on a large language model, and construct an experimental knowledge graph; an experimental knowledge graph intelligent agent, which can generate a hierarchical structure of domain knowledge outline according to the experimental knowledge graph, and perform multi-strategy fusion retrieval to obtain accurate structured facts and corresponding original text evidence; and an experimental process generation module, which can generate a complete experimental process to obtain an experimental scheme by four steps of structure guidance, detail decomposition, knowledge retrieval and chapter construction on the structured facts and original text evidence based on the hierarchical domain knowledge outline and a large language model. The system can automatically generate an experimental scheme with high scientificity, traceability and executability from unstructured literature, and significantly improves the efficiency and quality of experimental scheme construction.

Description

Automatic Experiment Plan Generation System Based on Knowledge Graph and Large Language Model Technical Field

[0001] This invention relates to the fields of artificial intelligence and chemical experiment automation technology, and in particular to an automatic experimental scheme generation system based on knowledge graphs and large language models. Background Technology

[0002] Experimental knowledge in chemistry and materials science is primarily found in scientific literature, which essentially serves as a manual for recording experimental methods. However, existing experimental procedures are written for human reading rather than machine reading or automated execution, and are not optimized for knowledge reuse. Key steps and parameters are deeply embedded in narrative texts, scattered across different chapters and modalities, and formatted inconsistently across different papers and journals. This situation leads to three major practical problems: First, it hinders accessibility for researchers—the free-text format makes it difficult to locate, compare, and control experimental details across literature, slowing down experimental reproduction and optimization. Second, it weakens machine operability—autonomous laboratories require standardized entities, orderly operations, and protocol-level instructions with quantified parameters, elements that are difficult to obtain directly from existing literature. Third, it limits the effectiveness of large language models—they process structured input far better than ad-hoc, non-standardized text. Therefore, transforming scientific literature into standardized experimental protocols can significantly improve knowledge retrieval, reasoning capabilities, and end-to-end experimental planning from conceptual design to parameter selection.

[0003] The limitations of existing solutions exacerbate these challenges. Traditional methods provide two international research case studies, but due to the fragmented nature of the output, they struggle to effectively generate experimental protocols from the papers. While the large-scale model solutions developed concurrently for two international research case studies are powerful, they are prone to missing crucial details and carry a high risk of misleading results, making it impossible to faithfully recreate the experimental procedures documented in the literature. In other words, although scientific literature is the primary carrier of experimental knowledge, most experimental protocol details are scattered throughout these scientific documents in free-text form. This inconsistency hinders researchers' reuse of knowledge and prevents autonomous laboratories from executing experiments end-to-end. Traditional methods often struggle to systematically and automatically generate experimental protocol steps from scientific literature, while existing solutions relying solely on large language models fail to reconstruct experimental protocols with high fidelity.

[0004] In view of this, the present invention is hereby proposed. Summary of the Invention

[0005] The purpose of this invention is to provide an automatic experimental scheme generation system based on knowledge graphs and large language models, which can automatically generate experimental operation steps from scientific literature, reconstruct experimental schemes with high fidelity, and thus solve the above-mentioned technical problems existing in the prior art.

[0006] The objective of this invention is achieved through the following technical solution:

[0007] An automatic experimental design generation system based on knowledge graphs and large language models is used to automatically construct experimental designs from scientific literature using large language models, including:

[0008] The system includes modules for constructing an experimental knowledge graph, generating a domain knowledge outline, creating an experimental knowledge graph agent, and generating an experimental workflow.

[0009] The experimental knowledge graph construction module can extract and structure experimental knowledge from scientific literature based on a large language model, and use the experimental knowledge to construct an experimental knowledge graph.

[0010] The domain knowledge outline generation module can generate a hierarchical domain knowledge outline based on a large speech model according to a pre-defined expert template.

[0011] The experimental knowledge graph intelligent agent is communicatively connected to the experimental knowledge graph construction module and can perform multi-strategy fusion retrieval on the experimental knowledge graph according to the domain knowledge outline to obtain the structured facts corresponding to each sub-chapter of the domain knowledge outline.

[0012] The experimental process generation module is communicatively connected to the domain knowledge outline generation module and the experimental knowledge graph agent. It can interact with the experimental knowledge graph agent based on the large language model, obtain the structured facts corresponding to each sub-chapter according to the domain knowledge outline and construct the experimental scheme of each sub-chapter, and combine the contents of the experimental schemes of each sub-chapter to generate a complete experimental scheme.

[0013] Compared with existing technologies, the automatic experimental scheme generation system based on knowledge graphs and large language models provided by this invention has the following advantages:

[0014] By combining an experimental knowledge graph construction module, a domain knowledge outline generation module, an experimental knowledge graph intelligent agent, and an experimental procedure generation module, this system couples the domain knowledge graph with a large language model. The experimental knowledge graph extracts and standardizes experimental information (such as reagents, equipment, operations, and conditions) into traceable triples. Subsequently, the large language model generates a domain knowledge outline based on a pre-defined outline by experts. Through retrieval-enhanced hierarchical prompts, a complete operational experimental procedure is generated, resulting in the final experimental plan. This system can automatically transform unstructured experimental information from published scientific literature into structured, executable experimental plans. It solves the problem that traditional methods often struggle to systematically generate experimental procedure steps from scientific literature, while existing methods relying solely on large language models often fail to reconstruct experimental plans with high fidelity. The experimental plans generated by this system outperform baseline models in terms of accuracy, information coverage, and operational clarity. By transforming literature into protocol-level, machine-operable data, it bridges the critical gap between published research and automated experiments, enhancing the automated planning capabilities of experiments from conceptual design to parameter selection. Attached Figure Description

[0015] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 is a block diagram of the automatic generation system for experimental schemes based on knowledge graphs and large language models provided in an embodiment of the present invention.

[0017] Figure 2 is a flowchart of the overall processing flow of the experimental scheme automatic generation system provided in the embodiment of the present invention.

[0018] Figure 3 is a flowchart of the experimental knowledge graph construction module of the experimental scheme automatic generation system provided in the embodiment of the present invention.

[0019] Figure 4 is a flowchart of the experimental knowledge graph intelligent agent of the experimental scheme automatic generation system provided in the embodiment of the present invention.

[0020] Figure 5 is a flowchart of the experimental process generation module of the automatic experimental scheme generation system provided in the embodiment of the present invention.

[0021] Figure 6 is a flowchart of the evaluation and verification module of the experimental scheme automatic generation system provided in the embodiment of the present invention.

[0022] Figure 7 is a schematic diagram of the comparative evaluation results of the content integrity dimension of the evaluation and verification module of the experimental scheme automatic generation system provided in the embodiment of the present invention.

[0023] Figure 8 is a schematic diagram of the comparative evaluation results of the information accuracy dimension of the evaluation and verification module of the experimental scheme automatic generation system provided in the embodiment of the present invention.

[0024] Figure 9 is a schematic diagram of the comparative evaluation results of the format standardization dimension of the evaluation and verification module of the experimental scheme automatic generation system provided in the embodiment of the present invention.

[0025] Figure 10 is a schematic diagram showing the comparative evaluation results of the operational usability dimension of the evaluation and verification module of the experimental scheme automatic generation system provided in the embodiment of the present invention. Detailed Implementation

[0026] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the specific content of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments, which do not constitute a limitation of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the protection scope of the present invention.

[0027] First, the following explanations are provided for the terms that may be used in this article:

[0028] The term "and / or" means that either or both can be achieved simultaneously. For example, X and / or Y means that it includes both "X" or "Y" as well as the three cases of "X and Y".

[0029] The terms "comprising," "including," "containing," "having," or other similar semantic descriptions should be interpreted as non-exclusive inclusion. For example, including a technical feature element (such as raw material, component, ingredient, carrier, dosage form, material, size, part, component, mechanism, device, step, process, method, reaction conditions, processing conditions, parameter, algorithm, signal, data, product or article of manufacture, etc.) should be interpreted as including not only the expressly listed technical feature element, but also other technical feature elements that are not expressly listed and are well-known in the art.

[0030] The term "composed of" excludes any technical features not expressly listed. When used in a claim, it closes the claim to exclude all technical features other than those expressly listed, except for associated conventional impurities. If the term appears only in a clause of a claim, it limits the claim to the elements expressly listed in that clause; elements recited in other clauses are not excluded from the overall claim.

[0031] Unless otherwise explicitly specified or limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to fixed connections, detachable connections, or integral connections; they can refer to mechanical connections or electrical connections; they can refer to direct connections or indirect connections through an intermediate medium; and they can refer to the internal connection between two components. Those skilled in the art can understand the specific meaning of the above terms in this document according to the specific circumstances.

[0032] When concentration, temperature, pressure, size, or other parameters are expressed as numerical ranges, such ranges should be understood to specifically disclose all ranges formed by any pairing of upper limits, lower limits, or preferred values ​​within that range, regardless of whether the range is explicitly stated; for example, if the numerical range "2 to 8" is stated, then that range should be interpreted to include ranges such as "2 to 7", "2 to 6", "5 to 7", "3 to 4 and 6 to 7", "3 to 5 and 7", "2 and 5 to 7", etc. Unless otherwise stated, the numerical ranges described herein include both their endpoints and all integers and fractions within that range.

[0033] The solution provided by this invention will be described in detail below. Contents not described in detail in the embodiments of this invention are prior art known to those skilled in the art. Where specific conditions are not specified in the embodiments of this invention, they shall be performed according to conventional conditions in the art or conditions recommended by the manufacturer. Reagents or instruments used in the embodiments of this invention whose manufacturers are not specified are all conventional products that can be purchased commercially.

[0034] As shown in Figures 1 and 2, this embodiment of the invention provides an automatic experimental scheme generation system based on knowledge graphs and large language models. This system automatically constructs experimental schemes from scientific literature using a large language model, including:

[0035] The system includes modules for constructing an experimental knowledge graph, generating a domain knowledge outline, creating an experimental knowledge graph agent, and generating an experimental workflow.

[0036] The experimental knowledge graph construction module can extract and structure experimental knowledge from scientific literature based on a large language model, and use the experimental knowledge to construct an experimental knowledge graph.

[0037] The domain knowledge outline generation module can generate a hierarchical domain knowledge outline based on a large speech model according to a pre-defined expert template.

[0038] The experimental knowledge graph intelligent agent is communicatively connected to the experimental knowledge graph construction module and can perform multi-strategy fusion retrieval on the experimental knowledge graph according to the domain knowledge outline to obtain the structured facts corresponding to each sub-chapter of the domain knowledge outline.

[0039] The experimental process generation module is communicatively connected to the domain knowledge outline generation module and the experimental knowledge graph agent. It can interact with the experimental knowledge graph agent based on the large language model, obtain the structured facts corresponding to each sub-chapter according to the domain knowledge outline and construct the experimental scheme of each sub-chapter, and combine the contents of the experimental schemes of each sub-chapter to generate a complete experimental scheme.

[0040] Preferably, the system further includes an evaluation and verification module, which is connected to the experimental procedure generation module and can perform objective indicator evaluation, text quality evaluation, and practicality evaluation on the experimental plan generated by the experimental procedure generation module.

[0041] In some preferred embodiments, the evaluation and verification module performs objective indicator evaluation, text quality evaluation, and practicality evaluation on the experimental plan generated by the experimental procedure generation module in the following manner:

[0042] Objective metrics for evaluation include: the accuracy of the experimental protocol. and the recall rate of the experimental design ;in,

[0043] Determine the precision of the experimental protocol as follows: :

[0044] The experimental design was divided into modules based on the domain knowledge outline, and all entities were extracted from each module. A large language model was used as the evaluator to determine whether the description of each entity was consistent with the source text of the scientific literature, and the accuracy of the experimental design was measured. N is defined as the number of entities accurately described in the experimental design. acc With the total number of entities N protocol The ratio:

[0045] (8);

[0046] Determine the recall rate of the experimental protocol as follows: :

[0047] By recall rate The effectiveness of the experimental design in covering predefined target information is evaluated. Based on the requirements of the domain knowledge outline, all target entities are identified, and the recall rate of the experimental design is assessed. Defined as the number of target entities that are successfully recalled and accurately described, N re The total number N of target entities defined in the domain knowledge outline template The ratio:

[0048] (9);

[0049] Fact reproducibility is assessed by a single comprehensive evaluation using the harmonic mean of precision and recall, i.e., the F1 score.

[0050] The G-Eval framework was used to evaluate the text of the generated experimental schemes, assessing the language quality and logical clarity of the schemes through coherence, consistency, fluency, and relevance.

[0051] A large language model was used as an evaluator to compare the content of the generated experimental protocol with a pre-obtained standard questionnaire containing multiple indicators and to score it on a 10-point scale; each indicator in the standard questionnaire was a standard answer retrieved from the original literature using an experimental knowledge graph.

[0052] Preferably, in the above system, the experimental knowledge graph construction module extracts and structures experimental knowledge from scientific literature based on a large language model in the following manner, and constructs an experimental knowledge graph using the extracted experimental knowledge, including:

[0053] Step 11: Convert the format of the scientific documents, preserve the hierarchical structure of the documents, and filter out irrelevant parts of the document content through a two-layer content screening mechanism of rule filtering and keyword and paragraph semantic feature matching to obtain a pure experimental procedure corpus. The first layer of the two-layer content screening mechanism is based on rule filtering of non-experimental paragraphs such as references and theoretical analysis in the documents; the second layer is to further filter by combining keyword and paragraph semantic feature matching to obtain a high-confidence corpus containing only experimental steps and experimental conditions.

[0054] Step 12: Using a hybrid strategy combining rule-based and large language model-assisted methods, entity-relationship-attribute triples are extracted from the experimental corpus, and the obtained triples are used to construct an experimental knowledge graph.

[0055] Preferably, in step 12 of the above system, the entities extracted from the experimental step corpus are: predefined entity types that conform to the domain to which the constructed experimental scheme belongs, and the extracted entities are used as nodes of the constructed experimental knowledge graph;

[0056] The extracted relationship is: a verb or verb phrase that can directly link the nodes corresponding to two entities;

[0057] The extracted attributes are: information representing the identity, status, behavior, and metrics of the node corresponding to the entity.

[0058] For example, in scientific literature on Fischer-Tropsch synthesis experiments, the extracted entities are: CatalystPoisoning, Ee@Mn@0.5Si-C, Reaction Chamber, etc., and the extracted relationships are: is_reduced_by, has_space_time_velocity, undertakes_deactivation_due_to, etc., and the extracted attributes are: reaction:H2_CO_ratio, Activation:reduction_gas, eactivation:deactivation_rate, etc.; in scientific literature on electrochemical experiments, the extracted entities are: Pec_Water_Splitting_System, Photocorrosion_Stability, Pec_Performance, etc., and the extracted relationships are: shows_capacity, has_crystal_structure, stores_energy, etc., and the extracted attributes are: Performance:Voltage, Structure:Porosity, Synthesis:Electrochemical_Synthesis, etc.

[0059] Preferably, in the above system, the hierarchical domain knowledge outline generated by the domain knowledge outline generation module includes a first-level outline and a second-level outline, wherein...

[0060] The first-level outline is an expert template for the content required to establish experimental plans in related fields;

[0061] The secondary outline is a set of structured prompts based on the experimental objectives, requirements, format specifications, scheme examples, and special notes of each sub-chapter in the primary outline. It is then used to generate detailed content for each sub-chapter based on a large language model.

[0062] Preferably, in the above system, the experimental knowledge graph agent performs multi-strategy fusion retrieval on the experimental knowledge graph based on a large language model according to the domain knowledge outline to obtain the structured facts corresponding to each sub-chapter of the domain knowledge outline, including:

[0063] The experimental knowledge graph agent takes each sub-chapter of the domain knowledge outline as input, extracts the content of each sub-chapter to obtain a summary, and uses the summary of each sub-chapter as input to the large language model to obtain the experimental sub-question set corresponding to the summary content output by the large language model. Then, each question in the experimental sub-question set is used as input to the experimental knowledge graph agent, enabling the agent to retrieve the corresponding answer from the experimental knowledge graph through structured query based on the input question. The input question and the corresponding answer are then combined to form question-answer pairs, and the question-answer pairs are combined into the question-answer pair set of each sub-chapter as the structured facts corresponding to each sub-chapter.

[0064] Preferably, in the above system, the experimental knowledge graph agent retrieves the experimental knowledge graph through two structured queries: attribute retrieval and entity retrieval.

[0065] The node that matches the attribute in the experimental knowledge graph with the query keywords determined based on the input question is identified by attribute retrieval. Its formal expression is as follows:

[0066] (1);

[0067] in, It is the result set of attribute matching nodes; v represents a node in the experimental knowledge graph; Represents all nodes in the experimental knowledge graph; Represents a node The set of attributes; It is a set of keywords extracted from the query; Represents the empty set; condition Ensure that a node is included only if its attributes share at least one keyword with the query.

[0068] Entity retrieval matches proper nouns or numerical entities in the query with corresponding nodes in the experimental knowledge graph based on semantic similarity, as follows:

[0069] (2);

[0070] in, It is the result set of entity matching nodes; It is a node The corresponding entity content; Symbols represent similarity matching; Represents the set of entities in the query;

[0071] The experimental knowledge graph agent uses text evidence recall to attach corresponding original text fragments from scientific literature to each retrieved node or relation. It then fuses, deduplicates, and expands the retrieved information to include adjacent nodes, selecting the K most relevant search results to form the final context, defined as follows:

[0072] (3);

[0073] Where R represents the final context formed; TopK(·) represents the selection function that selects the K most relevant search results; These are fusion and expansion functions;

[0074] The final context obtained by the experimental knowledge graph agent is used as the answer. The input question and the corresponding answer are combined to form a question-answer pair. The question-answer pairs are combined into a question-answer pair set for each sub-chapter as the obtained accurate structured facts.

[0075] Preferably, in the above system, the experimental process generation module interacts with the experimental knowledge graph agent based on a large language model in the following manner: it obtains the structured facts corresponding to each sub-chapter according to the domain knowledge outline and constructs them as experimental schemes for each sub-chapter; then, it combines the contents of the experimental schemes for each sub-chapter to generate a complete experimental scheme, including:

[0076] Step 31: Interact with the experimental knowledge graph agent through the large language model, and use the domain knowledge outline as the input of the experimental knowledge graph agent, so that the experimental knowledge graph agent can generate a global summary that summarizes the key compounds, synthetic routes and main conclusions according to the guidance of the domain knowledge outline.

[0077] Step 32: Guided by the structured prompts corresponding to each chapter of the global summary and domain knowledge outline, the experimental knowledge graph agent performs adaptive exploration processing on the experimental knowledge graph through the large language model to generate a fine-grained sub-query set to verify the experimental knowledge graph.

[0078] Step 33: Input the generated fine-grained sub-query set into the experimental knowledge graph agent. The structured facts used by the knowledge graph agent to locate knowledge in the experimental knowledge graph through a multi-strategy retrieval mechanism are used as the most relevant answers. The most relevant answers retrieved are returned in the form of structured question-answer pairs.

[0079] Step 34: Build a context-aware prompt for each chapter. Guided by the prompts in each chapter, generate the final text using a large language model as the experimental scheme for each chapter.

[0080] Step 35: After generating the experimental schemes for each chapter, aggregate the experimental schemes of each chapter in a predefined order to form a complete and logically coherent overall experimental scheme.

[0081] Preferably, in step 32 above, the formal representation of generating the fine-grained sub-query set to verify the experimental knowledge graph is as follows:

[0082] (4);

[0083] in, Indicates a specific chapter The set of subqueries; Indicates a specific chapter; Represents a global summary; It is a subquery generation function, based on a specific chapter. Outline template and global summary As input, output this specific chapter. subquery set .

[0084] Preferably, in step 33 above, each question-answer pair contains a precise answer and supporting evidence fragments from the original text in the scientific literature. The process of obtaining question-answer pairs based on experimental knowledge graph retrieval is represented as follows:

[0085] (5);

[0086] in, It generates a set of question-answer pairs. The function.

[0087] In step 34, the construction of a context-aware prompt for each chapter integrates three key components: a chapter-specific knowledge outline for structure and formatting. A global summary used for narrative guidance And all relevant question-and-answer pairs for verified factual details. ;

[0088] single chapter Corresponding experimental scheme The generation definition is: (6);

[0089] Among them, G LLM (·) indicates that the experimental scheme for this chapter is constructed by combining the knowledge outline of a specific chapter, the global summary, and all relevant question-and-answer pairs from the large language model; i=1,2,……n, where n represents the total number of chapters;

[0090] In step 35, the process of aggregating the experimental schemes of each chapter in a predefined order is as follows:

[0091] (7);

[0092] in, Represents the overall experimental scheme obtained from the aggregation; symbol Indicates all The structured arrangement of chapter content.

[0093] In summary, the automated experimental protocol generation system based on knowledge graphs and large language models provided by this invention couples domain knowledge graphs with large language models, transforming unstructured experimental information from published scientific literature into structured, executable experimental protocols. Furthermore, the system introduces a multi-dimensional evaluation system, measuring factual fidelity through entity-level precision and recall, assessing language quality through coherence, consistency, fluency, and relevance, and evaluating practical usability using expert questionnaires. Using the Fischer-Tropsch synthesis reaction as a validation scenario, the experimental protocols generated by this system outperform the baseline model in terms of accuracy, information coverage, and operational clarity. By transforming literature into protocol-level, machine-operable data, this system bridges the critical gap between published research and automated experiments, enhancing experimental planning capabilities throughout the entire process from conceptual design to parameter selection.

[0094] To more clearly demonstrate the technical solution and its effects provided by the present invention, the following detailed description of the solution provided by the embodiments of the present invention is provided with reference to specific examples.

[0095] Example 1

[0096] As shown in Figures 1 and 2, this embodiment provides an automatic experimental protocol generation system based on knowledge graphs and a large language model. This system integrates a domain knowledge graph and a large language model: the experimental knowledge graph provides standardized and traceable representations of experimental information, including reagents, equipment, operations, and conditions; subsequently, the large language model, guided by an expert-developed outline and combined with factual evidence fragments retrieved from the experimental knowledge graph, generates a complete and logically ordered operational flow as the final experimental protocol. This retains source information (each step can be traced back to the original text) and eliminates ambiguity through a controlled vocabulary and standardized units, ultimately outputting a result that is both human-readable and machine-executable. Figure 1 illustrates the complete process of constructing an experimental protocol from scientific literature, comprising three stages: Extraction: Extracting knowledge from scientific literature and organizing it into a knowledge graph and an experimental knowledge graph agent to build a solid knowledge foundation. Generation: The experimental knowledge graph agent and the large language model interact and collaborate to generate an experimental protocol that is both knowledge-based and operable. Evaluation: The generated protocol is compared with the experimental knowledge graph agent and the original literature through an evaluation and verification module, evaluating it from the dimensions of accuracy, consistency, and scientific reliability.

[0097] To evaluate the experimental schemes generated by the system, a comprehensive evaluation system for scientific research scenarios was constructed: First, the factual fidelity and coverage of the target process elements were quantified through entity-level precision and recall; second, text evaluation based on a large language model (coherence, consistency, fluency, relevance) was used to capture text clarity and internal logic; finally, an expert questionnaire was used to score from dimensions such as completeness, accuracy, format standardization, and operational feasibility—all of which are indispensable standards for achieving reliable reproducibility and automation.

[0098] The system was instantiated and stress-tested using Fischer-Tropsch synthesis (FTS) as a specific case study. Fischer-Tropsch synthesis involves multi-stage catalyst preparation, pretreatment, and tightly coupled operating conditions; its complexity precisely exposes the limitations of free-form reporting systems. By constructing a dedicated experimental knowledge graph for Fischer-Tropsch synthesis, defining an outline including chapter-level requirements such as standardized materials and process quantification constraints, and implementing a hierarchical generation workflow guided by the experimental knowledge graph, the system in this embodiment produced protocol-level output results as the final experimental scheme in tests with 125 papers. Compared to a strongly prompted baseline model, this system demonstrated superior performance in factual accuracy, breadth of information coverage, and clarity of text manipulation, achieving the current best performance. This invention provides a feasible path from scientific literature to the laboratory: by generating standardized experimental schemes that are easier for humans to follow and more suitable for large language model-assisted design and autonomous laboratory execution, it promotes the innovation of scientific research paradigms.

[0099] The system in this embodiment includes: an experimental knowledge graph construction module, a domain knowledge outline generation module, an experimental knowledge graph intelligent agent, and an experimental process generation module; wherein, the experimental knowledge graph construction module can extract and structure experimental knowledge from scientific literature based on a large language model, and construct an experimental knowledge graph containing multiple types of core entities through experimental knowledge.

[0100] To provide a factual basis for automated experimental protocol generation, this invention constructs a large-scale chemical experimental knowledge graph through an experimental knowledge graph construction module. The automated processing flow of this module successfully transforms unstructured scientific literature in PDF format into a structured experimental knowledge network (see Figure 3). The process first uses the open-source tool pdf-marker to convert the PDF to Markdown format, preserving the hierarchical structure of the scientific documents. Then, a two-layer content filtering mechanism removes irrelevant parts (such as references and theoretical discussions), resulting in a clean corpus of experimental procedures. Figure 3 illustrates the overall workflow for transforming scientific literature into a structured knowledge graph. This process includes several main stages: document parsing and information extraction mapping. In this stage, text, formulas, and tables are parsed, filtered, and refined into key entities (e.g., reagents, methods, results). Document parsing specifically extracts plain text, formulas, and tables from the PDF of the scientific literature, ensuring the preservation of their structure and semantics. This stage involves processing scanned and raw documents, transforming unstructured content into a machine-readable format to lay the data foundation for subsequent analysis. Information extraction and mapping specifically involves cleaning the parsed content and identifying key information. Using a large language model, key entities (such as materials, methods, and parameters) are identified, tables and / or formulas are parsed, and they are mapped to predefined semantic categories, achieving preliminary structuring. Next is knowledge graph construction. In this stage, entities are standardized, mapped to semantic categories, and interconnected to form a structured network. Specifically, this is achieved by normalizing entity names, associating with external ontologies and databases, and constructing a knowledge graph through entity and relation encoding. The final experimental knowledge graph provides a solid foundation for downstream scientific applications (such as automated experimental protocol generation).

[0101] From the corpus of experimental procedures, a hybrid strategy combining rule-based and large language model-assisted methods was employed to extract entity-relation-attribute triples. This process identified five key entity types in the Fischer-Tropsch synthesis domain—reagent, device, action, condition, product, or intermediate—which form the semantic backbone of the knowledge graph. A standardized relation dictionary (e.g., dissolve in, heat to) and normalized vocabulary ensured data consistency. For example, the sentence "dissolve cobalt nitrate in deionized water" was parsed into structured relations. The final constructed knowledge graph constitutes a comprehensive and logically consistent network of experimental facts, providing a reliable factual basis for the generation of downstream automated experimental protocols.

[0102] The domain knowledge outline generation module communicates and connects with the experimental process generation module. It provides a top-level domain knowledge outline for subsequent generation tasks, which is generated by combining a template pre-designed by experts, ensuring that the generated experimental plan is complete and has a clear logical structure.

[0103] For specific research topics like Fischer-Tropsch synthesis, the domain knowledge outline uses Markdown to define a hierarchical structure. The first-level outline establishes the main parts of the experimental protocol, such as "Experimental Materials," "Synthetic Methods," and "Experimental Procedures." This basic structure is refined into second-level outlines using structured cue words. These cue words elaborate on attributes such as the objectives, requirements, and format of each module. For example, in the "Experimental Materials" section, the second-level outline mandates standardized reagent and equipment details (e.g., purity, supplier, model). Similarly, for "Experimental Procedures," it mandates quantitative constraints on dosages and conditions, prohibiting the use of vague terminology. This hierarchical design ensures comprehensive coverage and scientific rigor while maintaining flexibility for different experimental scenarios. The process comprises two main parts: outline generation (top area): On the left, the large language model generates an initial high-level outline (e.g., materials, methods, procedures, results) based on the researcher's cue words. On the right, using structured cue words, the large language model refines the outline into detailed sub-sections (e.g., reagents, equipment, preparation), making it more specific and actionable.

[0104] The experimental knowledge graph agent communicates with both the experimental knowledge graph construction module and the experimental process generation module. It can perform multi-strategy fusion retrieval on the experimental knowledge graph constructed by the experimental knowledge graph construction module based on the domain knowledge outline generated by the domain knowledge outline generation module, which is input by the experimental process generation module, to obtain accurate structured facts.

[0105] By combining the factual basis of the experimental knowledge graph with the reasoning and linguistic capabilities of a large language model, structured knowledge is transformed into a coherent experimental scheme. This experimental knowledge graph agent interacts with the experimental workflow generation module, retrieving structured facts corresponding to each sub-chapter of the domain knowledge outline based on the large language model. Figure 4 illustrates the workflow of constructing the Fischer-Tropsch synthesis experimental template by combining the large language model and the experimental knowledge graph. Knowledge Retrieval (Bottom Area): The experimental knowledge graph agent retrieves domain knowledge from the experimental knowledge graph through entity search, attribute search, and text search to support the refinement of the domain knowledge outline and links it back to the experimental knowledge graph and original literature.

[0106] The experimental knowledge graph agent can perform dynamic information retrieval, responding to generated queries during the experimental scheme creation process and retrieving precise structured factual information. Drawing on previous work, this experimental knowledge graph agent adopts a multi-strategy retrieval mechanism that combines structured fact retrieval and textual evidence recall (see the bottom of Figure 4).

[0107] The retrieval process of the experimental knowledge graph agent begins with two parallel structured queries. Attribute retrieval identifies nodes in the experimental knowledge graph whose attributes match the query keywords (e.g., experimental operation, reaction conditions). Its formal expression is:

[0108] (1);

[0109] in, Represents all nodes in a knowledge graph; It is a set of keywords extracted from the query; Represents a node The set of attributes; It is the result set of attribute matching nodes; Represents the empty set. Condition A node is included only if its attributes share at least one common keyword with the query. Simultaneously, entity retrieval matches proper nouns or numerical entities in the query with corresponding nodes in the knowledge graph based on semantic similarity.

[0110] (2);

[0111] in, Represents the set of entities in the query; It is a node The corresponding entity content; Symbols represent similarity matching; It is the result set of entity matching nodes.

[0112] To complement these structured queries, text evidence recall appends raw text fragments from the source documents to each retrieved node or relation. This combination of structured facts and their original context ensures the integrity and consistency of the information. Finally, the retrieved information is fused, deduplicated, and expanded to include adjacent nodes. The top K most relevant results are selected to form the final context of the large language model. This step is defined as:

[0113] (3);

[0114] in, It is a fusion and expansion function. This fact + context pattern provides accurate, reliable, and traceable information for each generation step.

[0115] The experimental process generation module communicates with both the domain knowledge outline generation module and the experimental knowledge graph agent. It can interact with the experimental knowledge graph agent based on a large language model. According to the predefined hierarchical domain knowledge in the domain knowledge outline, it obtains the structured facts corresponding to each sub-chapter of the domain knowledge outline through four steps: structure guidance, detail decomposition, knowledge retrieval, and chapter construction, and constructs the experimental plan for each sub-chapter. The experimental plan content of each sub-chapter is then combined to generate a complete experimental plan.

[0116] The experimental workflow generation module interacts with the experimental knowledge graph agent, employing a hierarchical, four-step workflow to generate experimental plans, ensuring logical coherence at the macro level and factual accuracy at the micro level, as shown in Figure 5. This module systematically decomposes complex generation tasks into manageable, fact-checked stages. Figure 5 illustrates the overall flowchart of the automatic experimental plan generation system based on knowledge graphs and large language models. It uses a hierarchical, four-step workflow to generate experimental plans, ensuring logical coherence at the macro level and factual accuracy at the micro level. This workflow aims to systematically decompose complex generation tasks into manageable, fact-checked stages.

[0117] The processing flow of this experimental workflow generation module does not begin with details, but rather by first establishing a global semantic context through structural guidance. In this initial stage, the large language model interacts with the experimental knowledge graph agent to generate a global summary under the guidance of the domain knowledge outline. This summary summarizes the core experiments of the source scientific literature text, outlining key compounds, synthetic routes, and main conclusions. More importantly, it provides a unified global context that is continuously injected as background knowledge into all subsequent steps, ensuring that the model remains consistent with the narrative of the source scientific literature and preventing content deviation.

[0118] Following this, the experimental procedure generation module continues to break down each scheme section into details. Guided by a global summary and chapter-specific structured prompts, the large language model performs an adaptive exploration step. Instead of directly generating content, it formulates a series of fine-grained subqueries that need to be validated against the knowledge graph. For example, when processing the experimental procedure section, it might generate a query like: "What is the specific mass (in grams) of cobalt nitrate used in the synthesis of catalyst A?". This step effectively transforms a vague content generation goal into a set of clear, actionable information retrieval tasks. This subquery generation process is formalized as follows:

[0119] (4);

[0120] in, It is a subquery generation function, which is based on a specific chapter. Outline template and global summary As input, output the set of subqueries for this chapter. .

[0121] Generated subqueries The answers were then assigned to an experimental knowledge graph agent for knowledge retrieval. This agent utilized its multi-strategy retrieval mechanism to locate the most relevant answers within the experimental knowledge graph and returned the results in structured question-and-answer pairs. Crucially, each question-and-answer pair contained a precise answer and supporting evidence from the original scientific literature text. This step anchors the large language model's generation process within the structured facts of the experimental knowledge graph, significantly reducing the risk of factual errors or illusions. The knowledge graph-based question-and-answer process can be represented as:

[0122] (5);

[0123] in, It generates a set of question-answer pairs. The function.

[0124] Finally, the experimental workflow generation module enters the chapter construction phase. A context-aware cue is built for each chapter, integrating three key components: a global summary for narrative guidance (…). Chapter-specific knowledge outlines for structure and format ( ), and all related question-and-answer pairs ( This is used for verified factual details. Guided by this complete information, the large language model generates the final text, organizing discrete facts into logically fluent and professionally written paragraphs. (Single chapter) Content The generation definition is:

[0125] (6);

[0126] Where i = 1, 2, ..., n, n represents the total number of chapters;

[0127] After all chapters are generated, they are assembled in a predefined order to form a final, structurally complete, and logically coherent experimental design. This assembly process is defined as follows:

[0128] (7);

[0129] Among them, symbols Indicates all The structured arrangement of chapter content.

[0130] Furthermore, the system in this embodiment also includes: an evaluation and verification module, which, together with the experimental procedure generation module, can perform objective indicator evaluation, text quality evaluation, and practicality evaluation on the experimental plan generated by the experimental procedure generation module.

[0131] This evaluation and verification module comprehensively assesses the quality of the generated experimental protocols. It is a multi-layered evaluation system that integrates objective indicators, model-based text quality assessment, and a practical questionnaire for usability evaluation. The overall logic of this module systematically examines the accuracy, fluency, completeness, and executability of the output, as shown in Figure 6. Figure 6 illustrates the system's multi-dimensional evaluation framework for experimental protocols, covering three complementary dimensions: Objective Evaluation: Measure factual correctness by examining the accuracy of entities relative to scientific literature and their recall relative to the outline. Text Evaluation: Assess language quality through coherence, fluency, consistency, and relevance. Questionnaire Evaluation: Experimental researchers design evaluation questionnaires, while the evaluation agent analyzes responses based on literature knowledge to assess usability and completeness.

[0132] The objective evaluation of this verification module uses precision and recall metrics to quantify the factual fidelity of the generated experimental protocols relative to the original scientific literature. To measure precision, the experimental protocols are first segmented into modules based on a domain knowledge outline, and all entities are extracted from each module. A large language model, acting as the evaluator, then determines whether the description of each entity is consistent with the source text of the original scientific literature. Precision N is defined as the number of entities accurately described in the experimental design. acc With the total number of entities N protocol The ratio:

[0133] (8);

[0134] To measure recall, the extent to which the experimental design covers predefined target information is evaluated. Based on the requirements of the domain knowledge outline, all target entities are identified. Recall rate. Defined as the number of target entities that are successfully recalled and accurately described, N re The total number N of target entities defined in the outline template The ratio:

[0135] (9);

[0136] The F1 score provides a single, comprehensive assessment of reproducibility; it is the harmonic mean of precision and recall.

[0137] In addition to factual accuracy, this invention also conducts textual evaluation of the experimental design, assessing language quality and logical clarity. Specifically, it employs the G-Eval framework for automated evaluation, covering four key dimensions: coherence (evaluating the logical connections between paragraphs); consistency (evaluating the presence of internal contradictions); fluency (measuring the naturalness of the language); and relevance (ensuring that the content closely relates to the experimental theme).

[0138] Finally, the practical utility of the generated experimental protocol was examined from the researchers' perspective through a questionnaire evaluation using the evaluation and verification module. The evaluation and verification module provides a structured, automated evaluation mechanism centered around a questionnaire containing 38 indicators. These indicators were designed with reference to chemists' experimental practices, covering core experimental modules such as reagent specifications, synthesis parameters, and characterization methods. To achieve large-scale evaluation, an automated scoring process was implemented. For each indicator, a standard answer was retrieved from the original scientific literature using an experimental knowledge graph. Then, a large language model for evaluation compared the content of the generated experimental protocol with this standard and scored on a 10-point scale. This evaluation method simulates expert review while ensuring objectivity and scalability. These 38 indicators were categorized into four dimensions: content completeness (17 items), information accuracy (10 items), format conformity (6 items), and operational usability (5 items), enabling a comprehensive and multifaceted performance analysis.

[0139] Evaluation of the experimental setup and subsequent experimental verification of the experimental protocol:

[0140] To systematically evaluate the system in this embodiment, a benchmark dataset containing 125 research papers on Fischer-Tropsch synthesis was established—this complex reaction system is an ideal scenario for testing the generation of automated experimental protocols. The system was deeply adapted to the Fischer-Tropsch synthesis domain by: customizing specific attributes (such as syngas ratio) in a knowledge graph model, refining key sub-sections such as "catalyst preparation" in the domain knowledge outline, and optimizing prompt templates to prioritize the extraction of relevant experimental parameters. The proposed system (integrating a domain knowledge outline, abstract injection, and knowledge graph-based question answering) was compared with four baseline strategies: the basic large language model method (directly summarizing experimental steps), the large language model + outline L1 method (following the first-level outline in the domain knowledge outline), the large language model + outline L2 method (following a second-level detailed outline containing structured prompts), and the blog template method (narrative style). To ensure fair comparison, all methods were implemented using the GPT-4.1-2025-04-14 model.

[0141] The system in this embodiment achieves excellent factual accuracy and information density:

[0142] First, the factual fidelity of the generated experimental protocols was quantified using precision, recall, and F1 score. The results clearly show that the system achieves the optimal balance between accuracy and completeness (see Table 1), obtaining the highest F1 score (94.35%) and the highest effective information recall (91.04%). It is noteworthy that although the large language model + outline L2 extracted a larger total number of entities (15,714), its effective recalled entity count (4,686) was lower than that of this method (6,738). This indicates that relying solely on detailed text prompts, while enabling broad information extraction, introduces a large amount of redundant content and reduces information density. In contrast, the system of this invention, guided by a structured experimental knowledge graph, achieves more precise and goal-oriented information retrieval. The blog template method performed poorly in all objective metrics, confirming its unsuitability for generating rigorous scientific experimental protocols.

[0143] Table 1: Comparison of objective performance of different methods in the experimental scheme generation task:

[0144] .

[0145] This invention improves text quality and logical coherence through knowledge graph fusion:

[0146] In addition to factual accuracy, the linguistic and logical quality of the generated experimental protocol text was evaluated using the G-Eval framework (see Table 2). The system in this embodiment significantly outperformed all baseline models, achieving the highest scores in coherence (8.56), consistency (8.43), and relevance (8.85). This superior performance is attributed to the three-layered prompting strategy constructed in this invention: the domain knowledge outline provides a stable global framework structure, while summary injection ensures the narrative always focuses on the core experimental logic. This combination effectively avoids the step disorder and information redundancy problems common in baseline methods. The system demonstrates strong robustness when handling complex literature involving multiple catalysts, clearly distinguishing and organizing parallel experimental paths. Although the basic large language model method achieved the highest score (8.50) in fluency due to its narrative style, its weaker performance in other key dimensions highlights its limitations in scientific applications.

[0147] Table 2: G-Eval quality assessment of experimental protocol texts generated by different methods (out of 10):

[0148] .

[0149] The practical application value of the generated experimental design was confirmed through questionnaire evaluation.

[0150] To evaluate the practical value of the generated experimental plans from a researcher's perspective, this invention conducted a comprehensive questionnaire survey across four key dimensions (see Figures 7 to 10, which show the comparative evaluation results of the automatic experimental plan generation system across the four dimensions; Figure 7 shows the comparative evaluation results for content completeness, Figure 8 for information accuracy, Figure 9 for format standardization, and Figure 10 for operational usability). The results confirm that the experimental plans generated by the system of this invention are not only highly accurate but also extremely usable.

[0151] In terms of content completeness, the system of this invention significantly outperforms the baseline model, particularly in capturing logical chapter transitions, representing technical details, and performance data (see Figure 7). In all subcategories of information accuracy, this method achieved the highest score, thanks to the dual mechanism of knowledge graph retrieval ensuring parameter consistency and semantic constraints providing logical verification (see Figure 8). The system's embedded structured templates also guarantee excellent formatting, improving the readability and standardization of experimental reproducibility (see Figure 9). These advantages ultimately converge into a significant lead in operational usability (see Figure 10). The steps, usage, and parameters of knowledge graph verification are clearly presented, upgrading the generated experimental protocol from merely readable text into a usable document with high-value guidance for laboratory operations.

[0152] Example 2

[0153] This embodiment provides an automatic experimental scheme generation system based on knowledge graphs and large language models, including: an experimental knowledge graph construction module, a domain knowledge outline generation module, an experimental knowledge graph intelligent agent, an experimental process generation module, and an evaluation and verification module; wherein...

[0154] The process of constructing the experimental knowledge graph by the experimental knowledge graph construction module is as follows:

[0155] The knowledge graph for this experiment was constructed based on 125 research papers on Fischer-Tropsch synthesis (published between 1990 and 2024) obtained from the digital libraries of the American Chemical Society (ACS) and the Royal Society of Chemistry (RSC), and followed a three-stage automated process (see Figure 3).

[0156] Step 21, Document Preprocessing: The original PDF files of scientific literature are first converted into structured Markdown text using the pdf-marker library. To filter out relevant experimental content, a two-layer pruning mechanism is adopted: the first layer uses regular expressions to remove structured sections (such as references and acknowledgments); the second layer uses the GPT-4o mini model to further identify and filter non-procedural content—such as theoretical background, author contributions, and background discussion—thereby retaining only text paragraphs related to the experiment, resulting in a pruned experimental procedure corpus.

[0157] Step 22. Knowledge Extraction: From the pruned experimental procedure corpus, extract "entity-relationship-attribute" triples using the GPT-4o model via the OpenAI API. This process is guided by a detailed schema that specifies: five FTS-specific entity types (reagent, device, operation, condition, product / intermediate); 50 predefined relation types in this embodiment; and 66 node attributes. This allows for the definition of different relation types and node attributes for different experimental types, and the addition or removal of relation types and node attributes as needed for different experimental purposes. Extraction rules enforce consistency in node labels (e.g., using generic labels such as "catalyst" supplemented with specific attributes), numerical data format (using camelCase key-value pairs, such as activationTemperature: 350), dereference resolution (e.g., using consistent terminology for "programmed temperature reduction"), and relation directionality (using active voice interactions, such as "<reagent> performs <operation> in <device>"). Chemical-specific constraints include hierarchical characterization of catalyst components (active phase, promoter, support) and synthesis-specific relationships (e.g., "synthesized from...", "exhibits selectivity for..."). To achieve entity standardization, a specialized vocabulary was constructed to unify chemical nomenclature (e.g., resolving "Co / Al2O3" to its canonical name). The final knowledge graph contains approximately 12,000 nodes and 350,000 edges.

[0158] Experimental protocol generation guided by experimental knowledge graph:

[0159] It integrates the experimental knowledge graph with the large language model and operates under the guidance of the domain knowledge outline generation module and the experimental knowledge graph agent (see Figure 4).

[0160] Domain Knowledge Outline Generation Module: The generated domain knowledge outline is a static, pre-written template by experts (which can be saved as a Markdown file) that defines the hierarchical structure of the FTS experimental scheme. The first-level structure defines the main chapters, while the second-level structure uses structured hints containing five key fields: objectives, requirements, formatting information, examples, and special notes. This template remains consistent across all experiments in this invention system, for both the proposed method and the enhanced expert method.

[0161] Experimental Knowledge Graph Agent: This experimental knowledge graph agent retrieves information using two complementary strategies, both of which utilize the Neo4j graph database for querying and are executed after entity extraction is completed by the large language model. For entity retrieval, a Cypher query (Formula 2) is executed to: (i) search for matching initial nodes using a full-text index, (ii) retrieve associated text fragments via the ":MENTIONS" relation, (iii) extend to adjacent nodes connected by direct relations, and (iv) simultaneously return relation triples (formatted as "head entity-relation-tail entity") and the original text fragments.

[0162] For attribute retrieval (Formula 1), a similar Neo4j-based approach was adopted. The query matches nodes by node labels (using regular expression pattern matching) and retrieves linked text fragments, then expands to adjacent nodes to capture the connected relationships.

[0163] In both strategies, the fusion step (Formula 3 in Example 1) involves summarizing the different results from node matching and relation expansion, and removing duplicates. The TopK parameter is fixed at 25 to control the context length, ensuring that the most relevant graph elements and their corresponding text fragments are retrieved.

[0164] The processing method for the layered generation module of the experimental scheme is as follows:

[0165] The four-step experimental scheme's workflow (see Figure 4) is implemented as a sequential, hierarchical generation and inference process. All generative API calls are set to a temperature parameter of 0 and a Top-p parameter of 1.0 to ensure a balance between output determinism and language fluency.

[0166] Specifically, firstly, subqueries for each chapter are generated according to Formula 4 of Example 1, and then chapter-specific question-answer pairs are generated according to the definition of Formula 5. The retrieved question-answer pairs are then used in the chapter construction stage (Formula 6 of Example 1), and finally, the experimental scheme is assembled through the integration steps defined by Formula 7.

[0167] To balance accuracy and efficiency, multiple models were employed at different stages. The GPT-4.1-2025-04-14 model served as the core generator, responsible for chapter-level synthesis (Formula 6 in Example 1) and final integration (Formula 7 in Example 1). The o3-mini inference model and the 4o-mini lightweight model worked together for knowledge graph retrieval and initial query expansion. Notably, all entity and attribute extraction tasks in the knowledge graph retrieval module were performed by 4o-mini. For each chapter, a maximum of 30 subqueries were generated; for chapters with high conceptual or contextual complexity—such as the "activation" stage in Fischer-Tropsch synthesis (this term may refer to different experimental or explanatory contexts)—o3-mini was used to improve inference quality, while GPT-4.1 was used to generate subqueries and their answers for other chapters.

[0168] To improve generation efficiency, question-and-answer responses are generated in batches of five questions. The final chapter text is generated using Equation 6 under the GPT-4.1 model, and then assembled according to the first-level outline using Equation 7 to output the complete experimental design.

[0169] The evaluation metrics for the experimental protocol are assessed through the evaluation and verification module:

[0170] This invention implements four baseline methods with different prompting strategies to ensure fair comparison: **Large Language Model Approach:** Prompts contain only the source text, along with instructions to directly summarize the experimental procedure. **Large Language Model + Outline L1:** Prompts include the source text and first-level outline headings (e.g., "Experimental Materials"), along with instructions to fill in content for each section. **Large Language Model + Outline L2:** Prompts include the source text and complete second-level structured prompts from a domain knowledge outline, covering objectives, requirements, formatting information, examples, and special notes fields. **Blog Template Approach:** This method, adapted from the Multimodal Academic Interpretation Framework (MMAPIS), restates experimental sections as highly readable narrative text. The model is prompted to "rewrite the experimental portion of the provided text into a clear, understandable, and accessible narrative blog post suitable for general scientific readers," adhering to the hierarchical discourse structure and multimodal alignment principles introduced in the blog template approach.

[0171] (1) Objective assessment:

[0172] The objective evaluation comprises two sub-indicators: accuracy and completeness. In the accuracy evaluation, the generated experimental protocols are first divided by chapter. Taking materials-related chapters as an example, entities are extracted according to a unified three-category model, which covers: (i) reagents, (ii) experimental equipment, and (iii) commonly used laboratory equipment. The extraction model is prompted to "identify all explicitly mentioned material entities and classify them into three predefined categories based on their experimental function." Subsequently, each extracted entity is paired with the corresponding chapter of the generated protocol and relevant paragraphs retrieved from the original paper.

[0173] A standalone GPT-4o-mini instance acts as the evaluator, instructed to determine whether the description of each entity in the proposed solution is consistent with the original paper. The evaluation prompt begins with a role definition—"You are an expert reviewer in the field of experimental reproduction practice. Using the original paper as the sole benchmark, check whether there are any material conflicts in the description of {entity name} in the proposed solution." The rules explicitly state that a conflict should only be identified if the proposed solution explicitly contradicts the original paper. The model then generates structured binary judgments ("consistent" or "inconsistent") with brief reasons. The final accuracy metric is calculated as the percentage of entities marked as consistent.

[0174] In the integrity assessment, a list of target entities is first generated for each chapter based on a predefined experimental outline. Then, the 4o-mini model is used to evaluate whether each entity is "covered" or "missing" in the six generated scenarios. The assessment cues emphasize semantic understanding, allowing the identification of coverage through synonyms, abbreviations, or functionally equivalent terms (e.g., using "HCl" to refer to "hydrochloric acid"), while applying strict contextual rules to distinguish between relevant and irrelevant mentions. Each entity receives a binary coverage judgment under these conditions, which forms the integrity score.

[0175] (2) Text quality assessment:

[0176] The text quality of the generated schemes was evaluated using the G-Eval framework, with GPT-4o-mini as the evaluator. To ensure the stability and reproducibility of the scores and avoid potential randomness in a single evaluation, a multiple sampling strategy was adopted for each evaluation metric. Specifically, the temperature parameter was set to 0.7, and the model was required to generate 20 scores simultaneously for the same evaluation item, with the average score taken as the final score for that item. The evaluation covered four complementary linguistic dimensions: coherence, consistency, fluency, and relevance. Each metric was scored on a scale of 1-10, with higher scores indicating better text performance. Coherence: The evaluator was asked to assess the clarity and logic of the content conveyed in each generated section. The cue defines coherence as "clarity of information and fluency of logic," emphasizing structural rationality and readability. A summary (or generated section) with a score of 10 was described as "exceptionally coherent, with perfect structure and flow," while lower scores indicated obvious logical breaks or abrupt transitions. Consistency: Defined as the degree of factual alignment between the generated text and its source material. The evaluator receives both the source document and the corresponding generated chapters, and is explicitly instructed to "determine whether each chapter is factually consistent with the original text and contains only information supported by the source text." A score of 10 indicates complete factual fidelity, while lower scores reflect varying levels of unsupported or contradictory statements. Fluency: Measures the grammatical correctness and stylistic fluency of the generated language. The evaluator focuses on grammatical, syntax, punctuation, and lexical appropriateness, following a scoring standard: "10" corresponds to natural, error-free writing, while "1" represents text that is difficult to understand due to serious grammatical or stylistic flaws. Relevance: Assesses the degree to which each generated chapter captures the core information of the source text while avoiding redundant or irrelevant content. The suggestion defines relevance as "the degree to which key information is included while avoiding redundant or off-topic information." A perfect score (10) requires comprehensive coverage of important details and concise expression, while lower scores indicate omissions, redundancy, or the inclusion of peripheral material.

[0177] (3) Questionnaire assessment:

[0178] To comprehensively evaluate the generated experimental protocols from the perspective of human expert judgment, this invention designs a structured questionnaire containing 38 questions, entitled "Comprehensive Evaluation Questionnaire for Fischer-Tropsch Synthesis Experiment Reports." This questionnaire was designed by domain experts and organized into four dimensions: (i) completeness of content, (ii) accuracy of information, (iii) format compliance, and (iv) usability and functionality.

[0179] Each dimension encompasses several evaluation categories. (I) The completeness dimension assesses whether the generated protocol includes all key experimental information, including reagent and equipment categories, synthesis procedures, characterization methods, catalytic conditions, and results reporting. (II) The accuracy dimension assesses the factual correctness of the reported data, parameters, and conclusions compared to the original literature. (III) The format dimension checks structural consistency, heading hierarchy, list format, and unit standardization against the experimental protocol template. (IV) The usability dimension measures the reproducibility, logical organization, and user-friendliness of the protocol for experimental reproducibility.

[0180] During the evaluation process, each question is handled independently by a GPT-4o-mini-based scoring agent (referred to as the questionnaire evaluation agent). This agent receives three types of input: (1) the generated scheme section (report content), (2) expert-defined questions (evaluation prompts), and (3) supporting text retrieved from the original paper or domain knowledge graph (paper information). For questions involving fact-checking (such as completeness or accuracy), the evaluation verification module automatically invokes the knowledge graph retrieval mechanism to locate the corresponding evidence in the source literature before scoring. For format or clarity-related questions that do not require external references, the evaluation is based directly on the report content. Each question is scored on a ten-point scale, with higher scores indicating greater compliance with expert standards.

[0181] To ensure the stability and reproducibility of the scoring and avoid the potential randomness of a single evaluation, this invention employs a multiple sampling strategy for each question. Specifically, the temperature parameter is set to 0.7, and the model is required to generate 20 scores simultaneously for the same question. The average of these scores is then taken as the final score for that question. A score of 9-10 indicates that the reported data is completely accurate, comprehensive, logically rigorous, and all key information is clearly presented. A score of 6-8 indicates that the report generally meets the expected standards, but there may be minor differences in expression, slight omissions, or slightly unclear descriptions of non-critical information. A score of 1-5 reflects significant deviations in core data, missing key information, logical contradictions, or major errors in key experimental content.

[0182] For each question, the evaluation and validation module internally generates a structured JSON output containing the question text and numerical score suggestions, although only the numerical results are ultimately aggregated for analysis. The final questionnaire score for each generated experimental protocol is calculated as the average of all 38 metrics across 125 test documents, serving as a comprehensive, expert-standard metric for measuring the overall report quality.

[0183] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0184] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims. The information disclosed in the background section is intended only to enhance the understanding of the overall background technology of the present invention and should not be construed as an admission or implication in any way that such information constitutes prior art known to those skilled in the art.

Claims

1. An automatic experimental scheme generation system based on knowledge graphs and large language models, characterized in that, This system is designed to automatically construct experimental plans based on scientific literature using a large language model. It includes: an experimental knowledge graph construction module, a domain knowledge outline generation module, an experimental knowledge graph agent, and an experimental process generation module. The experimental knowledge graph construction module extracts and structures experimental knowledge from scientific literature based on a large language model and constructs an experimental knowledge graph using this knowledge. The domain knowledge outline generation module generates a hierarchical domain knowledge outline based on a large speech model using predetermined expert templates. The experimental knowledge graph agent communicates with the experimental knowledge graph construction module and performs multi-strategy fusion retrieval on the experimental knowledge graph based on the domain knowledge outline to obtain structured facts corresponding to each sub-chapter of the domain knowledge outline. The experimental knowledge graph agent performs multi-strategy fusion retrieval on the experimental knowledge graph based on the large language model according to the domain knowledge outline to obtain structured facts corresponding to each sub-chapter of the domain knowledge outline. The facts include: the experimental knowledge graph agent takes each sub-chapter of the domain knowledge outline as input, extracts the content of each sub-chapter to obtain a summary, uses the summary of each sub-chapter as input to the large language model, obtains the experimental sub-question set corresponding to the summary content output by the large language model, and then uses each question in the experimental sub-question set as input to the experimental knowledge graph agent, enabling the experimental knowledge graph agent to retrieve the answer corresponding to the input question by performing a structured query on the experimental knowledge graph based on the input question; then, the input question and the corresponding answer are combined into question-answer pairs, and each question-answer pair is combined into a question-answer pair set for each sub-chapter as the structured fact corresponding to each sub-chapter; the experimental knowledge graph agent retrieves the experimental knowledge graph through two structured queries: attribute retrieval and entity retrieval, wherein, attribute retrieval identifies nodes in the experimental knowledge graph whose attributes match the query keywords determined based on the input question, which is formally expressed as: (1); where, It is the result set of attribute matching nodes; v represents a node in the experimental knowledge graph; Represents all nodes in the experimental knowledge graph; Represents a node The set of attributes; It is a set of keywords extracted from the query; Represents the empty set; condition Ensure that a node is included only if its attributes share at least one common keyword with the query; match proper nouns or numerical entities in the query with corresponding nodes in the experimental knowledge graph based on semantic similarity using entity retrieval, as shown below: (2); where, It is the result set of entity matching nodes; It is a node The corresponding entity content; Symbols represent similarity matching; This represents the set of entities in the query; the experimental knowledge graph agent uses text evidence recall to attach corresponding original text fragments from scientific literature to each retrieved node or relation, and fuses, deduplicates, and expands the retrieved information to include adjacent nodes, selecting the K most relevant search results to form the final context, which is defined as: (3); where R represents the final context formed; TopK(·) represents the selection function that selects the K most relevant search results; It is a fusion and expansion function; the final context obtained by the experimental knowledge graph agent is used as the answer, the input question and the corresponding answer are combined to form a question-answer pair, and the question-answer pairs are combined into a question-answer pair set for each sub-chapter as the obtained accurate structured facts; the experimental process generation module is communicatively connected to the domain knowledge outline generation module and the experimental knowledge graph agent, respectively, and can interact with the experimental knowledge graph agent based on the large language model, obtain the structured facts corresponding to each sub-chapter according to the domain knowledge outline and construct the experimental scheme for each sub-chapter, and combine the contents of the experimental schemes of each sub-chapter to generate a complete experimental scheme.

2. The automatic experimental scheme generation system based on knowledge graphs and large language models according to claim 1, characterized in that, Also includes: The evaluation and verification module, connected to the experimental procedure generation module, can perform objective indicator evaluation, text quality evaluation, and practicality evaluation on the experimental plan generated by the experimental procedure generation module.

3. The automatic experimental scheme generation system based on knowledge graphs and large language models according to claim 1 or 2, characterized in that, The experimental knowledge graph construction module extracts and structures experimental knowledge from scientific literature based on a large language model in the following manner, and constructs an experimental knowledge graph using the extracted experimental knowledge, including: Step 11, converting the document format of the scientific literature, preserving the hierarchical structure of the document, and filtering out irrelevant parts of the document content through a two-layer content screening mechanism of rule filtering and keyword and paragraph semantic feature matching to obtain a clean experimental step corpus; Step 12, using a hybrid strategy combining rule-based and large language model assistance to extract entity-relationship-attribute triples from the experimental step corpus, and using the obtained triples to construct the experimental knowledge graph.

4. The automatic experimental scheme generation system based on knowledge graphs and large language models according to claim 3, characterized in that, In step 12, the entities extracted from the experimental step corpus are: predefined entity types that conform to the domain to which the constructed experimental scheme belongs, and the extracted entities serve as nodes in the constructed experimental knowledge graph; the extracted relationships are: a verb or verb phrase that can directly link the nodes corresponding to two entities; the extracted attributes are: information representing the identity, state, behavior, and measurement standards of the nodes corresponding to the entities.

5. The automatic experimental scheme generation system based on knowledge graphs and large language models according to claim 1, characterized in that, The domain knowledge outline generation module generates a hierarchical domain knowledge outline, including a first-level outline and a second-level outline. The first-level outline is an expert template for the content required to establish experimental plans in relevant directions. The second-level outline is the detailed content of each sub-chapter generated based on the structured prompts determined by the experimental objectives, experimental requirements and format instructions, plan examples and special notes of each sub-chapter in the first-level outline, and is generated based on a large language model.

6. The automatic experimental scheme generation system based on knowledge graphs and large language models according to claim 5, characterized in that, The experimental procedure generation module interacts with the experimental knowledge graph agent based on a large language model in the following manner: It obtains structured facts corresponding to each sub-chapter based on the domain knowledge outline and constructs experimental plans for each sub-chapter; it then combines the contents of the experimental plans for each sub-chapter to generate a complete experimental plan. This includes: Step 31, interacting with the experimental knowledge graph agent through the large language model, using the domain knowledge outline as input to the agent, enabling the agent to generate a global summary outlining key compounds, synthetic routes, and main conclusions based on the domain knowledge outline; Step 32, guided by the global summary and structured prompts corresponding to each chapter of the domain knowledge outline, enabling the agent to execute the experimental knowledge graph through the large language model. Step 33: Perform adaptive exploration processing to generate a fine-grained sub-query set to verify the experimental knowledge graph; Step 34: Input the generated fine-grained sub-query set into the experimental knowledge graph agent. The knowledge graph agent uses a multi-strategy retrieval mechanism to locate the structured facts in the experimental knowledge graph as the most relevant answers, and returns the most relevant answers in the form of structured question-answer pairs; Step 35: Build a context-aware prompt for each chapter. Guided by the prompts in each chapter, generate the final text as the experimental plan for each chapter through a large language model; Step 36: After generating the experimental plans for each chapter, aggregate the experimental plans for each chapter in a predefined order to form a complete and logically coherent overall experimental plan.

7. The automatic experimental scheme generation system based on knowledge graphs and large language models according to claim 6, characterized in that, In step 32, the formal representation of generating the fine-grained sub-query set to verify the experimental knowledge graph is as follows: (4); among them, Indicates a specific chapter The set of subqueries; Indicates a specific chapter; Represents a global summary; It is a subquery generation function, based on a specific chapter. Outline template and global summary As input, output this specific chapter. subquery set In step 33, each question-answer pair contains a precise answer and supporting evidence fragments from the original text in the scientific literature. The process of obtaining question-answer pairs based on experimental knowledge graph retrieval is represented as follows: (5); among them, It generates a set of question-answer pairs. The function; in step 34, in constructing a context-aware prompt for each chapter, three key components are integrated: a chapter-specific knowledge outline for structure and format. A global summary used for narrative guidance And all relevant question-and-answer pairs for verified factual details. Single chapter Corresponding experimental scheme The generation definition is: (6); where G LLM (·) indicates that the experimental scheme for that chapter is constructed by combining the large language model with the knowledge outline of that specific chapter, the global summary, and all relevant question-and-answer pairs; i=1,2,……n, where n represents the total number of chapters; in step 35, the process of aggregating the experimental schemes of each chapter in a predefined order is as follows: (7); among which, Represents the overall experimental scheme obtained from the aggregation; symbol Indicates all The structured arrangement of chapter content.

8. The automatic experimental scheme generation system based on knowledge graphs and large language models according to claim 2, characterized in that, The evaluation and verification module performs objective indicator evaluation, text quality evaluation, and practicality evaluation on the experimental plan generated by the experimental process generation module in the following manner, including: the objective indicators for evaluation include: the accuracy of the experimental plan. and the recall rate of the experimental design The accuracy of the experimental design was determined as follows: The experimental design is divided into modules based on the domain knowledge outline, and all entities are extracted from each module. A large language model is used as the evaluator to determine whether the description of each entity is consistent with the source text of the scientific literature, thus determining the accuracy of the experimental design. N is defined as the number of entities accurately described in the experimental design. acc With the total number of entities N protocol The ratio: (8); Determine the recall rate of the experimental protocol as follows. : Through recall rate The effectiveness of the experimental design in covering predefined target information is evaluated. Based on the requirements of the domain knowledge outline, all target entities are identified, and the recall rate of the experimental design is assessed. Defined as the number of target entities that are successfully recalled and accurately described, N re The total number N of target entities defined in the domain knowledge outline template The ratio: (9) The reproducibility of facts is evaluated by the harmonic mean of precision and recall, i.e., the F1 score. The generated experimental design is evaluated by text using the G-Eval framework. The language quality and logical clarity of the experimental design are evaluated by coherence, consistency, fluency and relevance. The content of the generated experimental design is compared with a pre-obtained standard questionnaire containing multiple indicators by a large language model as the evaluator, and scored on a 10-point scale. Each indicator in the standard questionnaire is a standard answer retrieved from the original literature using the experimental knowledge graph.

Images