Gene knowledge corpus transformation method and system based on evidence anchoring and multi-hop reasoning
By employing evidence anchoring and multi-hop reasoning methods, this approach addresses the issues of poor instruction compliance and a single logical hierarchy in existing technologies. It achieves efficient and professional corpus conversion and storage, and the generated question-answer pairs possess cross-dimensional logical reasoning capabilities, thereby reducing parsing anomaly rates and costs.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ZHEJIANG LAB
- Filing Date
- 2026-05-19
- Publication Date
- 2026-07-31
AI Technical Summary
Existing technologies suffer from poor instruction compliance when processing biological academic terms, generate question-and-answer logic with a single level, lack cross-dimensional reasoning, have insufficient engineering robustness, and have a high parsing failure rate, resulting in imprecise model outputs and wasted API tokens.
We adopt an evidence anchoring and multi-hop reasoning approach, construct multi-level evidence anchoring instructions through data feature anchoring and abundance scanning, use a multi-threaded architecture to schedule large language models for corpus conversion, and employ cascaded multi-level fault-tolerant data parsing and segmented hash mutex locks for secure storage.
It significantly improves the professionalism and logical depth of the generated question-answer pairs, reduces the parsing anomaly rate, achieves efficient parallel storage and cost control, ensures that the generated corpus is aligned with the original scientific research facts, and has cross-dimensional logical reasoning capabilities.
Smart Images
Figure CN122224296B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of data processing and artificial intelligence, and in particular to a method for converting structured genetic research data into highly logical fine-tuned corpora using a large language model (LLM). Specifically, it relates to a method and system for converting genetic knowledge corpora based on evidence anchoring and multi-hop reasoning. Background Technology
[0002] In the fields of agricultural genomics and molecular breeding, research data is typically stored in complex JSON formats, encompassing multiple dimensions such as gene function, phenotype, and regulatory pathways. To develop academic AI for these vertical fields, it is necessary to transform this structured knowledge into question-and-answer pairs (QA) that conform to the logic of human natural language.
[0003] However, existing technologies have the following drawbacks:
[0004] 1) Poor adherence to instructions: General LLM is prone to hallucination when dealing with biological academic terms, deviating from the original evidence.
[0005] 2) Single logical level: The generated questions and answers often remain at the level of superficial factual statements and lack multi-hop reasoning.
[0006] 3) Insufficient engineering robustness: When faced with massive amounts of literature data, there is a lack of effective deduplication, breakpoint resume, and concurrent conflict protection mechanisms.
[0007] 4) High parsing failure rate: Inaccurate model return format often leads to parsing errors, resulting in the waste of valuable API tokens. Summary of the Invention
[0008] To address the shortcomings of existing technologies, the present invention aims to provide a method and system for converting gene knowledge corpus based on evidence anchoring and multi-hop reasoning.
[0009] To achieve the above objectives, the technical solution adopted by this invention is as follows: a method for converting gene knowledge corpus based on evidence anchoring and multi-hop reasoning, comprising the following steps:
[0010] Data feature anchoring and abundance scanning: Redundancy filtering and recursive null value removal are performed on the structured gene knowledge file, and the knowledge abundance index of the structured gene knowledge file is calculated based on the weight distribution of non-null fields;
[0011] Multi-level evidence anchoring instruction construction: The depth of the question logic is dynamically matched according to the knowledge abundance index. If the knowledge abundance index exceeds the preset threshold, the multi-hop reasoning protocol is activated and semantic consistency constraint instructions are superimposed to construct system prompt words containing evidence anchoring requirements.
[0012] Concurrent task scheduling and LLM interaction: Based on the system prompts, a multi-threaded architecture is used to schedule large language models for corpus conversion;
[0013] Cascaded multi-level fault-tolerant data parsing: A multi-level modal scanning strategy with progressively increasing leniency is adopted. The non-standard strings returned by the large language model are structurally repaired and extracted through stack balancing matching logic to generate structured corpus pairs.
[0014] Thread-safe persistent storage: The corpus pairs are asynchronously written to the target corpus using file-level segmented hash mutex locks.
[0015] Furthermore, the calculation model for the knowledge abundance index I is as follows:
[0016] ;
[0017] in, The total number of core feature fields is preset. Assign importance weights to fields. For a feature, there exists an indicator function; when Exceeding the preset threshold When this occurs, the multi-hop inference protocol is triggered.
[0018] Furthermore, the multi-hop inference protocol guides the model to establish at least one of the following logical chains through mandatory instructions:
[0019] Breeding chain: linking "variation site - phenotypic effect - breeding value";
[0020] Mechanism chain: linking "experimental method - quantitative change - regulatory mechanism";
[0021] Comparison chain: linking "experimental materials - phenotypic differences - functional conclusions".
[0022] Furthermore, the semantic consistency constraint instructions include: performing range detection constraints on numeric fields and performing logical polarity confirmation constraints on boolean fields.
[0023] Furthermore, the method of using a multi-threaded architecture to schedule large language models for corpus conversion includes: real-time tracking of token consumption and model unit price in API responses, and performing dynamic window degradation when encountering rate limiting, and automatically adjusting the concurrency frequency.
[0024] Furthermore, the multi-level modal scanning strategy, starting from the increasing relaxation level, includes the following steps in sequence:
[0025] Level 1: Standard deserialization;
[0026] Level 2: Regular expression extraction of Markdown code blocks;
[0027] Level 3: Balanced bracket substring extraction based on stack structure;
[0028] Level 4: Heuristic completion of incomplete object closure symbols;
[0029] Level 5: NDJSON line-by-line streaming parsing;
[0030] Level 6: Semantic segmentation based on prefix tags.
[0031] Furthermore, the asynchronous writing of the corpus pairs to the target corpus via file-level segmented hash mutex locks includes: using a segmented hash mutex lock mechanism to map the target file path to an independent lock slot, achieving asynchronous and safe writing in a multi-threaded environment, generating records in JSONL format, wherein the records contain corpus pairs and encapsulate metadata including cost data and model version.
[0032] This invention also provides a gene knowledge corpus conversion system based on evidence anchoring and multi-hop reasoning, comprising:
[0033] The feature scanning module is used to perform redundancy filtering and recursive null value removal on the structured gene knowledge file, and calculate the knowledge abundance index of the structured gene knowledge file based on the weight distribution of non-null fields.
[0034] The dynamic prompt word module is used to dynamically match the depth of the question logic based on the knowledge abundance index. If the knowledge abundance index exceeds a preset threshold, the multi-hop inference protocol is activated and semantic consistency constraint instructions are superimposed to construct system prompt words that include evidence anchoring requirements.
[0035] The interactive control module is used to schedule a large language model for corpus conversion using a multi-threaded architecture based on the system prompts.
[0036] The fault-tolerant parsing module is configured to adopt a multi-level modal scanning strategy with progressively increasing leniency. It uses stack-balanced matching logic to perform structured repair and extraction on non-standard strings returned by the large language model, generating structured corpus pairs.
[0037] The secure write module is used to asynchronously write the corpus pairs to the target corpus using a file-level segmented hash mutex lock.
[0038] The present invention also provides an electronic device, including a memory and a processor, wherein the memory is coupled to the processor; wherein the memory is used to store program data, and the processor is used to execute the program data to implement the method.
[0039] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described thereon.
[0040] The beneficial effects of this invention are:
[0041] 1. Significantly improves the professionalism and authenticity of the corpus: Through quantitative knowledge abundance assessment and evidence anchoring mechanism, it ensures that the generated question-answer pairs are strictly aligned with the original scientific research facts. From the algorithm level, it forces the model to refuse over-inference when there is insufficient evidence, effectively eliminating the illusion problem of LLM in the biological vertical field.
[0042] 2. Enhance the logical depth and reasoning quality of the corpus: The multi-hop reasoning protocol forces the model to associate genotypes, phenotypes and regulatory mechanisms, and the generated corpus has cross-dimensional logical reasoning, which can train high-performance vertical domain models that better understand scientific research logic.
[0043] 3. Extremely high engineering robustness and corpus recovery rate: The cascaded six-level parsing strategy and stack balancing algorithm greatly reduce data loss caused by fluctuations in model output format.
[0044] 4. Superior engineering performance and precise cost control: The segmented hash lock mechanism solves the I / O bottleneck problem under high concurrency and realizes efficient parallel storage of massive corpora; at the same time, the real-time cost indicators provide a quantitative basis for resource planning for large-scale data transformation. Attached Figure Description
[0045] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0046] Figure 1 A schematic diagram of the overall process of a gene knowledge corpus conversion method based on evidence anchoring and multi-hop reasoning provided in an embodiment of the present invention;
[0047] Figure 2 This is a schematic diagram illustrating the path construction logic of the three types of core logic chains in the multi-hop inference protocol of this invention.
[0048] Figure 3 This is a flowchart illustrating the cascaded six-level fault-tolerant data parsing strategy in an embodiment of the present invention.
[0049] Figure 4 This is a structural block diagram of a gene knowledge corpus conversion system based on evidence anchoring and multi-hop reasoning, provided for an embodiment of the present invention. Detailed Implementation
[0050] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0051] It should be noted that, unless otherwise specified, the features in the following embodiments and implementation methods can be combined with each other.
[0052] like Figure 1 As shown, this embodiment of the invention provides a method for converting gene knowledge corpus based on evidence anchoring and multi-hop reasoning, including the following steps:
[0053] S101. Data feature anchoring and abundance scanning: Redundancy filtering and recursive null value removal are performed on the structured gene knowledge file, and the knowledge abundance index of the structured gene knowledge file is calculated based on the weight distribution of non-null fields.
[0054] S102, Construction of multi-level evidence anchoring instructions: Dynamically match the depth of question logic according to the knowledge abundance index. If the knowledge abundance index exceeds the preset threshold, activate the multi-hop reasoning protocol and superimpose semantic consistency constraint instructions to construct system prompt words containing evidence anchoring requirements.
[0055] S103. Concurrent task scheduling and LLM interaction: Based on the system prompt words, a multi-threaded architecture is used to schedule the large language model for corpus conversion;
[0056] S104, Cascaded Multi-Level Fault-Tolerant Data Parsing: Employs a multi-level modal scanning strategy with progressively increasing leniency, and uses stack-balanced matching logic to perform structured repair and extraction on non-standard strings returned by the large language model, generating structured corpus pairs;
[0057] S105. Thread-safe persistent storage: The corpus pairs are asynchronously written to the target corpus using a file-level segmented hash mutex lock.
[0058] Preferably, step S101 specifically involves: first, acquiring heterogeneous structured gene knowledge files (usually in JSON format). In this step, redundant filtering of literature records is performed using DOI unique identifiers to avoid duplicate corpus generation. Subsequently, recursive null value removal is performed on the data to ensure that subsequent processing only targets valid information. The core feature lies in calculating the knowledge abundance index. :
[0059] ;
[0060] in, The preset key fields are assigned higher weights (such as gene name, mutation site, regulatory mechanism, etc.). Indicates whether the field is not empty. (By...) The magnitude of the value allows the system to quantitatively assess the strength of the current data in supporting reasoning.
[0061] Preferably, step S102 specifically involves: based on the knowledge abundance... Dynamically match the logical depth of the question. If Exceeding the preset threshold If this is not the case, the multi-hop inference protocol is activated. At the same time, the system generates a system prompt containing "role definition (agricultural gene curator)," "boundary constraints (prohibition of fabrication)," and "semantic consistency protocol" to ensure that the Large Language Model (LLM) does not change the order of magnitude or logical polarity of the values during the transformation process.
[0062] Preferably, step S103 specifically involves: initiating concurrent requests using a multi-threaded architecture (such as ThreadPoolExecutor). The system tracks the token consumption and model unit price in the API response in real time, and performs dynamic window degradation when encountering rate limits, automatically adjusting the concurrency frequency to ensure task continuity.
[0063] Preferably, step S104 specifically involves: due to the uncertainty of LLM output, the system extracts QA pairs using a six-level parsing strategy, from strictest to broadest. The core lies in employing a stack-balanced matching algorithm for broken strings (see details). Figure 3 (Note: This ensures that even when the model output is truncated, effective knowledge blocks can be accurately salvaged.)
[0064] Preferably, step S105 specifically involves employing a segmented hash mutex lock mechanism. The system maps the target file path to an independent lock slot, enabling asynchronous and safe writing in a multi-threaded environment. The final generated JSONL record not only contains question-answer pairs but also encapsulates metadata including cost data and model version.
[0065] As a preferred embodiment, such as Figure 2 As shown, in the process of generating gene knowledge question-and-answer pairs, this embodiment details the specific execution process of the "multi-hop reasoning path" to ensure that the generated corpus possesses deep academic reasoning capabilities. The system does not simply extract information from a single field, but instead forces the large language model to establish a cross-field logical mapping through the Prompt command, including:
[0066] Path A (Breeding Chain): Forced association of "variant sites (e.g., SNP_652)" – "phenotypic effects (e.g., increased grain weight)" – "application value (e.g., improvement of high-yield varieties)". This path-guided model links the underlying genotypic variation with the final production practice.
[0067] Example of generation:
[0068] Question: How does the SNP at position 652 of the OsGhd8 gene influence rice yield and its practical breeding?
[0069] Answer: The SNP_652 in OsGhd8 (Os08g07750) leads to a 15% increase ingrain weight. This variant can be utilized as a molecular marker to improve high-yield rice varieties in temperate regions.
[0070] Path B (Mechanism Chain): Forced association of "experimental methods (e.g., Y2H, Co-IP)" – "quantitative changes (e.g., significant expression changes)" – "regulatory mechanisms (e.g., enhanced positive regulation)". This path requires the model to demonstrate a chain of evidence in its responses, i.e., what significant changes were observed through what experiments, and what molecular regulatory logic is derived from them, ensuring that the generated questions and answers have a deep scientific explanation.
[0071] Example of generation:
[0072] Question: What experimental evidence supports the regulatory interaction between Protein A and Protein B under drought stress?
[0073] Answer: Through Y2H and Co-IP assays, it was observed that Protein Adirectly interacts with Protein B. This interaction significantly enhances the expression of drought-responsive genes (P < 0.01), thereby establishing a positive regulatory mechanism for drought tolerance.
[0074] Path C (Comparison Chain): Forced association of "experimental materials (e.g., wild type vs. metamorphic)" – "phenotypic differences (e.g., delayed flowering)" – "functional conclusions". This path extracts the core functional conclusions of genes by comparing the performance of materials from different experimental groups.
[0075] Example of generation:
[0076] Question: What is the functional role of the ATP1 gene based on the phenotypic comparison of experimental materials?
[0077] Answer: Compared to the wild-type (Col-0), the mutant atp1 exhibited a significant flowering time delay under long-day conditions. This indicates that ATP1 functions as a key floral activator in the photoperiodic pathway.
[0078] As a preferred embodiment, such as Figure 3 As shown, to address the issue of non-standard model output formats, the fault-tolerant parser executes parsing in the order of L1 to L6:
[0079] L1 standard deserialization: directly uses the standard JSON parser for processing.
[0080] L2Markdown code block regular expression extraction: for models that typically wrap content in JSON.
[0081] L3 stack-based balanced bracket substring extraction: For cases where explanatory text is mixed in the output, the longest valid closed subset with a stack depth of 0 is found by depth-first traversal.
[0082] L4 incomplete object closure heuristic completion: If the model output is forcibly truncated (output length exceeds the limit), the system automatically fills in the missing} or} according to the current stack state.
[0083] L5NDJSON line-by-line streaming parsing: handles situations where the model returns multiple JSON objects at once.
[0084] L6 semantic segmentation based on prefix labels: As a fallback solution, it uses the "Q:" and "A:" identifiers for plain text segmentation.
[0085] like Figure 4 As shown, this embodiment of the invention also provides a gene knowledge corpus conversion system based on evidence anchoring and multi-hop reasoning, including:
[0086] The feature scanning module is used to perform redundancy filtering and recursive null value removal on the structured gene knowledge file, and calculate the knowledge abundance index of the structured gene knowledge file based on the weight distribution of non-null fields.
[0087] The dynamic prompt word engine is used to dynamically match the depth of the question logic based on the knowledge abundance index. If the knowledge abundance index exceeds a preset threshold, a multi-hop inference protocol is activated, and semantic consistency constraint instructions are superimposed to construct system prompt words that include evidence anchoring requirements.
[0088] The interactive control center is used to schedule a large language model for corpus conversion using a multi-threaded architecture based on the system prompts.
[0089] The fault-tolerant parser is configured to use a multi-level modal scanning strategy with progressively increasing leniency. It performs structured repair and extraction on non-standard strings returned by the large language model through stack-balanced matching logic, generating structured corpus pairs.
[0090] The secure write module is used to asynchronously write the corpus pairs to the target corpus using a file-level segmented hash mutex lock.
[0091] This invention also provides an electronic device, including a memory and a processor, wherein the memory is coupled to the processor; wherein the memory is used to store program data, and the processor is used to execute the program data to implement the method described in any embodiment.
[0092] This invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in any of the embodiments.
[0093] The computer-readable storage medium can be an internal storage unit of any data processing device described in any of the foregoing embodiments, such as a hard disk or memory. The computer-readable storage medium can also be any data processing device, such as a plug-in hard disk, smart media card (SMC), SD card, flash card, etc., equipped on the device. Furthermore, the computer-readable storage medium can include both internal storage units of any data processing device and external storage devices. The computer-readable storage medium is used to store the computer program and other programs and data required by the data processing device, and can also be used to temporarily store data that has been output or will be output.
[0094] The various embodiments in this invention are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.
[0095] The above description is merely an embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of the present invention should be included within the scope of the claims of the present invention.
Claims
1. A gene knowledge corpus transformation method based on evidence anchoring and multi-hop reasoning, characterized in that, Includes the following steps: Redundancy filtering and recursive null value removal are performed on the structured gene knowledge file, and the knowledge abundance index of the structured gene knowledge file is calculated based on the weight distribution of non-null fields. The depth of the question logic is dynamically matched based on the knowledge abundance index. If the knowledge abundance index exceeds a preset threshold, a multi-hop reasoning protocol is activated, and semantic consistency constraint instructions are superimposed to construct system prompt words that include evidence anchoring requirements. Based on the system prompts, a multi-threaded architecture is used to schedule a large language model for corpus conversion. A multi-level modal scanning strategy with progressively increasing leniency is adopted. The non-standard strings returned by the large language model are structurally repaired and extracted through stack balancing matching logic to generate structured corpus pairs. The corpus pairs are asynchronously written to the target corpus using a file-level segmented hash mutex lock.
2. The method of claim 1, wherein, The calculation model for the knowledge abundance index I is as follows: ； in, The total number of core feature fields is preset. Assign importance weights to fields. For a feature, there exists an indicator function; when Exceeding the preset threshold When this occurs, the multi-hop inference protocol is triggered.
3. The method according to claim 1, characterized in that, The multi-hop inference protocol guides the model to establish at least one of the following logical chains through mandatory instructions: Breeding chain: linking "variation site - phenotypic effect - breeding value"; Mechanism chain: linking "experimental method - quantitative change - regulatory mechanism"; Comparison chain: linking "experimental materials - phenotypic differences - functional conclusions".
4. The method of claim 1, wherein, The semantic consistency constraint instructions include: performing range detection constraints on numeric fields and performing logical polarity confirmation constraints on boolean fields.
5. The method of claim 1, wherein, The method of using a multi-threaded architecture to schedule large language models for corpus conversion includes: real-time tracking of token consumption and model unit price in API responses, and dynamic window degradation when encountering rate limiting, and automatic adjustment of concurrency frequency.
6. The method of claim 1, wherein, The multi-level modal scanning strategy, in ascending order of relaxation, includes the following: Level 1: Standard deserialization; Level 2: Regular expression extraction of Markdown code blocks; Level 3: Balanced bracket substring extraction based on stack structure; Level 4: Heuristic completion of incomplete object closure symbols; Level 5: NDJSON line-by-line streaming parsing; Level 6: Semantic segmentation based on prefix tags.
7. The method of claim 1, wherein, The method of asynchronously writing the corpus pairs to the target corpus using file-level segmented hash mutex locks includes: using a segmented hash mutex lock mechanism to map the target file path to an independent lock slot to achieve asynchronous and safe writing in a multi-threaded environment, generating a JSONL format record, which contains corpus pairs and encapsulates metadata including cost data and model version.
8. A gene knowledge corpus transformation system based on evidence anchoring and multi-hop reasoning, characterized in that, include: The feature scanning module is used to perform redundancy filtering and recursive null value removal on the structured gene knowledge file, and calculate the knowledge abundance index of the structured gene knowledge file based on the weight distribution of non-null fields. The dynamic prompt word module is used to dynamically match the depth of the question logic based on the knowledge abundance index. If the knowledge abundance index exceeds a preset threshold, the multi-hop inference protocol is activated and semantic consistency constraint instructions are superimposed to construct system prompt words that include evidence anchoring requirements. The interactive control module is used to schedule a large language model for corpus conversion using a multi-threaded architecture based on the system prompts. The fault-tolerant parsing module is configured to adopt a multi-level modal scanning strategy with progressively increasing leniency. It uses stack-balanced matching logic to perform structured repair and extraction on non-standard strings returned by the large language model, generating structured corpus pairs. The secure write module is used to asynchronously write the corpus pairs to the target corpus using a file-level segmented hash mutex lock.
9. An electronic device comprising a memory and a processor, characterized in that The memory is coupled to the processor; wherein the memory is used to store program data, and the processor is used to execute the program data to implement the method as described in any one of claims 1-7.
10. A computer-readable storage medium having stored thereon a computer program, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-7.