An augmented question-answer pair retrieval augmented generation method, system, device and medium
By constructing an augmented question-answer pair retrieval enhancement generation method, and utilizing hypergraph structure modeling and question-answer pair reverse verification mechanism, the illusion problem of LLM in multi-entity interaction scenarios in high-risk domains is solved, achieving efficient and accurate knowledge supplementation and generation, and improving the response accuracy and reliability of LLM.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- INFORMATION & COMM CO OF STATE GRID SHAANXI ELECTRIC POWER CO LTD
- Filing Date
- 2026-04-27
- Publication Date
- 2026-07-21
AI Technical Summary
Existing Large Language Models (LLMs) suffer from the illusion problem in high-risk domains, generating content that deviates from objective facts. In particular, they struggle to provide accurate high-order knowledge constraints in multi-entity interaction scenarios, leading to information distortion and content fabrication. Existing RAG technologies also struggle to balance accuracy and performance under complex queries.
An augmented question-answer pair retrieval enhancement method is constructed. By acquiring a structured corpus, entity sets and association sets are extracted. Combined with a large language model (LLM), entity association retrieval and question-answer pair matching are performed. Multi-entity collaborative knowledge is modeled using a hypergraph structure, and a question-answer pair reverse verification mechanism is introduced to generate and correct responses.
It significantly reduces the illusion rate of LLM, improves the accuracy and reliability of the response, ensures that the generation process has real high-order factual basis, solves the problem that existing RAG technology cannot balance accuracy and performance under complex queries, and improves the reliable application of LLM in high-risk fields.
Smart Images
Figure CN122432291A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of large model retrieval enhancement generation and illusion relief technology, and particularly to an augmented question-answering pair retrieval enhancement generation method, system, device and medium. Background Technology
[0002] Large Language Models (LLMs), with their superior natural language understanding and generation capabilities, have sparked a technological revolution globally, profoundly transforming operational models in key sectors such as education, finance, healthcare, and law. In education, LLMs support personalized learning path planning and intelligent Q&A, significantly improving learning efficiency. In the financial industry, they facilitate risk prediction, quantitative analysis, and customer service automation, reducing operating costs. In healthcare, LLMs are being explored for medical record analysis, preliminary diagnostic suggestions, and medical knowledge dissemination, potentially enabling the distribution of medical resources to lower levels of healthcare facilities. In legal practice, they can assist in legal retrieval, contract review, and case analysis, enhancing the professionalism and efficiency of legal work. However, despite the enormous application potential of LLMs, their large-scale implementation in high-risk areas consistently faces serious obstacles. The core issue lies in their pervasive "illusion" problem—the generated content deviates from objective facts, sometimes even completely fabricating non-existent information.
[0003] The dangers of LLM hallucinations are particularly pronounced in high-risk areas. In medical diagnosis, LLM may incorrectly attribute rare disease symptoms to common diseases, or fabricate drug indications and side effects, misleading medical professionals and directly threatening patients' lives and health. In legal matters, it may cite non-existent legal provisions or distort case judgments, rendering legal opinions invalid and leading to serious legal disputes and economic losses. In the financial sector, erroneous market predictions and unethical investment advice can cause users huge financial losses. These risks stem from the nature of LLM training: the model compresses and reconstructs massive amounts of data through self-supervised learning. Although it can accurately reproduce about 98% of the training data, the remaining 2% of information bias may be amplified during the generation process, forming factual errors. Furthermore, the domain limitations of the training data (such as the scarcity and uneven distribution of data in specialized fields), context-dependent bias (information forgetting during long text processing), and the model's insufficient modeling of complex relationships further exacerbate the hallucination problem. Existing strategies for alleviating hallucinations mostly focus on model structure optimization or training data augmentation. However, these methods not only require huge amounts of computing resources, but also cannot completely solve the problem of knowledge loss caused by training data compression, making it difficult to meet the stringent requirements for factual accuracy in high-risk domains.
[0004] To overcome this bottleneck, Retrieval-Enhanced Generation (RAG) technology emerged, becoming a core technological path connecting LLM with external expertise and suppressing illusions. The core logic of RAG is "retrieval-enhanced-generation": by constructing a domain-specific knowledge base, relevant factual information is retrieved and queried before generating a response, supplementing the LLM input with this context, constraining the model generation process, and ensuring the factual basis of the output content. After years of development, RAG technology has undergone multiple iterations from basic to advanced levels.
[0005] Early basic RAG technology employed a simple architecture of "text block segmentation + vector retrieval." Domain documents were segmented into independent text blocks, which were then converted into vectors using an embedding model and stored in a database. During a query, the user's question was vectorized and matched with the text block vectors for similarity, returning the most relevant text fragments as supplementary knowledge. The advantage of this approach is its simplicity and low deployment cost, but it has significant drawbacks: the text block segmentation method disrupts the inherent connections between knowledge points, the retrieval process relies solely on surface semantic similarity, lacking the capture of structured relationships within the knowledge base. This results in insufficient relevance and excessive redundant information in the retrieval results, making it difficult to accurately respond to complex queries.
[0006] To address the issue of missing structured associations, graph-based RAG techniques (such as Graph RAG and Light RAG) have emerged. Graph RAG extracts entities and binary relations from the corpus to construct a knowledge graph, using the graph structure to model pairwise associations between entities, thus improving retrieval accuracy. During the retrieval phase, it not only matches text semantics but also disseminates relevant knowledge through the graph's topological relationships, enhancing information completeness. Light RAG further optimizes efficiency by employing a "local-global" two-layer knowledge graph architecture, simplifying the clustering process of Graph RAG and balancing performance and computational cost to some extent. Compared to basic RAG, these graph-based methods offer significant improvements in complex query processing, but they still have insurmountable technical limitations:
[0007] Much of the critical knowledge is not simply "A acts on B," but rather the result of the combined action of multiple entities—for example, the contraindications of combined use of multiple drugs, or the complex co-occurrence patterns of disease-symptom-complication. Once this type of high-order knowledge is forcibly broken down into independent binary relations for storage, the collaborative semantics between entities will be lost. When user queries involve multi-entity combination logic, the retrieval system can only return fragmented independent facts and cannot provide key constraint information on multi-entity interactions. When LLM performs reasoning and generation under such knowledge gaps, it will inevitably lead to factual distortion or content fabrication, i.e., a serious "illusion" problem, which severely restricts its reliable application in high-risk domains. Summary of the Invention
[0008] The purpose of this invention is to address the shortcomings of the prior art by providing an augmented question-answer pair retrieval enhancement generation method, system, device, and medium to solve the problems in the prior art.
[0009] The present invention specifically provides the following technical solution: An augmented question-answering pair retrieval enhancement generation method includes: A structured corpus of question-answer pairs in the medical field is obtained, and entity sets and corresponding descriptive information are extracted from the structured corpus, as well as the association sets of paired entities and multiple entities. The domain knowledge base is obtained by combining entity-association-question-answer pairs. The entities include diseases, drugs, and symptoms, and the associations represent the interaction relationships between entities. After receiving a user query, the system extracts entity keywords and related keywords from the query. It then matches similar question-and-answer pairs from the domain knowledge base based on these keywords. The system retrieves information from the entity set, association set, and similar question-and-answer pairs through entity association retrieval, related entity retrieval, and question-and-answer pair matching retrieval, respectively. Taking into account the context length limitation of the Large Language Model (LLM), the system selects the optimal supplementary knowledge set from the retrieved information based on different weighted filtering dimensions. These dimensions include semantic similarity to the query, knowledge importance, information redundancy, and domain relevance. A query-supplementary knowledge-formatting requirement prompt template is constructed. The original query and the optimal supplementary knowledge set are merged through the prompt template. The merged prompt template is input into the LLM to generate the final response. The final response is then corrected using a question-and-answer pair reverse verification mechanism to obtain the corrected generated result.
[0010] Preferably, the acquisition of a structured corpus of question-answer pairs in the medical field specifically includes: We collect various types of corpora in the medical field and preprocess them to obtain preprocessed corpora. A dual-blocking strategy combining fixed-length blocks and semantic blocks is adopted to block the preprocessed corpus, generate basic question-answer pairs, and then augment the basic question-answer pairs in multiple dimensions to obtain augmented question-answer pairs. A question-answer pair resource library is constructed based on the aforementioned basic question-answer pairs and augmented question-answer pairs, serving as the final structured corpus.
[0011] Preferably, the step of extracting entity sets and corresponding descriptive information from the structured corpus, extracting the association sets of paired entities and multiple entities, and combining entity-association-question-answer pairs to obtain the domain knowledge base specifically involves: Define a hypergraph structure that contains a set of vertices and a set of associations of entities; Extract entities and their corresponding descriptive information from the structured corpus, design differentiated prompt templates, and extract the associations of paired entities and multiple entities. The system indexes entities, pairwise entity associations, multi-entity associations, and question-answer pairs to form a vector database. A list field of associated question-answer pair IDs is added to each entity and association to obtain a hypergraph database. Through collaboration between LLM and domain experts, the system verifies the consistency between augmented question-answer pairs and the knowledge in the hypergraph structure, and corrects the augmented question-answer pairs to form a question-answer pair index layer. A storage architecture is constructed using the aforementioned vector database, hypergraph database, and question-answering pair index layer, serving as a domain knowledge base.
[0012] Preferably, the step of extracting entities and their corresponding descriptive information from the structured corpus, designing differentiated prompt templates, and extracting pairwise entity associations and multi-entity associations specifically involves: Obtain the entity extraction template, drive the LLM to extract entities and description information from each corpus block, and when multiple corpus blocks contain the same entity, merge and deduplicate the description information through the LLM to obtain the final description information. Design differentiated prompt templates, identify the relationship types and descriptions between pairs of entities using LLM, and output the association of pairs of entities in the format of entity1-relationship-entity2-description. Identify the interaction logic of three or more entities using LLM, clarify the role and function of each entity in the association, and output the association of multiple entities in the format of entity1-entity2-...-entityN-relationship-description.
[0013] Preferably, the step of verifying the consistency between augmented question-answer pairs and the knowledge in the hypergraph structure through collaboration between LLM and domain experts, and correcting the augmented question-answer pairs as a question-answer pair index layer, specifically involves: For each augmented question-answer pair, the system automatically matches the corresponding entity set and association set in the Hypergraph database to verify whether the answer is consistent with the entity description and association information in the Hypergraph. If there are differences in expression but the core facts are consistent, the augmented question-and-answer pair answer will be corrected; if there are factual errors, it will be moved to a dedicated database for adversarial question-and-answer pairs, used only for robustness testing and not for regular retrieval. Each augmented question-answer pair after correction is bound to multiple entities and multiple associations in the hypergraph, serving as a question-answer pair index layer.
[0014] Preferably, after receiving a user query, the entity keywords and related keywords in the user query are extracted, specifically as follows: Based on a dedicated prompt template, the user's query input into the LLM is used to obtain entity keywords and related keywords; The user query is vectorized and matched with the question-answer pairs in the vector database. The top-K most similar question-answer pairs are selected, and the entity key set and association key set associated with the top-K most similar question-answer pairs are extracted as supplementary anchors for keyword retrieval.
[0015] Preferably, the query-supplementary knowledge-format requirements prompt template includes three parts: query restatement, knowledge reference, and format requirements. The knowledge reference is presented in the order of entity information-related information-original text.
[0016] This invention provides an augmented question-answering pair retrieval enhancement generation system, comprising: The corpus processing module is used to acquire a structured corpus of question-answer pairs in the medical field, extract entity sets and corresponding descriptive information from the structured corpus, and extract the association sets of paired entities and multiple entities. The domain knowledge base is obtained by combining entity-association-question-answer pairs. The entities include diseases, drugs, and symptoms, and the associations represent the interaction relationships between entities. The knowledge retrieval module receives user queries, extracts entity keywords and related keywords from the query, matches similar question-and-answer pairs in the domain knowledge base based on the entity keywords and related keywords, and retrieves search information from the entity set, association set, and similar question-and-answer pairs through entity association retrieval, related entity retrieval, and question-and-answer pair matching retrieval, respectively. Taking into account the context length limitation of the Large Language Model (LLM), the module selects the optimal supplementary knowledge set from the search information according to different weighted filtering dimensions. These filtering dimensions include semantic similarity to the query, knowledge importance, information redundancy, and domain relevance. The generation optimization module is used to construct a prompt template for query-supplementary knowledge-format requirements. The original query and the optimal supplementary knowledge set are merged through the prompt template. The merged prompt template is input into the LLM to generate the final response. The final response is then corrected using a question-and-answer pair reverse verification mechanism to obtain the corrected generation result.
[0017] The present invention provides a computer device, including a memory and a processor. The memory stores a program, and when the program is executed by the processor, the processor performs the steps of the above-described augmented question-answer pair retrieval enhancement generation method.
[0018] The present invention provides a storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the above-described augmented question-answer pair retrieval enhancement generation method.
[0019] Compared with the prior art, the present invention has the following significant advantages: This invention constructs a complete technical solution from knowledge modeling to generation optimization. It uses multi-entity collaborative knowledge such as combined medication and complication combinations as native searchable units. When a user query involves multi-entity interactions, the system can directly supplement complete collaborative constraint information, enabling the LLM generation process to have real high-order factual basis. This eliminates information distortion and content fabrication caused by the fragmentation of binary relations. At the same time, based on the above-mentioned optimized LLM generation process, native storage replaces indirect clustering, achieving efficient retrieval while ensuring full coverage of high-order associations. This solves the core contradiction of existing RAG technology in balancing accuracy and performance under complex queries, significantly reducing the illusion rate of LLM and improving the accuracy and reliability of the response. Attached Figure Description
[0020] Figure 1 This is a schematic diagram of the overall architecture of the present invention; Figure 2 This invention provides a diagram illustrating the construction of a multi-domain structured corpus. Figure 3 This invention relates to the construction of a hypergraph-driven domain knowledge base graph; Figure 4 This invention relates to a knowledge retrieval and diffusion graph based on a hypergraph. Figure 5 The flowchart illustrates an augmented question-answer pair retrieval enhancement generation method provided by this invention. Detailed Implementation
[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.
[0022] This invention, based on hypergraph modeling multi-entity association and question-answer pair reverse verification mechanism, improves the accuracy of question-answer pair augmentation and LLM response through structured corpus construction, multi-dimensional retrieval and supplementation, enhanced prompt generation, and result correction. Figure 1 and Figure 5 As shown, the augmented question-answering pair retrieval enhancement generation method provided in this embodiment includes the following steps: S1: Obtain a structured corpus of question-answer pairs in multiple domains (in one embodiment, the medical domain), extract entity sets and corresponding descriptive information from the structured corpus, and extract the association sets of paired entities and multiple entities respectively. Combine the entity-association-question-answer pairs to obtain the domain knowledge base; where entities include diseases, drugs, and symptoms, and associations represent the interaction relationships between entities.
[0023] Corpus Collection and Screening: High-quality corpora from multiple professional fields (in one embodiment, the medical field) are collected. The sources of the corpora include peer-reviewed academic literature, authoritative clinical guidelines, industry standard textbooks, regulatory documents, expert consensus, etc., to ensure the professionalism and factual accuracy of the content. The screening criteria are: removing duplicate content, low-quality literature (citations below the industry threshold) and unstructured and messy information, and retaining logically coherent text with high knowledge density. Finally, nine standardized corpora are formed, with a total of more than 230 million tokens, covering the core knowledge of various fields.
[0024] Corpus preprocessing: A three-step strategy of "cleaning-standardization-noise reduction" was adopted to process the data. First, non-text elements such as images, formulas, special symbols, headers and footers were removed using regular expressions and visual recognition technology, retaining only the plain text content. Second, the text format was standardized (encoded to UTF-8, standardized punctuation, and corrected typos and grammatical errors) to ensure data consistency. Finally, redundant information (such as repeated paragraphs and domain-irrelevant general expressions) was removed through keyword frequency analysis and semantic similarity detection to improve the quality of the corpus, resulting in the preprocessed corpus.
[0025] Dual-blocking strategy design: A dual-blocking strategy combining fixed-length blocks and semantic blocks is adopted to block the preprocessed corpus, generating basic question-answer pairs. These basic question-answer pairs are then augmented in multiple dimensions to obtain augmented question-answer pairs. Specifically:
[0026] First, fixed-length chunking divides the text into standard 1200-token chunks, with a 100-token overlap between adjacent chunks. This design ensures the independence of each chunk while avoiding knowledge breaks across chunks, thus ensuring contextual coherence and is suitable for building large-scale global knowledge bases. Second, semantic chunking, assisted by manual processing and LLM, divides the text into semantic units (average length 319 tokens) based on semantic coherence. Each chunk corresponds to a complete knowledge topic (such as a single scientific conclusion, clinical case analysis, or legal interpretation), which is suitable for mechanism verification experiments and ensures the accuracy of knowledge extraction.
[0027] Based on the processed bi-block corpus, basic question-answer pairs (BaseQA) are generated collaboratively by GPT-4o and domain experts. The rules are as follows:
[0028] (1) Fixed-length chunked corpus: 80 basic question-answer pairs are generated for each dataset, covering three core scenarios: "fact query, logical reasoning, and application scenarios". (2) Semantic chunked corpus: 2 basic question-answer pairs are generated for each semantic chunk, using "concise expression" and "detailed expression" respectively to ensure a strong binding between the question-answer pairs and the knowledge topic. All basic question-answer pairs are labeled with "standard answer source + associated entity set + association type (low-order / high-order)" to provide anchors for subsequent augmentation and retrieval.
[0029] We designed a "three-in-one" augmentation strategy to generate augmented question-answer pairs (AQAs), achieving a dual improvement in both the quantity and quality of AQAs: (1) Variation augmentation: Based on the basic question-answer pair, LLM is used to perform operations such as "synonym replacement, sentence reconstruction, word order adjustment, omission / supplementation of modifiers", generating 3 variant question-answer pairs for each basic question-answer pair.
[0030] (2) Cross-domain fusion augmentation: For the Mix Corpus, generate “cross-domain related question-answer pairs”, with 3 fusion question-answer pairs generated for each Mix Corpus block.
[0031] (3) Adversarial augmentation: Generate “adversarial QA” to simulate common hallucination scenarios in LLM. 20 adversarial QA pairs are generated for each domain dataset for subsequent hallucination suppression verification and retrieval robustness training.
[0032] Q&A on quality control and grading: Quality screening: Through “LLM semantic consistency detection + manual review (sampling)”, question-answer pairs with semantic repetition, factual errors and vague expressions are eliminated, and valid question-answer pairs with an accuracy of ≥90% are retained. (2) Grade labeling: The valid question-answer pairs are labeled with “difficulty level (low / medium / high)” and “domain relevance (1-5 points)”.
[0033] Final Results: A total of over 120,000 question-answer pairs (basic and augmented) were generated, with augmented pairs accounting for ≥60%. These basic and augmented pairs were used to construct a multi-domain, multi-scenario, and multi-difficulty question-answer pair resource library (QALibrary), which served as the final structured corpus. The specific construction process is as follows: Figure 2 As shown.
[0034] Hypergraph structure definition: Defines a normalized hypergraph structure containing a set of vertices and a set of associations of entities. This achieves modeling of the connections between knowledge across all dimensions from a mathematical perspective. Among these, V It is a set of entity vertices, containing all core entities in the domain (such as diseases, drugs, and symptoms in the medical field, and legal provisions, cases, and subjects in the legal field). E low A low-order association set is defined as a binary relation between paired entities (such as "drug A-treatment-disease B" or "legal provision X-constraint-subject Y"). This is a high-order association set, defined as a complex interaction relationship between three or more entities (e.g., "Drug A - Drug B - Combined Use - Treatment - Disease C" or "Legal Provision X - Case Y - Subject Z - Judgment Result W"). This structure breaks through the limitations of binary relationships in traditional knowledge graphs, enabling direct modeling of multi-entity collaborative interactions. The hypergraph-driven domain knowledge base includes, for example... Figure 3 As shown.
[0035] Entity Extraction and Standardization: Based on a dedicated entity extraction template for prompt-based engineering design P ext_entity Entity extraction template-driven LLM extracts data from each block of a structured corpus. D i Extracting entity and structured descriptive information, expressed by the formula is as follows: During the extraction process, the entity type (such as "disease", "drug", "legal provision") and description specifications (must include core attributes, such as the indications for drugs and the effective date of legal provisions) are clearly defined to ensure the completeness of entity information. When multiple corpus blocks contain the same entity, description merging and deduplication are performed using LLM to remove contradictory information and supplement missing attributes, forming a unified and authoritative entity description as the final description information, thus avoiding retrieval bias caused by inconsistent entity information.
[0036] Low-order and high-order association extraction: Design differentiated prompt templates to extract (low-order) associations between paired entities and (high-order) associations between multiple entities. For low-order association extraction, the template... Specific requirements: Identify the relationship type (e.g., "treatment," "constraint," "causation") and specific description between paired entities using LLM, outputting the associations of paired entities in the format "Entity 1-Relationship-Entity 2-Description"; for higher-order association extraction, a template... Guidelines: Using LLM (Local Level Modeling), identify the interaction logic of three or more entities, clarifying the role and function of each entity in the association. Output the associations of multiple entities in the format "Entity 1 - Entity 2 - ... - Entity N - Relationship - Description". After extraction, semantic similarity detection and manual review are used to eliminate false associations (such as causal relationships without factual basis) and duplicate associations, ensuring the authenticity and uniqueness of the association information. When the same association appears in multiple corpus blocks, their descriptions are merged to form a comprehensive and unified association representation.
[0037] Knowledge base storage and optimization: Based on the "vector database + hypergraph database", a question-answer pair index layer is added to achieve a three-in-one storage of "entity-relationship-question-answer pair". (1) Vector database: In addition to storing the 384-dimensional vectors of entities and low-order / high-order associations, it also stores the vectors of all valid question-answer pairs (question + answer) (generated by the all-MiniLM-L6-v2 encoder). It indexes the entities, pairwise entity associations, multi-entity associations and question-answer pairs. As a vector database, it supports millisecond-level cross-type vector matching.
[0038] (2) Hypergraph Database: Add a "List of Related Question-Answer Pair IDs" field to each entity and association, record all basic question-answer pairs and augmented question-answer pair IDs in which the entity / association participates, realize the reverse tracing of "hypergraph topology relationship - question-answer pair", and obtain the hypergraph database.
[0039] Question-Answer Pair-Hypergraph Association Calibration: Through LLM+domain expert collaboration, the consistency between augmented question-answer pairs and hypergraph knowledge is verified, and the augmented question-answer pairs are calibrated as a question-answer pair index layer; specifically: For each augmented question-answer pair, the corresponding "entity set + association set" in the hypergraph is automatically matched to verify whether the answer is consistent with the entity description and association information in the hypergraph. If there is a slight deviation (such as different expressions but consistent core facts), the question-answer pair answer is corrected. If there is a serious deviation (such as factual errors), it is moved to the "adversarial question-answer pair library" for retrieval robustness testing only and does not participate in regular retrieval. After calibration, each augmented question-answer pair is strongly bound to 1-3 entities and 1-2 associations (low-order / high-order) in the hypergraph as a question-answer pair index layer to ensure the accuracy of "question-answer pair retrieval → hypergraph knowledge positioning".
[0040] S2: After receiving the user query, extract the entity keywords and related keywords from the user query, match similar question-and-answer pairs in the domain knowledge base based on the entity keywords and related keywords, and retrieve the search information from the entity set, the association set, and the similar question-and-answer pairs through entity association retrieval, related entity retrieval, and question-and-answer pair matching retrieval respectively; combined with the context length limit of the Large Language Model (LLM), select the optimal supplementary knowledge set from the search information according to different weighted filtering dimensions; the filtering dimensions include semantic similarity with the query, knowledge importance, information redundancy, and domain relevance.
[0041] Keyword extraction in two types: receiving user queries q Then, based on a dedicated prompt template Inputting user queries into the LLM function yields entity keywords and related keywords, expressed as follows: ; in, X entThis is a set of entity keywords, containing discrete entities explicitly mentioned in the query (such as "Parkinson's disease", "Article 100 of the Civil Code", and "drug L-DOPA"), used to locate core entities in the knowledge base; X cor The associated keyword set contains abstract information describing interactions between entities (such as "combined treatment plan," "applicable legal provisions," and "side effect incidence"), used to locate relationships within the knowledge base. This design enables deep analysis of query intent, avoiding the information bias caused by relying solely on entity retrieval.
[0042] User queries q Vectorization is performed, and similarity matching is conducted with question-answer pairs in the vector database. The top-K (in this case, 5) most similar question-answer pairs (including basic and augmented question-answer pairs) are selected, and the "entity set" associated with these question-answer pairs is extracted. "and association set" ", as a supplementary anchor point for keyword retrieval.
[0043] Three-branch retrieval mechanism: Based on two types of keywords, a three-branch retrieval process of "entity-association-question-answer pair" is designed to ensure the comprehensiveness of the retrieved information.
[0044] (1) Entity Association Retrieval Branch: First, using the vector retrieval function, retrieve entities from the entity set V of the Hypergraph database that are related to the target entity. Matching related entities The formula is: ; in, To calculate the matching degree between keywords and entity vectors based on cosine similarity, the top-10 entities with the highest similarity were selected as the core search results. Subsequently, a knowledge diffusion process was initiated, leveraging the topological relationships of the hypergraph to obtain... All directly connected associations (including lower-order associations) E low Higher-order associations ),Right now This enables the extension and supplementation of information from core entities to related information.
[0045] (2) Related entity retrieval branch: using vector retrieval functions From the association set of the Hypergraph database ( ∪ Searching and Matching related associations The formula is: ; Similarly, the top-10 similarity associations were selected as the core search results. Then, a knowledge diffusion process was initiated to obtain related... All directly connected entities ,Right now:
[0046] ; It enables the extension and supplementation from the relationship to the related entities.
[0047] (3) Question-answer pair matching retrieval branch: For the selected Top-5 similar question-answer pairs, extract their corresponding "standard answer + associated hypergraph knowledge (entity + association)": If the semantic similarity between the similar question-answer pair and the user query is ≥0.7 (cosine similarity), directly use the hypergraph knowledge corresponding to its standard answer as "priority supplementary knowledge"; if the similarity is between 0.5 and 0.7, merge the hypergraph knowledge associated with the question-answer pair with the retrieval results of other branches; if the similarity is <0.5, only the entity / association anchor point of this branch is retained, and it is not included in the supplementary knowledge to avoid redundancy.
[0048] Search result filtering and sorting: Combining the context length limit of LLM (such as the 8k token context of GPT-4o), the results obtained from dual-branch retrieval and diffusion are analyzed. , , , Multi-dimensional filtering is performed. Filtering dimensions include: semantic similarity to the query (weight 0.4), knowledge importance (core entities / related information have higher weight than secondary information, weight 0.3), information redundancy (removing duplicate or highly similar content, weight 0.2), and domain relevance (ensuring information is consistent with the query's domain, weight 0.1). The results are ranked using weighted scoring, and Top-K relevant information (K value dynamically adjusted based on the LLM context length, typically 5-10 items) is filtered out. Simultaneously, the corresponding original corpus text is linked to form the final optimal supplementary knowledge set.
[0049] S3: Construct a prompt template for query, supplementary knowledge, and format requirements. The original query and the optimal supplementary knowledge set are merged using the prompt template. The merged prompt template is then input into the LLM to generate the final response. The final response is then corrected using a question-and-answer pair reverse verification mechanism to obtain the corrected generated result.
[0050] Structured prompt construction: A three-in-one prompt template integrating "query - supplementary knowledge - format requirements" is designed to organically combine the original query q with the filtered supplementary knowledge. The prompt template includes three parts: first, query restatement, which clarifies the core problem that LLM needs to solve; second, knowledge reference, which presents supplementary knowledge in the order of "entity information - related information - original text", and marks the source of each knowledge (e.g., "from corpus block D_123") to enhance credibility; and third, format requirements, which clarify that LLM should output a structured response (e.g., point-by-point explanation, annotation of factual basis) to avoid vague expressions. This template ensures that LLM can efficiently utilize supplementary knowledge, while standardizing the output format and improving the readability and verifiability of the response.
[0051] Enhanced generation process: Input the constructed prompts into the target LLM (supporting mainstream models such as GPT-4oMini, Qwen-Plus, and LLaMa-3.3-70B), and generate the final response, expressed by the formula: ; Where q represents the user query, D represents the structured corpus, RAG represents the optimal supplementary knowledge retrieved based on hypergraph topological relationships, and P represents the P-value. q To integrate the original query q and the optimal supplementary knowledge RAG() into a prompt template, the LLM model represents the model based on P. q The generated response.
[0052] During the generation process, LLM uses supplementary knowledge as a factual basis to constrain its own generation logic and avoid fabrications that deviate from the facts. At the same time, because the supplementary knowledge includes the original corpus text, it can effectively make up for the knowledge loss during the model training process and reduce the probability of hallucinations.
[0053] Result Validation and Correction: A "question-answer pair reverse validation mechanism" is introduced to perform secondary validation on the responses generated by LLM, further suppressing illusions. (1) Perform semantic consistency detection on the generated response and similar question-and-answer pairs and supplementary knowledge selected by S3, and calculate "Fact matching degree FM: length of fact-consistent response / total length of response": if FM≥0.9: output the response directly; if 0.7≤FM<0.9: mark the inconsistent parts, and output them after correction based on supplementary knowledge and similar question-and-answer pairs by LLM; if FM<0.7: refuse to output the response, return "The generated result of the current query has not passed the fact verification, it is recommended to supplement the query conditions or refer to relevant examples", and attach the Top-2 similar question-and-answer pairs for user reference; (2) The verification process is recorded in logs throughout, including "generated response → factual basis → matching degree → correction record", to ensure traceability.
[0054] This invention constructs a complete technical solution from knowledge modeling to generation optimization through the above four steps. The core innovation lies in the hypergraph-driven knowledge association modeling and the dual-branch retrieval diffusion mechanism, which fundamentally solves the problems of missing high-order associations and performance-efficiency imbalance in existing RAG technologies, and significantly improves the factual accuracy and adaptability of LLM response to complex queries.
[0055] In one embodiment: 1. The specific implementation of hypergraph construction is as follows: The core of the hypergraph construction in this invention lies in the accurate extraction of low-order and high-order associations, and the specific process is as follows: Entity extraction prompts Using structured templates, the LLM is explicitly required to output "entity name-entity description" key-value pairs. For example, in a medical corpus, structured information such as "Parkinson's disease - a neurodegenerative disease with symptoms including tremor and rigidity" can be extracted.
[0056] Low-order association extraction hints Focus on pairwise entity relationships, such as "Drug A - Drug B - Combined use can enhance efficacy"; extract higher-order association hints. This explicitly requires the identification of interactions between three or more entities, such as "Drug A - Drug B - Patient Group C - Combination therapy reduces the incidence of side effects in Group C by 30%".
[0057] The vector database is implemented using FAISS. The vectors of entities and relationships are generated by the all-MiniLM-L6-v2 encoder, with a dimension of 384, and support millisecond-level retrieval. The hypergraph database uses a dedicated graph storage engine to record the topological relationships between entities and relationships, and supports efficient diffusion queries.
[0058] 2. Mathematical Model for Knowledge Retrieval: The retrieval process of this invention ensures accuracy and completeness through the following mathematical model: Vector search function The matching degree between keywords and entities / associations is calculated based on cosine similarity, using the following formula: ; in This is a vector encoding function.
[0059] Knowledge diffusion employs a single-step diffusion strategy to avoid information redundancy caused by excessive diffusion. The diffusion scope is controlled to entities / associations directly related to the search results, ensuring the relevance of supplementary knowledge to the query. The hypergraph-driven domain knowledge base graph is as follows: Figure 4 As shown.
[0060] 3. The key mechanism for hallucination relief: This invention alleviates LLM hallucinations through a dual mechanism: Knowledge Coverage Mechanism: Hypergraphs simultaneously model low-order and high-order relationships, improving the coverage of relational, pattern-based, and reasoning-based knowledge by 7%-9.6% compared to traditional knowledge graphs, with a particularly significant improvement in the coverage of multi-entity pattern knowledge.
[0061] Error suppression mechanism: By retaining the original corpus text as supplementary knowledge, the secondary processing of retrieval results by LLM is reduced, the key knowledge missing rate (KMR) is reduced by 35%, the illusion error rate (HER) is reduced by 50%, and the error propagation coefficient (α) is reduced from 0.59 to 0.14, effectively suppressing error amplification.
[0062] 4. Performance Validation: On the Neurology Crop medical dataset, this invention improves the average accuracy of six mainstream LLMs by 12.3%, compared to Graph RAG by 6.3% and Light RAG by 6.0%. As query difficulty increases (from single-stage to three-stage nested queries), performance remains stable, with the relative improvement increasing from 12.7% to 15.0%, while existing methods show a significant performance decline. On nine multi-domain datasets, through selective evaluation, this invention achieves an average performance improvement of 35.5% compared to Light RAG, with a 55.3% improvement in the legal domain and a 37.5% improvement in the financial domain.
[0063] Based on the above content and statements, this invention proposes an augmented question-answering pair retrieval enhancement generation system, comprising a corpus processing module, a knowledge retrieval module, and a generation optimization module; specifically: The corpus processing module acquires a structured corpus of question-answer pairs in the medical field, extracts entity sets and corresponding descriptive information from the structured corpus, and extracts association sets for paired entities and multiple entities. It then combines these entity-association-question-answer pairs to obtain a domain knowledge base. Entities include diseases, drugs, and symptoms, and associations represent the interaction relationships between entities. The knowledge retrieval module receives user queries, extracts entity keywords and associated keywords from the query, matches similar question-answer pairs in the domain knowledge base based on these keywords, and performs entity association retrieval, associated entity retrieval, and question-answer pair matching retrieval. Instead of retrieving information from entity sets, association sets, and similar question-answer pairs, the optimal supplementary knowledge set is selected from the retrieved information based on different weighted filtering dimensions, taking into account the context length limitations of the Large Language Model (LLM). These filtering dimensions include semantic similarity to the query, knowledge importance, information redundancy, and domain relevance. The generation optimization module is used to construct a prompt template that combines query, supplementary knowledge, and format requirements. The original query and the optimal supplementary knowledge set are then fused using the prompt template. The fused prompt template is input into the LLM to generate the final response. The final response is then corrected using a question-answer pair reverse verification mechanism to obtain the corrected generated result.
[0064] The present invention also provides a computer device, including a memory and a processor, wherein the memory stores a program, and when the program is executed by the processor, the processor performs the steps of an augmented question-answer pair retrieval enhancement generation method.
[0065] According to the disclosed embodiments, the computer device can communicate with one or more external devices (e.g., keyboard, pointing device, Bluetooth communication, etc.) or with any device that enables the computing device to communicate with one or more other computing devices (e.g., router, demodulator, etc.).
[0066] The present invention also provides a storage medium storing a computer program thereon, characterized in that the computer program, when executed by a processor, implements the steps of an augmented question-answer pair retrieval enhancement generation method.
[0067] According to the disclosed embodiments, the storage medium can be a non-volatile computer-readable storage medium, such as, but not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this invention, the storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.
[0068] The above description, in conjunction with specific preferred embodiments, provides a more detailed explanation of the present invention. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such deductions or substitutions should be considered to fall within the scope of protection of the present invention.
Claims
1. A method for enhancing retrieval through augmented question-answering pairs, characterized in that, Including the following steps: We acquire a structured corpus of question-answer pairs in the medical field, extract entity sets and corresponding descriptive information from the structured corpus, extract the association sets of paired entities and multiple entities, and combine entity-association-question-answer pairs to obtain a domain knowledge base. The entities mentioned include diseases, drugs, and symptoms, and associations represent the interaction relationships between entities; After receiving a user query, the system extracts entity keywords and related keywords from the query. It then matches similar question-and-answer pairs from the domain knowledge base based on these keywords. The system retrieves information from the entity set, association set, and similar question-and-answer pairs through entity association retrieval, related entity retrieval, and question-and-answer pair matching retrieval, respectively. Taking into account the context length limitation of the Large Language Model (LLM), the system selects the optimal supplementary knowledge set from the retrieved information based on different weighted filtering dimensions. These dimensions include semantic similarity to the query, knowledge importance, information redundancy, and domain relevance. A query-supplementary knowledge-formatting requirement prompt template is constructed. The original query and the optimal supplementary knowledge set are merged through the prompt template. The merged prompt template is input into the LLM to generate the final response. The final response is then corrected using a question-and-answer pair reverse verification mechanism to obtain the corrected generated result.
2. The augmented question-answering pair retrieval enhancement generation method as described in claim 1, characterized in that, The acquisition of a structured corpus of question-answer pairs in the medical field specifically involves: We collect various types of corpora in the medical field and preprocess them to obtain preprocessed corpora. A dual-blocking strategy combining fixed-length blocks and semantic blocks is adopted to block the preprocessed corpus, generate basic question-answer pairs, and then augment the basic question-answer pairs in multiple dimensions to obtain augmented question-answer pairs. A question-answer pair resource library is constructed based on the aforementioned basic question-answer pairs and augmented question-answer pairs, serving as the final structured corpus.
3. The augmented question-answering pair retrieval enhancement generation method as described in claim 1, characterized in that, The process of extracting entity sets and corresponding descriptive information from the structured corpus, as well as extracting the association sets of paired entities and multiple entities, and combining these with entity-association-question-answer pairs to obtain the domain knowledge base, specifically involves: Define a hypergraph structure that contains a set of vertices and a set of associations of entities; Extract entities and their corresponding descriptive information from the structured corpus, design differentiated prompt templates, and extract the associations of paired entities and multiple entities. The system indexes entities, pairwise entity associations, multi-entity associations, and question-answer pairs to form a vector database. A list field of associated question-answer pair IDs is added to each entity and association to obtain a hypergraph database. Through collaboration between LLM and domain experts, the system verifies the consistency between augmented question-answer pairs and the knowledge in the hypergraph structure, and corrects the augmented question-answer pairs to form a question-answer pair index layer. A storage architecture is constructed using the aforementioned vector database, hypergraph database, and question-answering pair index layer, serving as a domain knowledge base.
4. The augmented question-answering pair retrieval enhancement generation method as described in claim 3, characterized in that, The process involves extracting entities and their corresponding descriptions from a structured corpus, designing differentiated prompt templates, and extracting pairwise entity associations and multi-entity associations. Specifically: Obtain the entity extraction template, drive the LLM to extract entities and description information from each corpus block, and when multiple corpus blocks contain the same entity, merge and deduplicate the description information through the LLM to obtain the final description information. Design differentiated prompt templates, identify the relationship types and descriptions between pairs of entities using LLM, and output the association of pairs of entities in the format of entity1-relationship-entity2-description. Identify the interaction logic of three or more entities using LLM, clarify the role and function of each entity in the association, and output the association of multiple entities in the format of entity1-entity2-...-entityN-relationship-description.
5. The augmented question-answering pair retrieval enhancement generation method as described in claim 3, characterized in that, The process involves collaborating with LLM and domain experts to verify the consistency between augmented question-answer pairs and the knowledge in the hypergraph structure, and to correct the augmented question-answer pairs as a question-answer pair index layer. Specifically: For each augmented question-answer pair, the system automatically matches the corresponding entity set and association set in the Hypergraph database to verify whether the answer is consistent with the entity description and association information in the Hypergraph. If there are differences in expression but the core facts are consistent, the augmented question-and-answer pair answer will be corrected; if there are factual errors, it will be moved to a dedicated database for adversarial question-and-answer pairs, used only for robustness testing and not for regular retrieval. Each augmented question-answer pair after correction is bound to multiple entities and multiple associations in the hypergraph, serving as a question-answer pair index layer.
6. The augmented question-answering pair retrieval enhancement generation method as described in claim 3, characterized in that, After receiving a user query, the entity keywords and related keywords in the user query are extracted, specifically as follows: Based on a dedicated prompt template, the user's query input into the LLM is used to obtain entity keywords and related keywords; User queries are vectorized and matched with question-answer pairs in the vector database for similarity. The top-K most similar question-answer pairs are selected, and the entity key set and association key set associated with the top-K most similar question-answer pairs are extracted as supplementary anchors for keyword retrieval.
7. The augmented question-answering pair retrieval enhancement generation method as described in claim 1, characterized in that, The query-supplementary knowledge-format requirements prompt template includes three parts: query restatement, knowledge reference, and format requirements. The knowledge reference is presented in the order of entity information-related information-original text.
8. An augmented question-answering pair retrieval enhancement generation system, characterized in that, include: The corpus processing module is used to acquire a structured corpus of question-answer pairs in the medical field, extract entity sets and corresponding descriptive information from the structured corpus, extract the association sets of paired entities and multiple entities, and combine entity-association-question-answer pairs to obtain the domain knowledge base. The entities mentioned include diseases, drugs, and symptoms, and associations represent the interaction relationships between entities; The knowledge retrieval module receives user queries, extracts entity keywords and related keywords from the query, matches similar question-and-answer pairs in the domain knowledge base based on the entity keywords and related keywords, and retrieves search information from the entity set, association set, and similar question-and-answer pairs through entity association retrieval, related entity retrieval, and question-and-answer pair matching retrieval, respectively. Taking into account the context length limitation of the Large Language Model (LLM), the module selects the optimal supplementary knowledge set from the search information according to different weighted filtering dimensions. These filtering dimensions include semantic similarity to the query, knowledge importance, information redundancy, and domain relevance. The generation optimization module is used to construct a prompt template for query-supplementary knowledge-format requirements. The original query and the optimal supplementary knowledge set are merged through the prompt template. The merged prompt template is input into the LLM to generate the final response. The final response is then corrected using a question-and-answer pair reverse verification mechanism to obtain the corrected generation result.
9. A computer device, characterized in that, The method includes a memory and a processor, wherein the memory stores a program that, when executed by the processor, causes the processor to perform the steps of an augmented question-answer pair retrieval enhancement generation method as described in any one of claims 1 to 7.
10. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the augmented question-answer pair retrieval enhancement generation method according to any one of claims 1 to 7.