Method and system for retrieval augmentation generation based on multi-level semantic hypergraph

Abstract

This invention belongs to the field of knowledge retrieval technology and discloses a retrieval enhancement generation method and system based on a multi-level semantic hypergraph. It constructs a multi-level semantic hypergraph containing fuzzy overlapping topics, semantic blocks, sentences, and entities. In the indexing stage, cascaded modeling is performed in the order of topics, semantic blocks, sentences, and entities, and the representation of each layer's nodes is updated using a dynamic self-attention fusion operator. In the retrieval stage, a query cascaded representation symmetrical to the index structure is constructed based on the user query. The weights of semantic blocks, sentences, and entity evidence are adaptively allocated using a dynamic multi-granularity scoring operator to obtain the matched sentences. Related evidence is obtained through multi-hop sharing of entities. Consistency verification and evidence rearrangement generate a reliable context for large-scale model responses. This invention avoids the semantic breaks, entity omissions, and evidence drift problems caused by fixed slices and fixed weights in traditional RAG, improving recall, evidence consistency, and generation reliability in complex question answering.

Description

Technical Field

[0001] This invention belongs to the field of knowledge retrieval technology, and more specifically, relates to a retrieval enhancement generation method and system based on a multi-level semantic hypergraph. Background Technology

[0002] Retrieval-Augmented Generation (RAG) can alleviate problems such as model illusion, outdated knowledge, and insufficient context in long documents by introducing external knowledge retrieval results before generating a large language model. However, existing RAG systems typically rely on fixed window slicing, vector similarity recall, and simple rearrangement strategies, which are insufficient to fully express the hierarchical semantic relationships within long documents.

[0003] In complex question-answering scenarios, user queries often involve topic intent, paragraph semantics, sentence facts, and entity constraints simultaneously. Retrieving solely based on semantic block vectors can easily result in semantic blocks being matched but key entities missing; conversely, relying solely on entity matching can easily lead to the loss of contextual causal relationships and paragraph-level semantics. Furthermore, if manually set weighting formulas are used, such as fixed weights for semantic blocks, sentences, and entities, these fixed weights are difficult to adapt to different query intents.

[0004] Furthermore, the same topic in multi-source documents often spans multiple chapters or documents, and topic boundaries are not mutually exclusive. Existing RAG methods based on clustering or single-layer graph structures struggle to handle issues such as topic overlap, cross-granularity evidence propagation, and inconsistencies between the query and index structures, leading to defects in recalled evidence such as fragmentation, drift, or excessive noise. Summary of the Invention

[0005] To address the aforementioned deficiencies or improvement needs of existing technologies, this invention provides a retrieval enhancement generation method and system based on a multi-level semantic hypergraph. Its purpose is to solve problems such as semantic fragmentation caused by fixed slices, insufficient intent adaptation capability, entity omission caused by single-layer vector retrieval, and insufficient evidence consistency in existing RAG methods.

[0006] To achieve the above objectives, this invention provides a retrieval enhancement generation method based on a multi-level semantic hypergraph, comprising: performing text cleaning and semantic block segmentation on the document to be indexed to obtain a semantic block set. Each semantic block is processed into sentences to construct a hypergraph of relationships between semantic blocks and sentences (L1). Each sentence is processed into entities to construct a hypergraph of relationships between sentences and entities (L2). Based on the set of semantic blocks... A semantic block relationship graph is constructed based on the similarity between semantic blocks. The semantic block relationship graph is then divided into communities to obtain a set of candidate topic clusters. K is The total number of candidate topic clusters; calculate the candidate topic clusters. Semantic blocks other than candidate topic clusters The fuzzy membership degree is used to add semantic blocks with a fuzzy membership degree greater than a preset membership degree threshold to the candidate topic cluster. To construct a hypergraph L0 relating candidate topic clusters and semantic blocks; The relational hypergraphs L0, L1, and L2 are cascaded together to form a multi-level semantic hypergraph. Based on the multi-level semantic hypergraph, the embedding vector of each semantic block, the topic center vector of the candidate topic cluster to which the semantic block belongs, and the aggregation vector of the sentences contained in the semantic block are dynamically self-attention fused to obtain the concatenated semantic vector of the semantic block; the embedding vector of each sentence, the concatenated semantic vector of the semantic block to which the sentence belongs, and the aggregation vector of the entities contained in the sentence are dynamically self-attention fused to obtain the concatenated semantic vector of the sentence; the embedding vector of each entity and the concatenated semantic vector of the sentence to which the entity belongs are dynamically self-attention fused to obtain the concatenated semantic vector of the entity. The user-input query is semantically expanded to form query text. , query text Perform entity extraction to form a query entity set. Based on query text Embedded vector The set of candidate topic clusters that are matched by the query is obtained by comparing the similarity between the topic center vectors of each candidate topic cluster and the topic center vectors of each candidate topic cluster. ;Will query topic aggregation vector and Dynamic self-attention fusion is performed to obtain the query semantic block vector. ,use exist Obtain the set of semantic blocks matched by the query. ;Will query entity aggregation vector , and Dynamic self-attention fusion is performed to obtain the query sentence vector. ; Will The included sentences form a candidate sentence set. For each candidate sentence, a separate calculation is performed. The relevance score between the candidate sentence and the concatenated semantic vector of the semantic block to which the candidate sentence belongs. ,calculate Relevance score between the concatenated semantic vectors of the candidate sentence Calculate the concatenated vector of query entities The relevance score between the candidate sentence and the concatenated semantic vectors of the entities contained in the candidate sentence. , based on and The query entity embedding vectors are dynamically self-attention fused to obtain the relevance score; , , Weighted fusion is performed to obtain the comprehensive retrieval score for each candidate sentence. ;choose The top k candidate sentences with the highest resolution are used as query hit sentences; the required context is obtained based on the query hit sentences so that the large model can output retrieval enhancement results based on the required context.

[0007] Furthermore, adopt The operator performs the dynamic self-attention fusion, the The operator is:

[0008]

[0009] in, For anchor vector, The set of vectors to be merged. , Let be the total number of vectors to be fused. This represents the x-th vector to be fused. Let y be the vector to be fused. , Anchor vector transpose, for Attention weights For temperature parameters, This indicates a normalization operation; use When operators compute concatenated semantic vectors of semantic blocks, anchor vectors For semantic blocks Embedded vector , , For semantic blocks For candidate topic clusters Fuzzy membership degree, Candidate topic clusters Theme center vector, Represents semantic blocks The weighted sum of the topic center vectors of its candidate topic clusters. For semantic blocks The aggregate vector of the contained sentences; use When operators compute the concatenated semantic vector of a sentence, the anchor vector... For sentences Embedded vector , ,in, , For sentences To which the semantic block set belongs For sentences its semantic block The attribution weight, For sentences Belonging semantic block Cascaded semantic vectors For sentences The aggregate vector of the contained entities; use When operators compute the concatenated semantic vectors of entities, anchor vectors For entities Embedded vector , , For entities Sentence Cascaded semantic vectors; use Operator computation query semantic block vector At that time, anchor vector For query text Embedded vector , ; use Operators calculate query sentence vectors At that time, anchor vector For query text Embedded vector , ; use Operators calculate the concatenated vector of query entities At that time, anchor vector For querying the entity set Query entities Embedded vector , .

[0010] Furthermore, the comprehensive search score for:

[0011]

[0012]

[0013]

[0014]

[0015] in, This represents the concatenated semantic vector of the semantic block to which the candidate sentence belongs. This represents the concatenated semantic vectors of the candidate sentences. Entities contained in the candidate sentence Cascaded semantic vectors; For sentences The set of entities contained in a sentence Represents the current candidate sentence; To query the set of concatenated vectors of entities; , , They are respectively , , The corresponding fusion weights; This refers to the temperature parameter.

[0016] Furthermore, the fuzzy membership degree is:

[0017]

[0018] in, For semantic blocks For candidate topic clusters Fuzzy membership degree, For semantic blocks Embedded vector, Candidate topic clusters Theme center vector, The preset temperature parameters; For candidate topic clusters The Middle Candidate topic clusters The theme center vector; This indicates a normalization operation; This is for taking the absolute value.

[0019] Furthermore, the aggregation vector of the sentences contained in the semantic block is:

[0020]

[0021] in, For semantic blocks The aggregate vector of the contained sentences, For semantic blocks The set of sentences contained therein For sentences Embedded vector, For sentences Attention weights For semantic blocks Embedded vector, This indicates a normalization operation; The aggregate vector of the entities contained in the sentence is:

[0022] in, Sentence The aggregation direction of the contained entities; Sentence The first in One entity, For sentences The set of entities contained therein Representing entities Embedded vector, Representing entities In the sentence Importance weights in; query entity aggregation vector for:

[0023] in, express Query entities In the query text Importance weights in; Indicates the query entity The embedding vector; query topic aggregation vector for:

[0024] in, For query text Embedded vector With candidate topic clusters Topic center vector The similarity.

[0025] Furthermore, candidate topic cluster set for: ;in, For query text Embedded vector With candidate topic clusters Topic center vector similarity, Represents the query text Embedded vector The maximum similarity between the topic center vectors of each candidate topic cluster and the topic center vector of each candidate topic cluster. This represents a relative threshold.

[0026] Furthermore, using exist Obtain the set of semantic blocks matched by the query. ,include: With the set of candidate topic clusters matched by the query The associated semantic blocks are used as candidate ranges to calculate the query semantic block vector. The relevance scores between the semantic vectors of each semantic block within the candidate range and the concatenated semantic vectors are ranked, and semantic blocks with scores exceeding a preset threshold are selected to form the set of semantic blocks matched by the query. ; The required context is obtained based on the query-matched sentence, including: The query-matched sentence is used as the first-hop evidence. Based on the shared entities in the first-hop evidence, multiple hops are performed to obtain related sentences. The query-matched sentence and the obtained related sentences are used as a candidate evidence set. A consistency check is performed on the candidate evidence in the candidate evidence set to determine whether the candidate evidence can answer the user's input query. Candidate evidence that can answer the query is then ranked according to the comprehensive search score. The semantic blocks to which the candidate evidence belongs are rearranged and spliced ​​together to form the required context.

[0027] The present invention also provides a retrieval enhancement generation system based on a multi-level semantic hypergraph, including a computer-readable storage medium and a processor; The computer-readable storage medium is used to store executable instructions; The processor is used to read executable instructions stored in the computer-readable storage medium and execute the retrieval enhancement generation method based on a multi-level semantic hypergraph as described above.

[0028] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the retrieval enhancement generation method based on a multi-level semantic hypergraph as described in any of the preceding claims.

[0029] The present invention also provides a computer program product, including a computer program that, when run on a computer, causes the computer to execute the retrieval enhancement generation method based on a multi-level semantic hypergraph as described above.

[0030] In summary, the above-described technical solutions conceived in this invention can achieve the following beneficial effects: (1) This invention constructs a multi-level semantic hypergraph by connecting topics, semantic blocks, sentences, and entities. This hypergraph consists of a topic layer (L0 topic hypergraph, used to represent the relationship between topics and semantic blocks), a semantic block layer (L1 semantic block hypergraph, used to represent the inclusion relationship between semantic blocks and sentences), and a sentence layer (L2 sentence hypergraph, used to represent the inclusion relationship and semantic role relationship between sentences and entities). This allows macro-level topic information, semantic block context information, sentence factual information, and entity constraint information in the knowledge base to be indexed to be modeled simultaneously, thereby reducing the semantic fragmentation problem caused by traditional fixed slicing methods. Through fuzzy overlapping topic relationships (fuzzy membership), a semantic block can be associated with multiple topics with different membership degrees, adapting to cross-topic, cross-chapter, and multi-semantic attribution scenarios common in long documents and multi-source knowledge bases. Based on the constructed multi-level semantic hypergraph, this invention achieves alignment between the semantic block-level, sentence-level, and entity-level representations of user queries and the multi-level semantic structure of the index through cascaded updates at the index end (calculation of cascaded semantic vectors for semantic blocks, sentences, and entities) and symmetrical cascading at the query end. This improves evidence recall capabilities in complex query scenarios. By dynamically calculating the cascaded semantic vectors of semantic blocks, sentences, and entities through self-attention fusion, and by replacing fixed empirical weights with dynamic fusion of semantic block-level, sentence-level, and entity-level relevance scores, this invention can automatically adjust the importance of evidence at the semantic block, sentence, and entity granularities according to the user's query intent. This solves the problems of semantic fragmentation caused by fixed slices, insufficient intent adaptation, entity omissions due to single-layer vector retrieval, and insufficient evidence consistency in existing RAG methods.

[0031] (2) Preferably, the dynamic self-attention fusion operator of the present invention calculates the attention weights by the dot product between the anchor vector and the vector to be fused, and obtains the cascaded update vector (dynamic self-attention fusion result) by weighted summation and normalization. The designed dynamic self-attention fusion operator does not introduce additional trainable parameters, only processes a small number of cross-layer correlation vectors of the current node, and does not require the construction of a global sequence attention matrix. Therefore, it has the advantages of low computational cost, no need for additional training, and easy incremental update.

[0032] In summary, this invention is a retrieval enhancement generation method that can simultaneously express multi-level relationships of topics, semantic blocks, sentences, and entities, and can dynamically adjust the weights of evidence at different granularities according to the query intent. It avoids the semantic breaks, entity omissions, and evidence drift problems caused by the fixed slices and fixed weights of traditional RAG, and improves the recall rate, evidence consistency, and generation reliability in complex question answering. Attached Figure Description

[0033] Figure 1A flowchart illustrating the method provided in an embodiment of the present invention.

[0034] Figure 2 This is a schematic diagram of a multi-level semantic hypergraph structure provided in an embodiment of the present invention.

[0035] Figure 3 A schematic diagram of the query process provided for the implementation of this invention.

[0036] Figure 4 A comparison chart of example experimental scores provided for embodiments of the present invention. Detailed Implementation

[0037] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0038] Example 1 This invention addresses scenarios involving long documents, multi-source knowledge bases, and complex semantic queries. It constructs a holographic hypergraph index comprising a fuzzy overlapping topic manifold layer, a macro-level semantic block layer, a meso-level sentence layer, and a micro-level entity layer. During the indexing phase, strict cascading modeling is performed in the order of topic, semantic block, sentence, and entity, and the node representations of each layer are updated using a dynamic self-attention fusion operator. In the retrieval phase, semantic expansion and entity extraction are performed on the user-input query. The expanded query is matched with the topic center vector at the index end to determine the hit topic cluster and obtain the query topic guidance vector (query topic aggregation vector). Based on the query topic guidance vector, the expanded query vector (embedding vector of the expanded query text), and the query entity aggregation vector, dynamic self-attention fusion is used to construct the query semantic block vector, query sentence vector, and context vector. The invention enhances the representation of query entities; based on the query semantic block vector, coarse-grained recall is performed within the semantic blocks associated with the hit topic cluster to determine candidate semantic blocks (the set of semantic blocks hit by the query) and their contained candidate sentences; for each candidate sentence, semantic block-level relevance score, sentence-level relevance score, and entity-level relevance score are calculated, and the fusion weight of each score is adaptively determined through a dynamic multi-granularity scoring operator to obtain the comprehensive retrieval score of the candidate sentence; the candidate sentences are ranked according to the comprehensive retrieval score, and the top-k candidate sentences that meet the preset conditions are selected as the query hit sentences, and multi-hops are performed according to their corresponding entities to find related candidate evidence to form the final set of candidate evidence hit by the query; subsequently, a credible context for answering large language models is generated through consistency verification and evidence rearrangement. This invention avoids the semantic breaks, entity omissions, and evidence drift problems caused by the fixed slices and fixed weights of traditional RAG, and improves the recall rate, evidence consistency, and generation reliability in complex question answering. The specific scheme is as follows.

[0039] like Figure 1-4 As shown, embodiments of the present invention provide a retrieval enhancement generation method and system based on a multi-level semantic hypergraph, such as... Figure 1 As shown, it includes the following: Text processing: The system first obtains the document to be processed (the knowledge base to be indexed). The document can be a Markdown file, a plain text file, knowledge entries exported from a database, or other parsable text. The system performs noise reduction, whitespace merging, title preservation, and fragment filtering on the document, and divides it into multiple semantic blocks according to the preset maximum number of tokens and the number of overlapping tokens. In this embodiment of the invention, the knowledge base to be indexed is represented as: ;in, This indicates the knowledge base to be indexed. Indicates the first There are 10 documents to be indexed (documents to be processed). After semantically segmenting the knowledge base to be indexed, a set of semantic blocks is obtained. ; For semantic block set The first in A semantic block, .

[0040] For each semantic block The system calls a sentence extraction model to obtain a set of sentences. At the semantic block level, a hypergraph representing the relationship between semantic blocks and sentences is constructed (L1 semantic block hypergraph, where hyperedges are semantic blocks and nodes are sentences, used to represent the inclusion relationship between semantic blocks and sentences). Subsequently, multiple rounds of entity extraction are performed on each sentence to obtain an entity set. At the sentence level, a hypergraph representing the relationship between sentences and entities is constructed (L2 sentence hypergraph, where hyperedges are sentences and nodes are entities extracted from sentences, used to represent the inclusion relationship and semantic role relationship between sentences and entities). Entity types include, but are not limited to, people, places, organizations, items, time, values, core actions, purposes, reasons, results, and methods. The entity extraction results carry the importance weight of the entity in the corresponding sentence for subsequent entity-level evidence aggregation. The system uses an embedding model to calculate the corresponding original vectors (embedding vectors) for semantic blocks, sentences, and entities respectively.

[0041] Fuzzy overlapping topic manifold construction: The system constructs a semantic block relationship graph based on the similarity between the original vectors of semantic blocks; the nodes in the semantic block relationship graph are sets of semantic blocks. In this invention, semantic blocks are defined, and edges represent the similarity between them. In this embodiment, cosine similarity is used; however, other similarity calculation methods may be used in other embodiments. A graph community detection algorithm (such as the Leiden community detection algorithm) is employed to partition the semantic block relationship graph, resulting in a set of candidate topic clusters. .

[0042] For each candidate topic cluster The system calculates the topic center vector corresponding to the topic cluster based on the semantic vectors (original vectors) of the semantic block nodes within the topic cluster. The topic center vector is used as the topic representation of the candidate topic cluster. This represents the k-th candidate topic cluster in the set of candidate topic clusters. The topic center vector (topic representation); the set of topic representations corresponding to the candidate topic cluster set is , k∈{1,2,…,K}. Unlike hard clustering, this invention allows semantic blocks to be associated with multiple candidate topic clusters. Specifically, for candidate topic clusters... Other semantic blocks and candidate topic clusters Corresponding topic representation Calculate the similarity between the two and then calculate the semantic block based on this similarity. For candidate topic clusters Fuzzy membership degree ;when Greater than or equal to the preset topic membership threshold At that time, in the semantic block and candidate topic clusters Establish topic-based cascading edges between semantic blocks. Add to candidate topic cluster In the process, the updated candidate topic clusters are obtained. At the topic level, a hypergraph representing the relationships between topic clusters and semantic blocks is constructed (L0 topic hypergraph, where hyperedges are topic clusters and nodes are semantic blocks, used to represent many-to-many membership relationships between topics and semantic blocks); this enables the expression of knowledge structures across chapters, topics, and multiple semantic affiliations in a document. The hypergraphs representing the relationships between topic clusters and semantic blocks (L0 topic hypergraph), semantic blocks and sentences (L1 semantic block hypergraph), and sentences and entities (L2 sentence hypergraph) are connected through cascading associations of topic-semantic block, semantic block-sentence, and sentence-entity, forming a multi-granularity nested holographic hypergraph index (multi-level semantic hypergraph); nodes in L0 are hyperedges of L1, and nodes in L1 are hyperedges of L2, thus forming a multi-level semantic hypergraph.

[0043] In the specific implementation, when constructing a semantic block relationship graph based on the similarity between the original vectors of semantic blocks, the semantic blocks in the semantic block relationship graph... With semantic blocks Similarity between edge weights It can be defined as: ; in, , For semantic blocks With semantic blocks The original vector, The similarity truncation threshold is determined empirically; Represents semantic blocks With semantic blocks The cosine similarity between them. When a community detection algorithm is used to obtain candidate topic clusters. Then, candidate topic clusters The topic center vector can be represented as: ; in, This indicates a normalization operation; For the absolute value operation, Represents candidate topic clusters The number of semantic blocks in the middle.

[0044] semantic blocks For candidate topic clusters Fuzzy membership degree It can be represented as: ; in, The preset temperature parameters are determined empirically. For candidate topic clusters The Middle The topic center vectors of candidate topic clusters.

[0045] when At that time, in the semantic block With candidate topic clusters Establish topic-cascading edges between them. The threshold for topic membership.

[0046] Cascaded attention index update: In this embodiment of the invention, the constructed multi-layer semantic hypergraph is as follows: Figure 2 As shown, at the semantic block level (corresponding to the topic level, reflecting the relationship between each topic cluster and its contained semantic blocks), the system inputs the original semantic block vector, the topic center vector corresponding to the semantic block, and the aggregate vector of the sentences contained in the semantic block into the dynamic self-attention fusion operator to obtain the concatenated semantic vector of the semantic block. This process enables the semantic block representation to simultaneously include topic constraints and sentence details.

[0047] At the sentence level (corresponding to the semantic block level, reflecting the relationship between each semantic block and the sentences it contains), the system inputs the original sentence vector, the concatenated semantic vector of its semantic block, and the aggregated vector of the entities contained in the sentence into a dynamic self-attention fusion operator to obtain the concatenated semantic vector of the sentence. This process enables the sentence representation to simultaneously perceive the context of its semantic block and key entities.

[0048] At the entity level (corresponding to the sentence level, reflecting the relationship between each sentence and its contained entities), the system inputs the original entity vector and the concatenated semantic vector of its sentence into a dynamic self-attention fusion operator to obtain the entity concatenated vector. This process ensures that entities are no longer just isolated terms, but carry the semantic role and context of the sentence in which they belong.

[0049] Specifically, semantic blocks The set of sentences contained is denoted as The aggregate vector of the sentences contained in this semantic block for: ; Among them, sentence attention weight It can be represented as: ; in, Represents semantic blocks Aggregate vectors containing sentences; Represents semantic blocks The original semantic vector; Sentence The original semantic vector.

[0050] The formula for semantic block cascading update is: ; in, Represents semantic blocks Cascaded semantic vectors; Represents semantic blocks Weighted topic representation of its associated candidate topic clusters; Represents semantic blocks Topic representation Fuzzy membership degree; represents the topic representation of the k-th candidate topic cluster; AttFuse represents the dynamic self-attention fusion operator, used to calculate the similarity between the query vector and one or more key vectors, and obtains the fusion weights through a temperature-parameter-controlled Softmax function, then weights and sums the key vectors and normalizes them to obtain the cascaded update vector, specifically: , , For anchor vector, The set of vectors to be merged. Let be the total number of vectors to be fused. Let x represent the x-th vector to be fused, then: ; ; in, Let y be the y-th vector in the set of vectors to be merged. , Anchor vector transpose, Let x be the attention weight for the x-th vector to be fused. This refers to the temperature parameter.

[0051] exist In the calculation, the anchor vector for The set of vectors to be fused for , , , , They are respectively , , The subsequent use of AttFuse for dynamic self-attention fusion is similar and will not be described further. In other embodiments, other self-attention fusion methods can also be used, such as using a transformer for dynamic self-attention fusion.

[0052] sentence The set of entities contained is denoted as ,sentence The aggregate vector of the included entities is: ; in, Sentence The aggregate vector containing the entities; Sentence The first in One entity; Representing entities The original semantic vector; Representing entities In the sentence Importance weights in the equation.

[0053] The formula for sentence cascading updates is: ; in, Sentence Cascaded semantic vectors; Sentence The original semantic vector; Sentence Belonging semantic block Cascaded semantic vectors; Sentence The semantic block to which it belongs. If the sentence When multiple semantic blocks exist, the resulting concatenated vectors of parent semantic blocks need to be averaged and normalized before sentence concatenation updates are performed. Specifically: Will Updated to: ; in, For sentences The set of semantic blocks to which it belongs. For sentences its semantic block In this embodiment of the invention, the weighting of the sentence node is set equally according to the number of parent semantic blocks associated with it, that is, the weighting of each parent semantic block is 1 / |C(j)|.

[0054] The entity cascading update formula is: ; in, Representing entities The sentence to which it belongs, Representing entities Sentence Cascaded semantic vectors.

[0055] Query-side symmetrical cascading retrieval: After the user enters a query (the question to be searched), the entire process is as follows: Figure 3 As shown, the query is first semantically expanded to form an expanded query text (query sentence). The query sentence is then input into an embedding model to obtain the corresponding query source vector (embedding vector). The system initializes the query sentence and extracts query entities to form a query entity set. The query sentence serves as both the source of the query semantic block and the source of the query sentence itself. The system uses the query source vector... With the central vector of each theme The similarity score yields a set of candidate topic clusters that match the query. ,Will Perform average aggregation and normalization to obtain the query topic aggregation vector. And will query the original vector and query topic aggregation vector The query semantic block vector is obtained through a dynamic self-attention fusion operator. .

[0056] Secondly, the system calculates the aggregated vector of query entities based on the set of query entities, and then converts the original query vector into a single vector. Query semantic block vector and query entity aggregation vector Input the dynamic self-attention fusion operator to obtain the query sentence vector. Then, the system uses the query sentence vector... Update the representation of query entities (query entity concatenation vector) This allows for more precise entity constraints to be reinforced during retrieval.

[0057] Specifically, in the candidate recall phase, the system employs a top-down cascading query approach to filter candidate evidence within the processed indexed knowledge base. First, based on the similarity between the original query vector and the topic center vectors, the system determines the matching topic clusters that correspond to the user's query. Then, it performs weighted aggregation of the topic center vectors of these matching topic clusters to obtain the query topic aggregation vector. Then, using semantic blocks associated with the hit topic clusters as the candidate range, based on the query semantic block vector... The relevance scores between the semantic vectors of each semantic block and the concatenated semantic vectors are ranked, and the semantic blocks with higher scores are selected as the candidate semantic block set, i.e., the set of semantic blocks that are matched by the query. Secondly, within the scope of the candidate semantic block set, the system determines the sentence nodes contained in the candidate semantic block set as the candidate sentence set. For each candidate sentence, the system simultaneously obtains the concatenated semantic vector of the semantic block to which the candidate sentence belongs. The concatenated semantic vector of the candidate sentence The candidate sentence contains concatenated semantic vectors of its entities, and semantic block-level relevance scores are calculated based on the relevant query representations. Sentence-level relevance score Entity-level relevance score Among them, entity-level relevance score By querying the entity concatenation vector The entity matching is performed with the concatenated vectors of entities contained in the candidate sentences to obtain the entity evidence relevance. Entity matching is used to evaluate the entity evidence relevance of the candidate sentences, without performing entity node recall or forming a separate candidate entity evidence set. Subsequently, the system inputs the semantic block-level relevance score, sentence-level relevance score, and entity-level relevance score into a dynamic multi-granularity scoring operator. The system adaptively generates the fusion weights of each granularity score through a Softmax function with a temperature parameter, and calculates the comprehensive retrieval score for each candidate sentence based on the fusion weights. The system sorts candidate sentences according to their comprehensive search scores, selects the top-k candidate sentences as first-hop evidence based on their scores, and expands the evidence through multiple hops based on the shared entity relationships in the first-hop evidence.

[0058] Specifically, the construction process of the query-side concatenated vector can be represented as follows: After the user enters a query, the system first processes the original query text. Semantic augmentation is performed to obtain the augmented query text. Vector embedding can be represented as: ; ; in, To query the original vector (embedded vector). For semantic augmentation operations, This is a vector embedding operation.

[0059] Query entity set Represented as: ; The query entity aggregation vector is represented as: ; in, Represents the aggregate vector of the query entity; Indicates the query entity Expanding the query text Importance weights in; Indicates the query entity The original entity vector (embedded vector).

[0060] Query the original vector With topic center vector The similarity is expressed as: ; The corresponding candidate topic set is represented as follows: ; in, This represents the set of candidate topics (the set of candidate topic clusters) that were matched by the query. This represents a relative threshold, an empirically determined value. Indicates querying the original vector The maximum similarity among the similarities with each topic center vector.

[0061] Based on the set of candidate topics matched by the query, the query topic aggregation vector is represented as follows: ; The query semantic block vector is represented as: ; The query sentence vector is represented as: ; The query entity concatenation vector is represented as: ; From top to bottom, find the corresponding topics, then find the corresponding semantic blocks based on the topics, and finally find the corresponding sentences based on the semantic blocks. The final score is: ; in: ; ; ; ; in, For semantic block-level relevance scores, Sentence-level relevance score, Entity-level relevance score; Entities contained in the candidate sentence Cascaded vectors; To query the set of concatenated vectors of entities; The fusion weights for semantic block-level relevance scores The fusion weights for sentence-level relevance scores The fusion weights for entity-level relevance scores; This refers to the temperature parameter.

[0062] Consistency verification and answer generation: The system calls the consistency verification model to determine whether candidate evidence can answer the user's query. If yes, the candidate sentence is used as evidence; otherwise, it is deleted, resulting in verified evidence (verified candidate sentences). The verified evidence is then rearranged according to its final score and source level (which semantic block it belongs to), and concatenated into a generation context as the content retrieved from external knowledge. The generation model (language model) outputs enhanced retrieval results based on this context and the segmentation results (a set of candidate semantic blocks), i.e., the answer to the query question. It can also return the citation source, matching entity, and candidate weights for subsequent manual review or system interpretability demonstration.

[0063] This invention constructs a multi-level semantic hypergraph through topic, semantic block, sentence, and entity layers, enabling the simultaneous modeling of macro-level topic information, semantic block context information, sentence factual information, and entity constraint information in the knowledge base, thereby reducing the semantic fragmentation problem caused by traditional fixed-slice methods. By using fuzzy overlapping topic relationships, a semantic block can be associated with multiple topics with different membership degrees, adapting to cross-topic, cross-chapter, and multi-semantic affiliation scenarios common in long documents and multi-source knowledge bases. Through index-side cascading updates and query-side symmetrical cascading, the semantic block-level, sentence-level, and entity-level representations of user queries can be aligned with the multi-level semantic structure of the index, thereby improving evidence recall capabilities in complex query scenarios. Through dynamic self-attention fusion operators and dynamic multi-granularity scoring operators, the system can automatically adjust the importance of semantic blocks, sentences, and entities based on the user's query intent. Through consistency checks and evidence rearrangement, irrelevant or conflicting evidence is reduced from entering the generation model, improving the traceability, consistency, and reliability of the generated results.

[0064] To verify the retrieval enhancement effect of the proposed method, an automatic scoring method based on a large language model was used to compare and evaluate the answers generated by different methods. The evaluation process used user questions, relevant reference documents, and the answers generated by each method as input. The scoring model scored the answers across six dimensions: Comprehensiveness, Diversity, Empowerment, Logicality, Readability, and Relevance, with scores ranging from 0 to 100. The meanings of each indicator are as follows.

[0065] Comprehensiveness: This measures whether the answer fully covers the key aspects of the question and whether any important information is omitted. A higher score indicates that the answer comprehensively covers the question's requirements.

[0066] Diversity: This measures whether the response provides sufficient information richness, including background knowledge, supplementary information, case studies, or extended analysis. A higher score indicates a richer information dimension in the response.

[0067] Empowerment (credibility and support): This measures the credibility of an answer and whether it can convince the reader that the answer is correct through relevant documents, factual evidence, or logical reasoning. A higher score indicates more sufficient evidence and stronger credibility.

[0068] Logical: This measures whether the answer is coherent, clear, logically structured, and easy to understand. A higher score indicates a more rigorous logical response and smoother expression.

[0069] Readability measures whether an answer is clearly formatted, well-organized, and user-friendly. A higher score indicates a more readable and logically structured answer.

[0070] Relevance: This measures whether the answer is highly relevant to the user's question and whether it directly helps solve the problem. A higher score indicates that the answer is more focused on the question itself, with less off-topic or irrelevant content.

[0071] During the scoring process, each indicator was divided into five levels: Level 1 corresponds to a score of 0-20, indicating poor performance; Level 2 corresponds to a score of 20-40, indicating limited effectiveness. Level 3 corresponds to a score of 40-60, indicating that the basic requirements are met; Level 4 corresponds to a score of 60-80, indicating a good result; Level 5 corresponds to a score of 80-100, indicating excellent results.

[0072] The scoring model assigns a grade and a specific score to each indicator and calculates the average score of the six indicators as the comprehensive evaluation result.

[0073] In the experiment, the method of this invention was compared with related retrieval enhancement generation methods such as Simple RAG, LLM Only, LightRAG, and HyperRAG. Each method generated answers under the same question set, the same reference documents, and the same generation model, and was scored according to the six indicators mentioned above. The experimental results are as follows: Figure 4 As shown, the method of this invention (Ours) achieved high scores in indicators such as comprehensiveness, credibility, logic, and relevance, indicating that through multi-level semantic hypergraph indexing, query-end cascading retrieval, dynamic multi-granularity scoring, multi-hop evidence retrieval, and consistency verification mechanisms, it can more fully recall evidence related to the question and generate a more complete, credible, and focused answer.

[0074] Example 2 This invention provides a retrieval enhancement generation system based on a multi-level semantic hypergraph, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of the retrieval enhancement generation method based on a multi-level semantic hypergraph in Embodiment 1 above.

[0075] The relevant technical solutions are the same as above, and will not be repeated here.

[0076] Example 3 This invention provides a computer-readable storage medium storing a computer program thereon. When the computer program is executed by a processor, it implements the steps of the retrieval enhancement generation method based on a multi-level semantic hypergraph in Embodiment 1 above.

[0077] Specifically, the memory may include high-speed random access memory, as well as non-volatile memory, such as hard disks, RAM, plug-in hard disks, smart media cards (SMC), secure digital (SD) cards, flash cards, at least one disk storage device, flash memory device, or other volatile solid-state storage devices.

[0078] The relevant technical solutions are the same as above, and will not be repeated here.

[0079] Example 4 This invention provides a computer program product, including a computer program that, when run on a computer, causes the computer to perform the steps of the retrieval enhancement generation method based on a multi-level semantic hypergraph in Embodiment 1 above.

[0080] The relevant technical solutions are the same as above, and will not be repeated here.

[0081] Those skilled in the art will readily understand that the above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A retrieval enhancement generation method based on a multi-level semantic hypergraph, characterized in that, include: The document to be indexed is cleaned and segmented into semantic blocks to obtain a set of semantic blocks. ; Each semantic block is processed into sentences to construct a hypergraph of relationships between semantic blocks and sentences (L1). Each sentence is processed into entities to construct a hypergraph of relationships between sentences and entities (L2). Based on the set of semantic blocks... A semantic block relationship graph is constructed based on the similarity between semantic blocks. The semantic block relationship graph is then divided into communities to obtain a set of candidate topic clusters. K is The total number of candidate topic clusters; calculate the candidate topic clusters. Semantic blocks other than candidate topic clusters The fuzzy membership degree is used to add semantic blocks with a fuzzy membership degree greater than a preset membership degree threshold to the candidate topic cluster. To construct a hypergraph L0 relating candidate topic clusters and semantic blocks; The relational hypergraphs L0, L1, and L2 are cascaded together to form a multi-level semantic hypergraph. Based on the multi-level semantic hypergraph, the embedding vector of each semantic block, the topic center vector of the candidate topic cluster to which the semantic block belongs, and the aggregation vector of the sentences contained in the semantic block are dynamically self-attention fused to obtain the concatenated semantic vector of the semantic block; the embedding vector of each sentence, the concatenated semantic vector of the semantic block to which the sentence belongs, and the aggregation vector of the entities contained in the sentence are dynamically self-attention fused to obtain the concatenated semantic vector of the sentence; the embedding vector of each entity and the concatenated semantic vector of the sentence to which the entity belongs are dynamically self-attention fused to obtain the concatenated semantic vector of the entity. The user-input query is semantically expanded to form query text. , query text Perform entity extraction to form a query entity set. Based on query text Embedded vector The set of candidate topic clusters that are matched by the query is obtained by comparing the similarity between the topic center vectors of each candidate topic cluster and the topic center vectors of each candidate topic cluster. ;Will query topic aggregation vector and Dynamic self-attention fusion is performed to obtain the query semantic block vector. ,use exist Obtain the set of semantic blocks matched by the query. ;Will query entity aggregation vector , and Dynamic self-attention fusion is performed to obtain the query sentence vector. ; Will The included sentences form a candidate sentence set. For each candidate sentence, a separate calculation is performed. The relevance score between the candidate sentence and the concatenated semantic vector of the semantic block to which the candidate sentence belongs. ,calculate Relevance score between the concatenated semantic vectors of the candidate sentence Calculate the concatenated vector of query entities The relevance score between the candidate sentence and the concatenated semantic vectors of the entities contained in the candidate sentence. , based on and The query entity embedding vectors are dynamically self-attention fused to obtain the relevance score; , , A weighted fusion is performed to obtain the comprehensive retrieval score for the candidate sentence. ;choose The top k candidate sentences with the highest resolution are selected as the query hit sentences; The required context is obtained based on the query hit sentence, so that the large model can output enhanced retrieval results based on the required context.

2. The retrieval enhancement generation method according to claim 1, characterized in that, use The operator performs the dynamic self-attention fusion, the The operator is: in, For anchor vector, The set of vectors to be merged. , Let be the total number of vectors to be fused. This represents the x-th vector to be fused. Let y be the vector to be fused. , Anchor vector transpose, for Attention weights For temperature parameters, This indicates a normalization operation; use When operators compute concatenated semantic vectors of semantic blocks, anchor vectors For semantic blocks Embedded vector , , For semantic blocks For candidate topic clusters Fuzzy membership degree, Candidate topic clusters Theme center vector, Represents semantic blocks The weighted sum of the topic center vectors of its candidate topic clusters. For semantic blocks The aggregate vector of the contained sentences; use When operators compute the concatenated semantic vector of a sentence, the anchor vector... For sentences Embedded vector , ,in, , For sentences To which the semantic block set belongs For sentences its semantic block The attribution weight, For sentences Belonging semantic block Cascaded semantic vectors For sentences The aggregate vector of the contained entities; use When operators compute the concatenated semantic vectors of entities, anchor vectors For entities Embedded vector , , For entities Sentence Cascaded semantic vectors; use Operator computation query semantic block vector At that time, anchor vector For query text Embedded vector , ; use Operators calculate query sentence vectors At that time, anchor vector For query text Embedded vector , ; use Operators calculate the concatenated vector of query entities At that time, anchor vector For querying the entity set Query entities Embedded vector , .

3. The retrieval enhancement generation method according to claim 2, characterized in that, The comprehensive search score for: in, This represents the concatenated semantic vector of the semantic block to which the candidate sentence belongs. This represents the concatenated semantic vectors of the candidate sentences. Entities contained in the candidate sentence Cascaded semantic vectors; For sentences The set of entities contained in a sentence Represents the current candidate sentence; To query the set of concatenated vectors of entities; , , They are respectively , , The corresponding fusion weights; This refers to the temperature parameter.

4. The retrieval enhancement generation method according to any one of claims 1-3, characterized in that, The fuzzy membership degree is: in, For semantic blocks For candidate topic clusters Fuzzy membership degree, For semantic blocks Embedded vector, Candidate topic clusters Theme center vector, The preset temperature parameters; For candidate topic clusters The Middle Candidate topic clusters The theme center vector; This indicates a normalization operation; This is for taking the absolute value.

5. The retrieval enhancement generation method according to claim 4, characterized in that, The aggregation vector of the sentences contained in the semantic block is: in, For semantic blocks The aggregate vector of the contained sentences, For semantic blocks The set of sentences contained therein For sentences Embedded vector, For sentences Attention weights For semantic blocks Embedded vector, This indicates a normalization operation; The aggregate vector of the entities contained in the sentence is: in, Sentence The aggregation direction of the contained entities; Sentence The first in One entity, For sentences The set of entities contained therein Representing entities Embedded vector, Representing entities In the sentence Importance weights in; query entity aggregation vector for: in, express Query entities In the query text Importance weights in; Indicates the query entity The embedding vector; query topic aggregation vector for: in, For query text Embedded vector With candidate topic clusters Topic center vector The similarity.

6. The retrieval enhancement generation method according to any one of claims 1-3, characterized in that, Candidate topic cluster set for: ;in, For query text Embedded vector With candidate topic clusters Topic center vector similarity, Represents the query text Embedded vector The maximum similarity between the topic center vectors of each candidate topic cluster and the topic center vector of each candidate topic cluster. This represents a relative threshold.

7. The retrieval enhancement generation method according to any one of claims 1-3, characterized in that, use exist Obtain the set of semantic blocks matched by the query. ,include: With the set of candidate topic clusters matched by the query The associated semantic blocks are used as candidate ranges to calculate the query semantic block vector. The relevance scores between the semantic vectors of each semantic block within the candidate range and the concatenated semantic vectors are ranked, and semantic blocks with scores exceeding a preset threshold are selected to form the set of semantic blocks matched by the query. ; The required context is obtained based on the query-matched sentence, including: The query-matched sentence is used as the first-hop evidence. Based on the shared entities in the first-hop evidence, multiple hops are performed to obtain related sentences. The query-matched sentence and the obtained related sentences are used as a candidate evidence set. A consistency check is performed on the candidate evidence in the candidate evidence set to determine whether the candidate evidence can answer the user's input query. Candidate evidence that can answer the query is then ranked according to the comprehensive search score. The semantic blocks to which the candidate evidence belongs are rearranged and spliced ​​together to form the required context.

8. A retrieval enhancement generation system based on a multi-level semantic hypergraph, characterized in that, Includes computer-readable storage media and processors; The computer-readable storage medium is used to store executable instructions; The processor is used to read executable instructions stored in the computer-readable storage medium and execute the retrieval enhancement generation method based on a multi-level semantic hypergraph as described in any one of claims 1-7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the retrieval enhancement generation method based on a multi-level semantic hypergraph as described in any one of claims 1-7.

10. A computer program product, characterized in that, Includes a computer program that, when run on a computer, causes the computer to perform the retrieval enhancement generation method based on a multi-level semantic hypergraph as described in any one of claims 1-7.

Images