Retrieval enhancement generation method and system based on sparse graph neural network

Abstract

The invention discloses a retrieval enhancement generation method and system based on a sparse graph neural network, and aims to solve the problems of large-scale knowledge graph retrieval enhancement generation and a multi-hop reasoning task. The method comprises the following steps: embedding a query vector, receiving a user query, and encoding the user query into a query embedded vector through a sentence embedding model; performing sub-graph retrieval, performing hash processing on the query embedded vector by using a hierarchical hash index to generate a composite hash key, and matching entities in the knowledge graph layer by layer according to the queried hash key to construct a sub-graph; dynamic sparse message transmission: executing dynamic sparse message transmission update entity embedding on the constructed subgraph; document retrieval sorting is carried out, document correlation scores are calculated according to final entity embedding, and first K documents are selected to form a retrieval result document set; and answer generation: inputting the query and retrieval result document set into a generation model, and outputting a final answer. The method is suitable for large-scale knowledge graph application, and an efficient retrieval enhancement generation solution is provided.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence and natural language processing technology, and in particular to a retrieval enhancement generation method and system based on a sparse graph neural network. Background Art

[0002] As knowledge graphs continue to expand in size and complexity, traditional retrieval-augmented generation (RAG) methods face challenges with high computational complexity and resource consumption when processing large-scale graphs. While existing technologies (such as GraphRAG) have improved retrieval efficiency and accuracy to a certain extent, they still lack sparsity and real-time performance. Therefore, a more efficient RAG method is needed to meet the needs of large-scale knowledge graphs and real-time applications. Summary of the Invention

[0003] In view of this, in order to solve the above-mentioned problems in the prior art, the present invention proposes a retrieval enhancement generation method and system based on sparse graph neural networks, which realizes efficient retrieval and multi-hop reasoning generation tasks of large-scale knowledge graphs through hierarchical hash indexing and dynamic sparse message passing technology.

[0004] The present invention solves the above problems through the following technical means:

[0005] In one aspect, the present invention provides a retrieval enhancement generation method based on a sparse graph neural network, comprising the following steps:

[0006] S1. Query vector embedding: Receive user queries and encode them into query embedding vectors through sentence embedding models;

[0007] S2. Subgraph retrieval: Use hierarchical hash indexing to hash the query embedding vector to generate a composite hash key, and then match entities in the knowledge graph layer by layer based on the query hash key to construct a subgraph;

[0008] S3. Dynamic Sparse Message Passing: Based on the query-dependent sparse graph neural network, dynamic sparse message passing is performed on the constructed subgraph to output the entity weight vector;

[0009] S4, document retrieval ranking: Calculate the document relevance score based on the entity weight vector, and select the top K documents to form the retrieval result document set;

[0010] S5. Answer generation: Input the query and retrieval result document set into the generation model and output the final answer.

[0011] Preferably, in step S2, the hierarchical hash index includes three levels of hashing:

[0012] (1) Original embedding hash: embed the entity h e Execute k1 bit locality sensitive hashing LSH to generate hash key

[0013] (2) Dimensionality reduction embedding hash: embed the entity h e After performing principal component analysis (PCA) dimensionality reduction, perform k2-bit LSH to generate a hash key.

[0014] (3) Cluster center embedding hash: embedding h into entity e Perform K-means clustering and generate cluster center hash keys

[0015] Among them, k1>k2>k3, the high-level hash key is used for fast coarse screening, and the low-level hash key is used for fine matching.

[0016] Preferably, in step S3, dynamic sparse message passing is performed on the constructed subgraph to output the entity weight vector, including calculating sparse attention scores, dynamic pruning based on path information entropy, and entity weight vectors.

[0017] Preferably, the calculation formula of the sparse attention score is:

[0018]

[0019] Among them, LSH k is a k-bit hash function, is the sparse neighbor set of entity e at layer l; e' is the neighbor entity of entity e, represents the sparse neighbor set of entity e in layer l, and e″ represents Any neighbor entity in represents the embedding vector of entity e in layer l, represents the embedding vector of entity e' in layer l, represents the embedding vector of entity e″ in layer l;

[0020] The calculation formula for the dynamic pruning based on path information entropy is:

[0021]

[0022] in, is the sparse neighbor set of entity e at layer l, e' is the neighbor set of entity e, represents the sparse neighbor set of entity e in the lth layer, Entropy(P(e'|q)) is the path information entropy, which is used to measure the contribution of neighbor entities to the query, τ l is the adaptive entropy threshold of the lth layer, which decreases as the number of layers increases.

[0023] Preferably, the calculation formula of the entity weight vector is:

[0024]

[0025] in, is the sparse neighbor set of entity e at layer l, is the sparse attention score, Represents a message from neighbor entity e' to entity e.

[0026] Preferably, step S3 further includes model training of the sparse graph neural network as follows:

[0027] A two-stage training approach is adopted. The first stage uses unsupervised knowledge graph completion pre-training, and the second stage uses supervised document retrieval fine-tuning. The unsupervised knowledge graph completion pre-training is pre-trained on the large-scale knowledge graph completion task, creating synthetic queries by masking entities and predicting masked entities. The second stage uses supervised document retrieval fine-tuning, which is fine-tuned on the labeled document retrieval task, using natural language questions as queries to extract target entities from annotated supporting documents.

[0028] Preferably, the sparse graph neural network training loss function is a weighted combination of binary cross entropy loss, sparse consistency loss and hash contrast learning loss, and the calculation formula is as follows:

[0029]

[0030] Among them, α, β, and γ are weight parameters used to balance the influence of different loss functions;

[0031] Binary cross entropy loss, which is calculated as:

[0032]

[0033] Among them, A q is the set of positive entities related to the query q, E is the set of negative entities irrelevant to the query q, and P q is the probability that the model predicts that entity e is relevant to query q;

[0034] The sparse consistency loss regularizes the gradient direction by L1, making most attention weights approach zero and passing information through a small number of key paths. Its calculation formula is:

[0035]

[0036] in is the binary cross entropy loss, l represents the number of layers of the graph network, α l represents the sparse attention score of entity e in layer l;

[0037] Hash contrast learning loss, calculated as:

[0038]

[0039] By optimizing the hash function through contrastive learning, we ensure that the hash keys of similar entities are close in the Hamming space, where e + is a positive sample, e - is a negative sample, h e represents the embedding vector of entity e, Represents the positive sample entity e + The embedding direction, Represents the negative sample entity e - The embedding vector of .

[0040] As an example, in step S4, the entity-document inverted index matrix M and the entity weight vector F are used. e To calculate the document relevance score, the calculation formula is as follows:

[0041]

[0042] Among them, M is the entity-document inverted index matrix, F e The entity weight vector is generated by the Dynamic Sparse Message Passing (DSMP) module.

[0043] Preferably, the method further comprises:

[0044] S6. Incremental graph update mechanism: This mechanism supports real-time updates of hash indexes and model parameters when the knowledge graph changes dynamically, ensuring real-time and accurate retrieval. The incremental graph update mechanism is as follows:

[0045] (1) New entity processing

[0046] The embedding vector of the newly added entity is calculated using the same sentence embedding model as the query encoding. The embedding vector of the newly added entity is hashed using the hierarchical hash index (HLSH) to generate a composite hash key. The newly added entity and its hash key are inserted into the HLSH index to ensure that subsequent searches can quickly locate the entity.

[0047] (2) Local parameter fine-tuning

[0048] Only the affected graph model is fine-tuned, and relevant parameters are updated to adapt to the information of the newly added entities. The same loss function as the original training is used during the fine-tuning process to avoid retraining the entire model.

[0049] (3) Index synchronization

[0050] The embedding and hash key of the newly added entity are broadcast to all parameter servers. Each parameter server asynchronously updates the inverted index matrix it maintains to ensure that the information of the newly added entity is included in the search scope in a timely manner. The index consistency on each parameter server is regularly checked to ensure the accuracy of the search in a distributed environment.

[0051] In another aspect, the present invention provides a retrieval enhancement generation system based on a sparse graph neural network, comprising:

[0052] The vector embedding module is used to convert the user's natural language query into a query embedding vector, extract key features, and provide a basis for subsequent subgraph retrieval and entity matching;

[0053] A hierarchical hash index construction module is used to hash the query embedding vector, generate a composite hash key, and match the entities in the knowledge graph layer by layer. It also builds a subgraph based on the query hash key.

[0054] Dynamic sparse message passing module, used for query-dependent sparse graph neural networks, captures the semantic association between queries and knowledge graphs through sparse attention scores and dynamic pruning based on path information entropy, and outputs entity weight vectors;

[0055] Document retrieval and ranking module: used to calculate the document relevance score using the entity weight vector and inverted index matrix, perform weighted sorting on the documents according to the entity weight vector, and take the top K documents as the retrieval result document set;

[0056] Answer generation module: used to input the query and retrieval result document set into the generation model and output the final answer.

[0057] Compared with the prior art, the beneficial effects of the present invention include at least:

[0058] This paper presents a sparse graph neural network-based retrieval enhancement generation method and system based on a large language model. Through dynamic sparse message passing (DSMP), dynamic path pruning, and layer-by-layer update mechanisms, it significantly reduces computational complexity while retaining multi-hop reasoning capabilities. Its adaptive adjustment mechanism and theoretical guarantees ensure high efficiency and accuracy on large-scale graphs. It supports distributed storage and retrieval, making it suitable for applications such as large-scale graphs and real-time reasoning. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0060] Figure 1 This is a flow chart of the retrieval enhancement generation method based on sparse graph neural network of the present invention;

[0061] Figure 2 This is a structural block diagram of the retrieval enhancement generation system based on sparse graph neural network of the present invention. DETAILED DESCRIPTION

[0062] To make the above-mentioned objectives, features, and advantages of the present invention more clearly understood, the technical solutions of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that the embodiments described are only some of the embodiments of the present invention, not all of them. All other embodiments obtained by persons of ordinary skill in the art based on the embodiments of the present invention without creative effort are also within the scope of protection of the present invention.

[0063] References herein to "embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0064] Example 1

[0065] like Figure 1 As shown, an embodiment of the present invention provides a retrieval enhancement generation method based on a sparse graph neural network. The method optimizes the message passing process of the graph neural network through dynamic sparse message passing and hierarchical hash indexing, reduces computational overhead, and maintains multi-hop reasoning capabilities.

[0066] The present invention provides a retrieval enhancement generation method based on a sparse graph neural network, comprising the following steps:

[0067] S1. Query vector embedding: Receive user queries and encode them into query embedding vectors through sentence embedding models;

[0068] S2. Subgraph retrieval: Use hierarchical hash indexing to hash the query embedding vector to generate a composite hash key, and then match entities in the knowledge graph layer by layer based on the query hash key to construct a subgraph;

[0069] S3. Dynamic Sparse Message Passing: Based on the query-dependent sparse graph neural network, dynamic sparse message passing is performed on the constructed subgraph to output the entity weight vector;

[0070] S4, document retrieval ranking: Calculate the document relevance score based on the entity weight vector, and select the top K documents to form the retrieval result document set;

[0071] S5. Answer generation: Input the query and retrieval result document set into the generation model and output the final answer.

[0072] Specifically, query vector embedding is to input the preprocessed query into the sentence embedding model to generate an embedding vector h of fixed dimension. q Entities are extracted from the query through the Open Information Extraction (OpenIE, LLM) tool, which will be used in the subsequent subgraph retrieval and dynamic sparse message passing steps to ensure that the retrieval and reasoning processes can efficiently utilize the semantic information of the query.

[0073] Subgraph retrieval uses a hierarchical hash index to hash the query embedding vector, generate a composite hash key, and match entities in the knowledge graph layer by layer based on the query hash key to construct a subgraph.

[0074] Specifically, in step S2, the hierarchical hash index includes three levels of hashing:

[0075] (1) Original embedding hash: embed the entity h e Perform k1-bit locality sensitive hashing (LSH) to generate a hash key

[0076] (2) Dimensionality reduction embedding hash: embed the entity h e After performing principal component analysis (PCA) dimensionality reduction, perform k2-bit LSH to generate hash keys

[0077] (3) Cluster center embedding hash: embedding h into entity e Perform K-means clustering and generate cluster center hash keys

[0078] Among them, k1>k2>k3, the high-level hash key is used for fast coarse screening, and the low-level hash key is used for fine matching.

[0079] Specifically, in step S3, dynamic sparse message passing is performed on the constructed subgraph to output the entity weight vector, including calculating the sparse attention score, dynamic pruning based on path information entropy, and the entity weight vector.

[0080] Specifically, the calculation formula of the sparse attention score is:

[0081]

[0082] Among them, LSH k is a k-bit hash function, is the sparse neighbor set of entity e at layer l; e' is the neighbor entity of entity e, represents the sparse neighbor set of entity e in layer l, and e″ represents Any neighbor entity in represents the embedding vector of entity e in layer l, represents the embedding vector of entity e' in layer l, represents the embedding vector of entity e″ in layer l;

[0083] The calculation formula for the dynamic pruning based on path information entropy is:

[0084]

[0085] in, is the sparse neighbor set of entity e at layer l, e' is the neighbor set of entity e, represents the sparse neighbor set of entity e in the lth layer, Entropy(P(e'|q)) is the path information entropy, which is used to measure the contribution of neighbor entities to the query, τ l is the adaptive entropy threshold of the lth layer, which decreases as the number of layers increases.

[0086] Specifically, the calculation formula of the entity weight vector is:

[0087]

[0088] in, is the sparse neighbor set of entity e at layer l, is the sparse attention score, Represents a message from neighbor entity e' to entity e.

[0089] Specifically, step S3 also includes model training of the sparse graph neural network as follows:

[0090] A two-stage training process is employed: the first stage uses unsupervised knowledge graph completion pre-training, and the second stage uses supervised document retrieval fine-tuning. Unsupervised knowledge graph completion pre-training is pre-trained on a large-scale knowledge graph completion task, creating synthetic queries by masking entities and predicting the masked entities. This step helps enhance the model's understanding and reasoning capabilities of the knowledge graph structure. The second stage uses supervised document retrieval fine-tuning, fine-tuning on a labeled document retrieval task, using natural language questions as queries and extracting target entities from annotated supporting documents. This step enables the model to better understand user queries and capture the relevance between questions and knowledge, thereby improving retrieval quality.

[0091] Specifically, the sparse graph neural network training loss function is a weighted combination of binary cross entropy loss, sparse consistency loss, and hash contrastive learning loss, and the calculation formula is as follows:

[0092]

[0093] Among them, α, β, and γ are weight parameters used to balance the influence of different loss functions;

[0094] Binary cross entropy loss, which is calculated as:

[0095]

[0096] Among them, A q is the set of positive entities related to the query q, E is the set of negative entities irrelevant to the query q, and P q is the probability that the model predicts that entity e is relevant to query q;

[0097] The sparse consistency loss regularizes the gradient direction by L1, making most attention weights approach zero and passing information through a small number of key paths. Its calculation formula is:

[0098]

[0099] in is the binary cross entropy loss, l represents the number of layers of the graph network, α l represents the sparse attention score of entity e in layer l;

[0100] Hash contrast learning loss, calculated as:

[0101]

[0102] By optimizing the hash function through contrastive learning, we ensure that the hash keys of similar entities are close in the Hamming space, where e + is a positive sample, e - is a negative sample, h e represents the embedding vector of entity e, Represents the positive sample entity e + The embedding direction, Represents the negative sample entity e - The embedding vector of .

[0103] Specifically, the document ranking is to calculate the document relevance score based on the final entity weight vector, and select the top K documents to form the retrieval result document set. In step S4, the entity-document inverted index matrix M and the entity weight vector F are used to calculate the document relevance score. e To calculate the document relevance score, the calculation formula is as follows:

[0104]

[0105] Among them, M is the entity-document inverted index matrix, F e The entity weight vector is generated by the Dynamic Sparse Message Passing (DSMP) module.

[0106] Specifically, the method further includes:

[0107] S6. Incremental graph update mechanism: This mechanism supports real-time updates of hash indexes and model parameters when the knowledge graph changes dynamically, ensuring real-time and accurate retrieval. The incremental graph update mechanism is as follows:

[0108] (1) New entity processing

[0109] The embedding vector of the newly added entity is calculated using the same sentence embedding model as the query encoding. The embedding vector of the newly added entity is hashed using the Hierarchical Hash Index (HLSH) to generate a composite hash key. The newly added entity and its hash key are inserted into the HLSH index to ensure that subsequent searches can quickly locate the entity.

[0110] (2) Local parameter fine-tuning

[0111] Only the affected graph model is fine-tuned, and relevant parameters are updated to adapt to the information of the newly added entities. During the fine-tuning process, the same loss function as the original training (such as sparse consistency loss and hash contrast learning loss) is used to avoid retraining the entire model.

[0112] (3) Index synchronization

[0113] The embedding and hash key of the newly added entity are broadcast to all parameter servers. Each parameter server asynchronously updates the inverted index matrix it maintains to ensure that the information of the newly added entity is included in the search scope in a timely manner. The index consistency on each parameter server is regularly checked to ensure the accuracy of the search in a distributed environment.

[0114] In order to better demonstrate this solution, the present invention provides the following embodiments in combination with specific application scenarios, and describes in detail the retrieval enhancement generation method based on a sparse graph neural network based on a large language model provided by the present invention.

[0115] It is constructed using open source knowledge graphs, including a medical literature corpus (1 million papers, PubMed dataset), open source biomedical knowledge graphs (such as UMLS, DrugBank), including drugs, genes, disease entities and relationships.

[0116] The processing flow is as follows:

[0117] (1) Entity and relationship extraction

[0118] Use an open information extraction tool driven by LLM (such as REBEL or GPT-4o-mini) to extract triples (head entity-relationship-tail entity) from the literature, such as `(Aspirin, inhibitors, COX-2)`. Add semantically equivalent edges to entities with the same name (such as "COX-2" and "PTGS2"), and generate a global knowledge graph G (node ​​number |ε| = 5M, edge number 5M) after disambiguation. ).

[0119] (2) Hierarchical Hash Index (HLSH) Construction

[0120] Generate entity embeddings using the pre-trained PubMedBERT model Using three-level hash projection, the first level hash projection is the original embedding h e Execute 64-bit LSH to generate hash key LSH 64 (h e The second level hash projection is to use PCA to reduce the dimension to 32 dimensions and then perform 32-bit LSH to generate LSH 32 (PCA(h e The second level hash projection is to use K-means clustering (number of clusters = 256) and then perform 16-bit LSH on the cluster centers to generate LSH 16 (Cluster(h e )). Combine the three-level hash keys into HLSH(h e ), and build an inverted index matrix Records the occurrences of an entity in a document.

[0121] Semantic equivalence relationship expansion, through entity resolution and disambiguation technology, adds semantic equivalence edges for entities with the same name (such as "COX-2" and "PTGS2").

[0122] (3) Model training

[0123] A two-stage training process is employed: the first stage uses unsupervised knowledge graph completion pre-training, and the second stage uses supervised document retrieval fine-tuning. Unsupervised knowledge graph completion pre-training is pre-trained on a large-scale knowledge graph completion task, creating synthetic queries by masking entities and predicting the masked entities. This step helps enhance the model's understanding and reasoning capabilities of the knowledge graph structure. The second stage uses supervised document retrieval fine-tuning, fine-tuning on a labeled document retrieval task, using natural language questions as queries and extracting target entities from annotated supporting documents. This step enables the model to better understand user queries and capture the relevance between questions and knowledge, thereby improving retrieval quality.

[0124] a) Self-supervised pre-training stage

[0125] The unsupervised pre-training dataset uses the open source biomedical knowledge graph (14 million triples) to generate synthetic queries such as (?,r,e')(e,r,?).

[0126] The model architecture uses a four-layer graph neural network (GNN), with each layer performing entropy-driven pruning and sparse attention. The pruning strength is dynamically adjusted based on a set threshold. LSH sparse attention is calculated using bucketing, retaining only the top-20% edges within the same bucket. Target mask entity prediction is optimized by minimizing the cross-entropy loss. Sparse consistency constraints are implemented using layer-wise gradient constraints to keep most attention weights close to zero. The AdamW optimizer is used.

[0127] b) Supervised pre-training phase

[0128] Supervised pre-training dataset: uses 500,000 question-document question-answer pairs across medical fields.

[0129] For example, consider the query "Which genes does aspirin affect inflammatory response?" and a dataset of medical papers containing annotated documents containing entities such as "COX-2," "PTGS2," and "NF-κB."

[0130] The training loss function is a weighted combination of binary cross entropy loss, sparse consistency loss, and hash contrastive learning loss, and is calculated as follows:

[0131]

[0132] Among them, α, β, and γ are weight parameters used to balance the influence of different loss functions.

[0133] Specifically, the binary cross entropy loss is calculated as:

[0134]

[0135] Among them, A q P is the set of positive entities related to query q, and E is the set of negative entities irrelevant to query q. q is the model-predicted probability that entity e is relevant to query q.

[0136] Specifically, the sparse consistency loss regularizes the gradient direction by L1, making most attention weights approach zero and passing information through a small number of key paths. Its calculation formula is:

[0137]

[0138] Specifically, the hash contrast learning loss is calculated as:

[0139]

[0140] By optimizing the hash function through contrastive learning, we ensure that the hash keys of similar entities are close in the Hamming space, where e + is a positive sample, e - is a negative sample.

[0141] (4) Graph retrieval reasoning

[0142] Enter the query: "How does aspirin affect PTGS2 gene expression through methylation modification?"

[0143] The reasoning process is as follows:

[0144] a) Subgraph retrieval

[0145] Use the HLSH index to match the third-level hash key and retrieve the subgraph G. sub (about 1000 nodes), covering entities such as "Aspirin", "PTGS2", and "DNAMethylation".

[0146] b) Dynamic Sparse Message Passing

[0147] Information entropy dynamic pruning: Calculate information entropy dynamic pruning, threshold τ l It decreases from 0.7 (layer 1) to 0.3 (layer 4). LSH sparse attention uses bucketing to calculate attention weights, retaining only the top-20% edges in the same bucket.

[0148] Output entity weight P q , related entities include “PTGS2” (weight 0.92), “COX-2” (0.85), and “NF-κB” (0.78).

[0149] c) Document Sorting

[0150] Entity weight F e Calculate the document score by IDF weighting (suppressing high-frequency entities such as "Gene"). Select the first K=5 documents D K , all focused on the "aspirin-epigenetic regulation" mechanism.

[0151] (5) Answer generation and verification

[0152] The question and the retrieved documents are fed into a large language model, which generates answers, such as:

[0153] "Aspirin inhibits DNA methyltransferase (DNMT) activity and reduces the methylation level of the PTGS2 gene promoter region, thereby upregulating its expression and reducing the inflammatory response."

[0154] Rethinking Mark Verification: Checking if "DNMT" is in D KIf not found, expand the HLSH layer number to search again and generate the correct answer.

[0155] (6) Interpretability Analysis

[0156] The model's reasoning process is explained through path importance analysis, ensuring the transparency and explainability of multi-hop reasoning.

[0157] Through the above steps, the present invention can achieve efficient knowledge retrieval and answer generation in multi-domain question-answering tasks, while supporting zero-sample migration and performance monitoring.

[0158] Example 2

[0159] like Figure 2 As shown, the present invention provides a retrieval enhancement generation system based on a sparse graph neural network, comprising a vector embedding module 300, a hierarchical hash index construction module 301, a dynamic sparse message passing module 302, a document retrieval ranking module 303 and an answer generation module 304;

[0160] The vector embedding module 300 is used to convert the user's natural language query into a query embedding vector, extract key features, and provide a basis for subsequent subgraph retrieval and entity matching;

[0161] The hierarchical hash index construction module 301 is used to perform hash processing on the query embedding vector, generate a composite hash key, and match the entities in the knowledge graph layer by layer, and build a subgraph based on the query hash key.

[0162] The dynamic sparse message passing module 302 is used for a query-dependent sparse graph neural network, which captures the semantic association between the query and the knowledge graph through sparse attention scores and dynamic pruning based on path information entropy, and outputs an entity weight vector;

[0163] The document retrieval and ranking module 303 is used to calculate the document relevance score using the entity weight vector and the inverted index matrix, perform weighted ranking on the documents according to the entity weight vector, and use the top K documents as the retrieval result document set;

[0164] The answer generation module 304 is used to input the query and the retrieval result document set into the generation model (LLM) and output the final answer.

[0165] The other features of this embodiment are the same as those of embodiment 1, so they will not be repeated here.

[0166] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

[0167] The above-described embodiments merely illustrate several implementations of the present invention, and while their descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art would be able to make numerous variations and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be determined by the appended claims.

Claims

1. A retrieval enhancement generation method based on sparse graph neural network, characterized in that: The steps include: S1. Query vector embedding: Receive user queries and encode them into query embedding vectors through sentence embedding models; S2. Subgraph retrieval: Use hierarchical hash indexing to hash the query embedding vector to generate a composite hash key, and then match entities in the knowledge graph layer by layer based on the query hash key to construct a subgraph; S3. Dynamic Sparse Message Passing: Based on the query-dependent sparse graph neural network, dynamic sparse message passing is performed on the constructed subgraph to output the entity weight vector; S4, document retrieval ranking: Calculate the document relevance score based on the entity weight vector, and select the top K documents to form the retrieval result document set; S5. Answer generation: Input the query and retrieval result document set into the generation model and output the final answer.

2. The retrieval enhancement generation method based on sparse graph neural network according to claim 1 is characterized in that In step S2, the hierarchical hash index includes three levels of hashing: (1) Original embedding hash: embed the entity h e Execute k1 bit locality sensitive hashing LSH to generate hash key (2) Dimensionality reduction embedding hash: embed the entity h e After performing principal component analysis (PCA) dimensionality reduction, perform k2-bit LSH to generate a hash key. (3) Cluster center embedding hash: embedding h into entity e Perform K-means clustering and generate cluster center hash keys Among them, k1>k2>k3, the high-level hash key is used for fast coarse screening, and the low-level hash key is used for fine matching.

3. The retrieval enhancement generation method based on sparse graph neural network according to claim 1 is characterized in that In step S3, dynamic sparse message passing is performed on the constructed subgraph to output the entity weight vector, including calculating the sparse attention score, dynamic pruning based on path information entropy, and the entity weight vector.

4. The retrieval enhancement generation method based on sparse graph neural network according to claim 3 is characterized in that The calculation formula of the sparse attention score is: Among them, LSH k is a k-bit hash function, is the sparse neighbor set of entity e at layer l; e' is the neighbor entity of entity e, represents the sparse neighbor set of entity e in layer l, and e″ represents Any neighbor entity in represents the embedding vector of entity e in layer l, represents the embedding vector of entity e' in layer l, represents the embedding vector of entity e″ in layer l; The calculation formula for the dynamic pruning based on path information entropy is: in, is the sparse neighbor set of entity e at layer l, e' is the neighbor set of entity e, represents the sparse neighbor set of entity e in the lth layer, Entropy(P(e'|q)) is the path information entropy, which is used to measure the contribution of neighbor entities to the query, τ l is the adaptive entropy threshold of the lth layer, which decreases as the number of layers increases.

5. The retrieval enhancement generation method based on sparse graph neural network according to claim 3 is characterized in that The calculation formula of the entity weight vector is: in, is the sparse neighbor set of entity e at layer l, is the sparse attention score, Represents a message from neighbor entity e' to entity e.

6. The retrieval enhancement generation method based on sparse graph neural network according to claim 1 is characterized in that Step S3 also includes model training of the sparse graph neural network as follows: A two-stage training approach is used: the first stage uses unsupervised knowledge graph completion pre-training, and the second stage uses supervised document retrieval fine-tuning; the unsupervised knowledge graph completion pre-training is pre-trained on a large-scale knowledge graph completion task, creating synthetic queries by masking entities and predicting the masked entities; The second stage uses supervised document retrieval fine-tuning on the labeled document retrieval task, using natural language questions as queries to extract target entities from annotated supporting documents.

7. The retrieval enhancement generation method based on sparse graph neural network according to claim 1 is characterized in that The sparse graph neural network training loss function is a weighted combination of binary cross entropy loss, sparse consistency loss, and hash contrastive learning loss, and is calculated as follows: Among them, α, β, and γ are weight parameters used to balance the influence of different loss functions; Binary cross entropy loss, which is calculated as: Among them, A q is the set of positive entities related to the query q, E is the set of negative entities irrelevant to the query q, and P q is the probability that the model predicts that entity e is relevant to query q; The sparse consistency loss regularizes the gradient direction by L1, making most attention weights approach zero and passing information through a small number of key paths. Its calculation formula is: in is the binary cross entropy loss, l represents the number of layers of the graph network, α l represents the sparse attention score of entity e in layer l; Hash contrast learning loss, calculated as: By optimizing the hash function through contrastive learning, we ensure that the hash keys of similar entities are close in the Hamming space, where e + is a positive sample, e - is a negative sample, h e represents the embedding vector of entity e, Represents the positive sample entity e + The embedding direction, Represents the negative sample entity e - The embedding vector of .

8. The retrieval enhancement generation method based on sparse graph neural network according to claim 1 is characterized in that: In step S4, the entity-document inverted index matrix M and the entity weight vector F e To calculate the document relevance score, the calculation formula is as follows: Among them, M is the entity-document inverted index matrix, F e The entity weight vector is generated by the Dynamic Sparse Message Passing (DSMP) module.

9. The retrieval enhancement generation method based on sparse graph neural network according to claim 1 is characterized in that: The method further comprises: S6. Incremental graph update mechanism: This mechanism supports real-time updates of hash indexes and model parameters when the knowledge graph changes dynamically, ensuring real-time and accurate retrieval. The incremental graph update mechanism is as follows: (1) New entity processing The embedding vector of the newly added entity is calculated using the same sentence embedding model as the query encoding. The embedding vector of the newly added entity is hashed using the hierarchical hash index (HLSH) to generate a composite hash key. The newly added entity and its hash key are inserted into the HLSH index to ensure that subsequent searches can quickly locate the entity. (2) Local parameter fine-tuning Only the affected graph model is fine-tuned, and relevant parameters are updated to adapt to the information of the newly added entities. The same loss function as the original training is used during the fine-tuning process to avoid retraining the entire model. (3) Index synchronization The embedding and hash key of the newly added entity are broadcast to all parameter servers. Each parameter server asynchronously updates the inverted index matrix it maintains to ensure that the information of the newly added entity is included in the search scope in a timely manner. The index consistency on each parameter server is regularly checked to ensure the accuracy of the search in a distributed environment.

10. A retrieval enhancement generation system based on sparse graph neural network, characterized in that: include: The vector embedding module is used to convert the user's natural language query into a query embedding vector, extract key features, and provide a basis for subsequent subgraph retrieval and entity matching; A hierarchical hash index construction module is used to hash the query embedding vector, generate a composite hash key, and match the entities in the knowledge graph layer by layer. It also builds a subgraph based on the query hash key. Dynamic sparse message passing module, used for query-dependent sparse graph neural networks, captures the semantic association between queries and knowledge graphs through sparse attention scores and dynamic pruning based on path information entropy, and outputs entity weight vectors; Document retrieval and ranking module: used to calculate the document relevance score using the entity weight vector and inverted index matrix, perform weighted sorting on the documents according to the entity weight vector, and take the top K documents as the retrieval result document set; Answer generation module: used to input the query and retrieval result document set into the generation model and output the final answer.

Images