Multi-path hybrid knowledge retrieval enhancement generation method and system applied to vertical fields

Abstract

The application belongs to the field of knowledge retrieval enhancement generation, and discloses a multi-path mixed knowledge retrieval enhancement generation method and system applied to a vertical field. Through collaborative optimization of query understanding, multi-path recall, result reordering and knowledge fusion generation, the whole process from query to content generation is enhanced. A query rewriting strategy based on a large language model is introduced to improve semantic analysis and intent recognition capability. A multi-path recall mechanism is adopted, combined with text feature retrieval, vector semantic retrieval and multi-hop retrieval mechanism based on brain inspiration, to realize high-coverage recall of deep semantic information. Through multi-stage reordering optimization by a multi-stage reordering model, the accuracy and robustness of the recall result are improved. Finally, in the knowledge fusion and generation stage, weighted average or graph attention mechanism is used to realize dynamic fusion of multi-source knowledge and context-aware generation, thereby significantly improving the knowledge accuracy, generation consistency and semantic interpretability in professional field tasks.

Description

Technical Field

[0001] This invention belongs to the field of knowledge retrieval-augmented generation (RAG), specifically relating to a multi-path hybrid knowledge retrieval-augmented generation method and system applied to vertical domains. Background Technology

[0002] In recent years, significant progress has been made in information retrieval (IR), natural language processing (NLP), and machine learning (ML) technologies, especially the emergence of large language models (LLMs), which has provided new technical pathways for knowledge retrieval and generation tasks. Traditional information retrieval methods, such as TF-IDF (Term Frequency-Inverse Document Frequency), BM25 (Binary Independence Model 25), and N-gram (N-gram Language Model), mainly match and rank texts based on statistical features of term frequency and document frequency. While they perform well in general text retrieval, they often suffer from insufficient recall, semantic bias, and inadequate contextual understanding when facing complex semantic queries in vertical domains (such as healthcare, law, and finance).

[0003] With the development of semantic vector models, embedding-based vector retrieval methods, such as BGE (BAAI General Embedding), M3E (Massive Multilingual Multimodal Embedding), and E5 (Embedding for Everything), can map text into a high-dimensional semantic space and achieve semantic-level matching and recall through vector similarity calculation. These methods have stronger semantic expressive power than traditional statistical retrieval, but their recall scope is still limited by embedding coverage and single-hop semantic matching capabilities, making it difficult to effectively capture implicit knowledge connections across documents and paragraphs. To further improve the depth and accuracy of knowledge acquisition, multi-hop retrieval simulates the human reasoning process, step-by-step mining and integrating evidence from multiple knowledge fragments to achieve hierarchical understanding of complex problems and cross-document knowledge reasoning. Existing multi-hop retrieval methods include layer-by-layer retrieval guided by rewriting queries multiple times, semantic propagation methods based on graph structures, knowledge path connections achieved by constructing semantic graphs between documents or entities, and multi-round retrieval methods controlled by generative models. However, existing multi-hop retrieval methods still have some shortcomings in practical applications. The construction of retrieval paths relies on static heuristic rules or fixed query rewriting strategies, which makes the retrieval chain prone to deviating from the semantic goal, thereby affecting the depth and accuracy of cross-document knowledge integration.

[0004] Existing technologies, such as the patent with publication number CN120541206A, compare the overall score of the document set to be scored and the scoring threshold, and use reinforcement learning networks to generate retrieval strategies, adjusting the retrieval strategies to obtain high-quality recalled documents. Another example is the patent with publication number CN119938932A, which provides a knowledge graph-enhanced retrieval application system and method, including a public domain database, a private domain database, an extended query module, an LLM-agent interaction module, a memory prompt setting module, and an integrated language module. It generates extended queries through a large model, combines knowledge graphs and historical information to generate the final question-and-answer results, ensuring semantic consistency. However, neither of these technologies addresses the multi-hop problem or the complexities of structured data scenarios, nor do they consider strategies for processing key information and cross-document information during the ranking stage. They cannot effectively extract and integrate information from multiple documents, and these problems limit the application effectiveness of existing technologies in vertical fields. Summary of the Invention

[0005] The purpose of this invention is to provide a multi-path hybrid knowledge retrieval enhancement generation method and system applicable to vertical domains, addressing the problems of insufficient recall, incomplete semantic coverage, and poor contextual relevance of generated results in existing knowledge retrieval enhancement generation applications in vertical domains. This invention achieves end-to-end enhancement from query to content generation through collaborative optimization of query understanding, multi-path recall, result reordering, and knowledge fusion generation. This invention introduces a query rewriting strategy based on a large language model to improve semantic parsing and intent recognition capabilities; it employs a multi-path recall mechanism, combining text feature retrieval, vector semantic retrieval, and a brain-inspired multi-hop retrieval mechanism to achieve high-coverage recall of deep semantic information; it optimizes multi-stage reordering through a multi-stage reordering model to improve the accuracy and robustness of recall results; and finally, in the knowledge fusion and generation stage, it utilizes weighted averaging or graph attention mechanisms to achieve dynamic fusion and context-aware generation of multi-source knowledge, thereby significantly improving knowledge accuracy, generation consistency, and semantic interpretability in professional domain tasks.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0007] Firstly, a multi-path hybrid knowledge retrieval enhancement generation method for vertical domains is provided, including:

[0008] Receive natural language text input from the user as a user query;

[0009] Based on user queries, text feature recall retrieval method, vector model recall retrieval method and brain-inspired multi-hop retrieval method are used respectively to output preliminary candidate documents;

[0010] The initial candidate documents are segmented to obtain multiple candidate text fragments. Meta-information of the document in which they belong is added to each candidate text fragment. The user query and the candidate text fragments with meta-information are input into a multi-stage re-ranking model to obtain the final ranking and score of the selected text fragments.

[0011] Each final selected text fragment is assigned a corresponding weight based on its ranking and score. All final selected text fragments are then weighted and fused based on their weights. The user query and the weighted fused text are then input into the generation model to obtain the answer text corresponding to the user query.

[0012] Several alternative methods are provided below, but they are not intended as additional limitations on the overall solution above. They are merely further additions or optimizations. Provided there are no technical or logical contradictions, each alternative method can be combined individually with respect to the overall solution above, or multiple alternative methods can be combined with each other.

[0013] Preferably, the user query uses a rewriting strategy based on a large language model to generate an optimized query representation for recall, and the optimized query representation is used as input for text feature recall retrieval method, vector model recall retrieval method and brain-inspired multi-hop retrieval method respectively.

[0014] Preferably, the brain-inspired multi-hop retrieval method includes:

[0015] The large language model is used to identify entities and relations in user queries and generate original query triples.

[0016] The entities and relations in the original query triples are expanded using synonyms to obtain the expanded query triples;

[0017] A query knowledge graph is constructed based on the original query triples and the expanded query triples, and the meta-information of the document in which the node is located is added as an attribute to the node in the query knowledge graph.

[0018] Using nodes in the query knowledge graph as initial nodes, a personalized PageRank algorithm is used to perform multi-hop retrieval in the knowledge graph to be recalled, which consists of candidate documents.

[0019] Based on the node importance scores in the knowledge graph to be recalled output by the personalized PageRank algorithm, the top few candidate documents with the highest recall node importance scores are selected as the initial screening candidate documents output by the brain-inspired multi-hop retrieval method.

[0020] Preferably, the multi-stage reordering model includes:

[0021] Based on the similarity to the user query, the sentence order of each candidate text segment is rearranged.

[0022] The user query and the candidate text fragments after sentence reordering are input into multiple reordering models. The ensemble method is used to integrate the outputs of all reordering models to obtain the ranking and ensemble score of each candidate text fragment. The top few candidate text fragments with the highest ensemble scores are selected as the initial screened text fragments.

[0023] For the initial screening text fragments containing tabular data, the MRC model is used to parse the tabular data, and the parsing results are used to replace the corresponding tabular data to obtain the final screening text fragments; for the initial screening text fragments that do not contain tabular data, the initial screening text fragments are directly used as the final screening text fragments.

[0024] The ranking and integrated score of the initial screened text fragments are used as the ranking and score of the corresponding final screened text fragments.

[0025] Preferably, the sentence reordering of each candidate text segment includes:

[0026] Convert sentences from user queries and candidate text fragments into sparse vectors;

[0027] Calculate the similarity between the sparse vectors of each sentence in the candidate text fragment and the sparse vector of the user query;

[0028] The sentences in the candidate text fragments are reordered according to their similarity from high to low to obtain the candidate text fragments with rearranged sentence order.

[0029] Preferably, the step of parsing the tabular data using an MRC model includes:

[0030] A table detection algorithm is used to scan and initially screen text fragments, identifying the location and range of all tables.

[0031] Extract the table content based on its location and range, and concatenate the content of each cell in the table with the corresponding row and column headers to convert the table content into a text sequence;

[0032] Input the user query and text sequence into the MRC model to obtain the start and end positions in the text sequence output by the MRC model;

[0033] Based on the start and end positions, extract the corresponding text content from the text sequence as the parsing result.

[0034] Preferably, the step of assigning a corresponding weight to each final selected text segment based on its ranking and score, and then performing a weighted fusion of all final selected text segments based on these weights, includes:

[0035] The final score of the selected text segment will be used as the relevance score;

[0036] The source reliability score is assigned based on the metadata of the final selected text fragments;

[0037] Calculate the mean semantic similarity between the final selected text fragment and other final selected text fragments, and use it as the context consistency score;

[0038] Based on relevance score, source reliability score, and context consistency score, a weighted average algorithm or graph attention mechanism is used for dynamic fusion to obtain weighted fused text.

[0039] Secondly, a multi-path hybrid knowledge retrieval enhancement generation system for vertical domains is provided, including a processor and a memory storing a number of computer instructions, wherein the computer instructions, when executed by the processor, implement the steps of the multi-path hybrid knowledge retrieval enhancement generation method for vertical domains.

[0040] The multi-path hybrid knowledge retrieval enhancement generation method and system provided by this invention, applied to vertical fields, has the following advantages compared with the prior art:

[0041] 1) By adopting a multi-path recall strategy, text feature retrieval, vector semantic retrieval and brain-inspired multi-hop retrieval method are organically combined. This not only ensures the high accuracy of traditional retrieval, but also mines potential semantic relationships across documents and paragraphs through multi-hop retrieval, realizing in-depth knowledge discovery and semantic expansion recall, thereby significantly improving the breadth and depth of knowledge acquisition in vertical domains.

[0042] 2) A multi-stage re-ranking model is adopted, combined with Test Time Augmentation (TTA) and ensemble strategies, which effectively mitigates the bias problem of a single model and improves the robustness and generalization ability of the ranking results. By fusing sentence order reordering with meta-information and parsing tabular data using a Machine Reading Comprehension (MRC) model, more accurate retrieval results are provided, further enhancing the contextual relevance and ranking accuracy of candidate knowledge. Attached Figure Description

[0043] Figure 1 This is a flowchart of the multi-path hybrid knowledge retrieval enhancement generation method applied to vertical fields according to the present invention;

[0044] Figure 2 This is a flowchart of the brain-inspired multi-hop retrieval method of the present invention;

[0045] Figure 3 This is a flowchart illustrating the MRC model analysis process of the present invention. Detailed Implementation

[0046] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0047] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to limit the invention.

[0048] Example 1:

[0049] To address the problems of existing technologies, this embodiment proposes a multi-path hybrid knowledge retrieval enhancement and generation method applied to vertical domains. This method achieves end-to-end optimization from query to knowledge generation through the collaborative work of four core steps: query understanding and rewriting, multi-path recall, reorganization enhancement, and knowledge fusion and generation. The multi-path recall mechanism combines various retrieval methods, including text feature retrieval, vector semantic retrieval, and brain-inspired multi-hop information retrieval. The brain-inspired multi-hop retrieval simulates the hierarchical reasoning process of humans and the retrieval mechanism of the hippocampus memory processing flow. Through continuous retrieval and query expansion, it mines indirect evidence of connections from multiple documents, achieving cross-document knowledge integration and deep semantic reasoning, thus more comprehensively covering relevant information and improving retrieval recall and ranking accuracy. The introduction of multi-stage re-ranking and TTA enhancement strategies improves the stability and generalization ability of the ranking. In the knowledge fusion stage, a weighted or graph attention mechanism is used to integrate multi-source knowledge, ensuring the logical consistency and domain adaptability of the generated content. This method enables high-precision and robust knowledge retrieval and generation in a multi-source knowledge environment, providing a new technical solution for applications such as intelligent question answering, knowledge assistants, and document summarization in vertical fields.

[0050] like Figure 1 As shown in the figure, the multi-path hybrid knowledge retrieval enhancement generation method applied to vertical fields in this embodiment specifically includes the following steps:

[0051] Step 1: Receive the natural language text input by the user as the user query.

[0052] User queries can be directly used as input for subsequent retrieval methods. For example, in the medical field, a user might enter "the latest treatment for diabetes" as a query request. The user query is parsed, the query keywords are extracted, and the query request is passed to subsequent retrieval methods.

[0053] To improve the targeting and coverage of subsequent retrieval, this embodiment performs semantic parsing and intent recognition on user queries. Specifically, it adopts a query rewriting strategy based on a large language model to normalize, expand, or decompose the original user query (including splitting compound / multi-question queries into several sub-questions) and generate an optimized query representation for retrieval, which is then passed to the subsequent retrieval method.

[0054] Step 2: Based on user queries, output preliminary candidate documents using text feature recall retrieval method, vector model recall retrieval method, and brain-inspired multi-hop retrieval method respectively.

[0055] This embodiment implements a multi-path recall strategy based on multi-source data pathways, recalling and optimizing preliminary candidate documents related to the query from heterogeneous knowledge sources such as text corpora, structured databases, and knowledge graphs. The multi-path recall strategy includes text feature-based retrieval, vector model-based similarity retrieval, and brain-inspired multi-hop retrieval methods to achieve the mining and extended recall of deep semantic information.

[0056] (1) In the text feature recall retrieval method, the system uses methods such as TF-IDF, BM25, N-gram, and Keyword (Keyword Matching Method) to quickly filter out preliminary candidate documents that highly match the query terms. For example, for the query "the latest treatment for diabetes", the system will prioritize recalling documents containing keywords such as "diabetes", "treatment", and "latest". The retrieval objects are vertical domain text corpora (papers, manuals, reports) and structured documents (tables, parameter tables).

[0057] (2) In the vector model recall retrieval method, the system utilizes models such as BGE, M3E, and E5, combined with Faiss (Facebook AI Similarity Search), to perform vector encoding on documents. In particular, this invention incorporates the document's meta information (such as title, time information, etc.) during vector encoding to improve retrieval performance. For example, for a document titled "Advances in Diabetes Treatment in 2023," the system considers not only the vector representation of the document content but also its title and publication time, thereby more accurately locating the document in the vector space. In this way, the system can more comprehensively evaluate the relevance of documents and improve recall accuracy. Furthermore, this embodiment employs a dual-tower structure or a multi-vector semantic matching structure to achieve high-dimensional semantic alignment between queries and knowledge fragments. The retrieval object is a vertical domain text fragment vector library (semantic vectors encoded with BGE / E5 after splitting documents into text fragments), stored in a vector database (such as Faiss, Milvus).

[0058] (3) The brain-inspired multi-hop retrieval method simulates the hippocampal memory processing flow, handling multi-hop information retrieval and cross-document retrieval. It can extract relevant information from multiple documents and connect knowledge in different documents through multi-hop methods to achieve deeper information retrieval and knowledge generation, further improving the flexibility and accuracy of information retrieval. Specifically, LLM is used to extract triple relations, and the embedding model is used to provide synonym expansion to complete the construction of the knowledge graph. For example, for the query "the latest treatment for diabetes", the system will extract triple relations of keywords such as "diabetes", "treatment", and "latest", such as "diabetes-treatment-insulin" and "diabetes-treatment-diet control".

[0059] After extracting key features from the query, the system applies a personalized PageRank algorithm to the knowledge graph to retrieve results. When the retrieval is primarily time-based or keyword-based, text feature retrieval and vector retrieval methods are prioritized. The personalized PageRank algorithm weights and sorts nodes in the knowledge graph according to the specific needs of the query, thereby retrieving the most relevant documents. This mechanism can handle not only multi-hop information retrieval but also cross-document retrieval, thus improving the flexibility and accuracy of information retrieval. Cross-document retrieval involves constructing a relational graph between documents, using nodes and edges in the graph to represent the relationships between documents, thereby achieving efficient retrieval of document content. The retrieval objects are vertical domain knowledge graphs (entities: algorithms, technical parameters, diseases, drugs; relations: applied to, optimized, based on, treated) and text corpora (serving as the original supporting fragments for knowledge graph entities / relationships).

[0060] like Figure 2 As shown, the brain-inspired multi-hop retrieval method in this embodiment has the following specific steps:

[0061] (3.1) Use a large language model to identify entities and relationships in user queries and generate original query triples. For example, in the legal field, if a user queries "legal provisions on liability for breach of contract in contract law", use LLM to perform semantic analysis on the query, identify key entities and relationships in the query, identify "contract law" as the subject, "inclusion" as the relationship, and "liability for breach of contract" as the object, forming the triple "contract law-inclusion-liability for breach of contract".

[0062] (3.2) The entities and relations in the original query triples are expanded with synonyms to obtain the expanded query triples; a query knowledge graph is constructed based on the original query triples and the expanded query triples, and the meta-information of the document in which the node is located is added as an attribute to the node in the query knowledge graph.

[0063] This embodiment constructs a knowledge graph by using the subject and object of each triple as nodes and relations as edges. Specifically, it utilizes an embedding model to provide synonym expansion. For example, "breach of contract" and "violation" can be considered synonyms, as can "responsibility" and "obligation," thereby adding more nodes and edges to the knowledge graph and improving its richness and accuracy.

[0064] Meanwhile, when constructing the knowledge graph, document meta information (such as title, time information, etc.) is also added, which can be attached to the corresponding subject entity and object entity as node attributes. This enables semantic clustering and navigation using document-level contextual information, thereby improving the contextual relevance of the knowledge graph.

[0065] Similarly, the same method is used to construct a knowledge graph for the candidate documents, which will serve as the knowledge graph to be recalled.

[0066] (3.3) After the knowledge graph is constructed, a personalized PageRank algorithm is applied to improve the efficiency and accuracy of cross-document retrieval and multi-hop information retrieval. The personalized PageRank algorithm is a graph-based ranking algorithm that ranks documents by calculating the importance of nodes. Specifically, the keywords in the user query are used as initial nodes, and multi-hop retrieval is performed through the edges in the knowledge graph to calculate the importance score of each node. The higher the importance score of a node, the greater its importance in the knowledge graph to be recalled, and the more relevant the corresponding candidate documents are.

[0067] Specifically, the calculation formula for the personalized PageRank algorithm is as follows:

[0068]

[0069] in, Represents a node Importance score This represents the damping factor (usually taken as 0.85). Indicates pointing to a node The set of nodes, Represents a node The number of outgoing edges, Represents a node Importance score.

[0070] (3.4) Based on the node importance scores calculated by the personalized PageRank algorithm in the knowledge graph to be recalled, the top few candidate documents with the highest node importance scores are selected as the initial screening candidate documents output by the brain-inspired multi-hop retrieval method. The recalled documents include not only documents directly related to the query keywords, but also related documents found through multi-hop retrieval, enabling the method of this invention to provide more comprehensive and accurate retrieval results and avoid missing important content.

[0071] Through the steps described above, the brain-inspired multi-hop retrieval method proposed in this embodiment significantly improves the efficiency and accuracy of cross-document retrieval and multi-hop information retrieval in the legal field. The knowledge graph constructed using LLM and embedding models can better handle the complexity and diversity of legal documents, providing higher-quality search results. It not only improves the response efficiency and accuracy of complex queries in vertical domains but also provides users with a more intelligent and personalized information retrieval experience, demonstrating significant application value and market potential.

[0072] Step 3: Segment the initial candidate documents to obtain multiple candidate text fragments, add metadata about the document to each candidate text fragment, input the user query and the candidate text fragments with metadata into the multi-stage re-ranking model to obtain the final ranking and score of the selected text fragments.

[0073] This embodiment evaluates and re-ranks the candidate knowledge set obtained from multi-path recall. A vector database is used for initial ranking by similarity, and a multi-stage re-ranking model is employed for further optimization to improve the accuracy and relevance of the final ranking. The multi-stage re-ranking model uses a TTA strategy or other ensemble strategies to enhance ranking generalization ability, robustness, and ranking performance; the MRC model is used to further parse the tabular data before outputting the results.

[0074] Before inputting candidate text chunks into the reranking model, the TTA strategy is used to reorder the chunks. The TTA strategy is a machine learning approach that performs multiple transformations on input samples during the inference phase to improve the model's predictive robustness and generalization performance. In this invention, it is primarily applied to sentence reordering, eliminating occasional errors through multi-version fusion, improving Top-K accuracy, adapting to candidate knowledge with different formats and semantic representations, and reducing the risk of overfitting the rerank model to a single input format.

[0075] First, the initial candidate documents obtained through the multi-path recall strategy are segmented by periods to obtain multiple chunks. TF-IDF is then used to convert the segmented sentences into sparse vectors. The sparse vector representation of a sentence can be calculated using the following formula:

[0076]

[0077] in, Indicator In the sentence The weights in Indicator In the sentence Frequency of occurrence in This represents the total number of documents in the document set. Indicates the inclusion word The number of documents. By calculating the sparse vector of each sentence, the similarity between sentences can be quantified, thus providing a basis for subsequent ranking.

[0078] To further improve the accuracy of sentence similarity calculation, cosine similarity is used. The cosine similarity formula is as follows:

[0079]

[0080] in, This represents the similarity between the sparse vectors of each sentence in the candidate text segment and the sparse vector of the user query. and Let each represent a sparse vector representing a sentence in the candidate text segment and a user query, respectively. Represents the dot product of vectors. and These represent the magnitudes of the vectors.

[0081] Then, the sentences in the candidate text segments are reordered according to their similarity from high to low. The reordering alternates between left-to-right and right-to-left, placing more important sentence information at the beginning of the chunk segments. The reordered sentences are then recombined into chunk segments, ensuring that each chunk segment begins with a sentence with a high similarity score. This ensures that the ReRank model encounters the most relevant sentence information first when processing each chunk segment, thereby improving the input quality of the ReRank model.

[0082] Then, the chunk fragments with meta information are input into the rerank model. The rerank models selected in this invention are JINA-reranker-v2-base-multilingual and LdIR-Qwen2-reranker-1.5B. The ensemble method is used to integrate the outputs of all rerank models to obtain the ranking and ensemble score of each candidate text fragment. The top few candidate text fragments with the highest ensemble scores are selected as the initial screened text fragments.

[0083] Finally, the table data is parsed again using the MRC model, and the output is the start position and end position, which are then used to locate the index of the specific answer. Specifically: for the initial screening text fragments containing table data, the MRC model is used to parse the table data, and the parsing results are used to replace the corresponding table data to obtain the final screening text fragments; for the initial screening text fragments that do not contain table data, the initial screening text fragments are directly used as the final screening text fragments; the ranking and integration score of the initial screening text fragments are used as the ranking and score of the corresponding final screening text fragments.

[0084] This embodiment linearizes the table content into a semantic sequence that the model can understand, and combines user queries to achieve precise positioning at the multi-cell level within the table, providing high-quality knowledge input for subsequent model generation.

[0085] like Figure 3 As shown, the specific steps for parsing tabular data using the MRC model in this embodiment are as follows:

[0086] (1) Table detection: A table detection algorithm is used to scan and initially screen text fragments, identifying the location and extent of all tables. The table detection algorithm can be implemented based on a deep convolutional neural network (CNN) or a document understanding model based on Transformer (such as TableNet, CascadeTabNet, etc.), and can accurately detect the bounding boxes of tables in the document. The detection results are represented as a set of tables:

[0087]

[0088] in, and They represent the first The coordinates of the top left and bottom right corners of each table; these coordinates can be pixel values ​​or normalized coordinates. This represents the total number of tables.

[0089] (2) Table content extraction and preprocessing: Extracting table content and its metadata. Extracted content includes the table title, row titles, column titles, and cell data. The extracted original table content is represented as follows:

[0090]

[0091] in, Indicates the first Line number The content of the cells in the column.

[0092] The query request entered by the user is denoted as ,in Indicates the first Each query term or phrase. To improve the model's understanding and matching performance, the system analyzes the table content. Preprocessing and structure transformation are performed to linearize the two-dimensional table into a text sequence, enabling the model to capture the semantic relationships and hierarchical structure of the table. The linearized table is represented as follows:

[0093]

[0094] in, Represents table content The transformed text sequence, function This represents a text-based function used to concatenate cell content with the corresponding row / column headers to create a natural language description. For the first The title of the line. For the first The column header.

[0095] (3) MRC model analysis, which analyzes the preprocessed text sequence. User query The information is then fed into the MRC model for reading comprehension and reasoning. The MRC model is a Transformer-based deep language model that predicts the start and end positions of the answer in the input sequence by jointly modeling contextual semantics and query intent. This process can be formalized as follows:

[0096]

[0097] in, and These represent the start and end positions of the answer in the linearized sequence of the table, respectively. If we map the positions back to the original table's two-dimensional coordinate space, then:

[0098]

[0099] in, The starting position is the first in the table. Line number List, The end position is the first in the table. Line number List.

[0100] (4) Answer location and output: Based on the start and end positions, extract the corresponding text content from the text sequence as the parsing result. , is represented as:

[0101]

[0102] Extract the content of the answer area and return it to the user, or use it as input for knowledge enhancement processing (step 4) for subsequent generative models.

[0103] Through the above steps, the MRC model parsing process in this embodiment can automatically detect and extract the table data in the document, linearize the table content into a semantic sequence that the model can understand, and combine user queries to achieve precise positioning at the multi-cell level within the table, providing high-quality knowledge input for subsequent model generation.

[0104] Step 4: Assign a corresponding weight to each final selected text fragment according to the sorting and score of the final selected text fragments, and perform weighted fusion of all final selected text fragments based on the weights. Input the user query and the weighted fused text into the generation model to obtain the answer text corresponding to the user query.

[0105] The final selected text fragments, after being re-ranked by the Rerank model, are dynamically fused using a weighted average algorithm or a graph attention mechanism based on their source reliability, relevance score, and contextual consistency to generate a unified, high-quality knowledge representation, thus avoiding knowledge redundancy and semantic conflicts. This fused knowledge representation, along with the user's optimized query, is then input into the generative model (large language model) to generate output content that is semantically consistent with the query, logically complete, and conforms to the vertical domain's knowledge specifications. Further integration with domain knowledge templates can enhance the accuracy, interpretability, and domain adaptability of the generated content, thereby ensuring consistency and traceability of the generated results at the knowledge level.

[0106] Step 4.1: Use the final score of the selected text fragments as the relevance score.

[0107] Step 4.2: Assign a source reliability score based on the metadata of the final selected text fragments. The assignment rules are preset, for example, the score for top conference papers (ICRA / IROS) is 0.9-1.0 (0.95 in this example); the score for core journals is 0.8-0.9 (0.85 in this example); the score for ordinary literature is 0.6-0.8 (0.7 in this example); and the score for non-academic sources is 0.3-0.6 (0.4 in this example).

[0108] Step 4.3: Calculate the mean semantic similarity between the final selected text fragments and other final selected text fragments, as the context consistency score. For example, use the E5 model to encode all chunk fragments into vectors; calculate the cosine similarity of each chunk fragment with all other chunk fragments; take the mean similarity and normalize it to obtain the context consistency score, the higher the score, the more consistent the context.

[0109] Step 4.4: Based on the relevance score, source reliability score, and context consistency score, a weighted average algorithm or graph attention mechanism is used for dynamic fusion to obtain the weighted fused text.

[0110] When using a weighted average algorithm for dynamic fusion, the weights of each score are pre-set, for example, the relevance score has a weight of 0.5; the source reliability score has a weight of 0.3; and the context consistency score has a weight of 0.2. Then, the scores and their weights are weighted, summed, and normalized to obtain the final weight of the final selected text fragments. All the final selected text fragments are then fused according to the final weights to obtain the weighted fused text.

[0111] When using a graph attention mechanism for dynamic fusion, each candidate chunk is treated as a node in the graph, and the associations between chunks (such as semantic similarity exceeding a threshold) are treated as edges. The weight of each edge is the context consistency score of the chunk. The relevance score and source reliability score of each node are used as the initial feature vector of the node. Through the graph attention layer, the attention score of each node relative to other nodes is calculated. The attention score is added to the node's initial feature vector and normalized to obtain the fusion weight. The node's own initial feature vector is aggregated with the initial feature vectors of all neighboring nodes according to the attention score to obtain the node's enhanced feature vector. Based on the node's fusion weight and the enhanced feature vector, a weighted fusion is performed to obtain the weighted fused text.

[0112] When the method of this invention is applied to UAV path planning question answering, the user query is a natural language question (e.g., what is the energy consumption optimization rate of the improved A* algorithm in multi-UAV collaboration), and the output is the answer to the corresponding natural language question and the source of the answer (e.g., the energy consumption optimization rate of the improved A* algorithm is 22%, refer to ICRA 2024 paper "XXX").

[0113] When the method of this invention is applied to a research assistant, the user query is a natural language description of the request (such as generating a review outline of multi-UAV energy consumption optimization algorithms in 2024), and the output is the answer to the corresponding natural language question and the source citation of the answer (such as the outline overview description, referencing the ICRA 2024 paper "XXX").

[0114] When the method of this invention is applied to the generation of scientific paper abstracts, the user query is a natural language requirement description (such as a top conference paper on UAV path planning (or its name) and abstract requirement), and the output is the answer to the corresponding natural language question and the source citation of the answer (such as a structured information abstract, page numbers and chapters explaining the source of the core fragments, etc.).

[0115] The specific settings for the large language model, E5 model, JINA-reranker-v2-base-multilingual reordering model, LdIR-Qwen2-reranker-1.5B reordering model, MRC model, and TableNet model used in this embodiment are as follows:

[0116] (1) Large language model (taking Qwen2-type Transformer Decoder architecture as an example) is a standard Transformer inter-layer connection; the pre-training stage uses a large-scale general corpus, including but not limited to Common Crawl, books, academic literature, etc. (vertical domain fine-tuning can supplement industry corpus, such as UAV technical manuals, medical literature). The general corpus is cleaned, segmented and truncated / filled before use. Autoregressive cross-entropy loss is used as the loss function, and the AdamW (Adam with Weight Decay) optimizer is used to optimize the model weights. The initial learning rate in the pre-training stage is 2e-5, and the fine-tuning stage is 1e-6. The pre-training stage uses 1024 tokens / batch. The weight decay coefficient is 0.01. The optimization strategy combines linear preheating of the learning rate and cosine annealing strategy.

[0117] (2) The E5 model (the specific model used in the embedding model) adopts the standard inter-layer connections of the Transformer Encoder; the MTEB (Massive Text Embedding Benchmark) dataset and general corpus are used in the pre-training stage (industry corpus, such as UAV algorithm literature, can be supplemented for fine-tuning in vertical domains); the training data is cleaned, segmented, and truncated / padded before use, and contrastive loss (InfoNCE Loss) and AdamW optimizer are adopted. The learning rate is set to 2e-5 in the pre-training stage and 1e-5 in the fine-tuning stage; the batch size is 100 text pairs / batch; the hidden layer dimension is 768 dimensions; the optimization strategy adopts cosine annealing of the learning rate and the weight decay coefficient is 0.01.

[0118] (3) The JINA-reranker-v2-base-multilingual reranking model uses standard inter-layer connections in the Transformer Encoder. Training was conducted using multilingual ranking datasets, such as multilingual MS MARCO and XOR-TyDi QA. The training data underwent text cleaning, word segmentation, and sequence truncation / padding before use. Cross-entropy loss and the AdamW optimizer were employed. The learning rate was set to 2e-5, the batch size was 80 samples / batch, and the number of multi-head attention heads was 12. The optimization strategy combined linear warm-up (500 steps) and cosine annealing.

[0119] (4) The LdIR-Qwen2-reranker-1.5B reranking model uses standard inter-layer connections in the Transformer Decoder. During training, fine-ranking task datasets such as MS MARCO Passage Ranking and NQ (Natural Questions) are used. The training data undergoes text cleaning, word segmentation, and sequence truncation / padding before use. Pairwise contrastive loss and the AdamW optimizer are employed. The learning rate is set to 1e-6; the batch size is 50 samples / batch; the hidden layer dimension is 1024; the optimization strategy combines a constant learning rate with weight decay (0.01).

[0120] (5) The MRC model uses standard inter-layer connections in the Transformer Encoder. The training data consists of extractive MRC datasets such as SQuAD (Stanford Question Answering Dataset) and DRCD (industry-specific question-and-answer data, such as UAV troubleshooting questions and answers, can be supplemented for vertical domains). The training data is cleaned, segmented, and truncated / padded before use. Cross-entropy loss is calculated at the start and end positions, and the total loss is obtained by summing them. The AdamW optimizer is used. The learning rate is set to 2e-5; the batch size is 30 samples / batch; the optimization strategy combines linear warm-up (1000 steps) and linear decay of the learning rate.

[0121] (6) The TableNet model is a fully convolutional neural network (FCN), consisting of an encoder and a decoder. The encoder uses a pre-trained VGG16 network (with fully connected layers removed), and the decoder uses a deconvolutional layer (transposed convolution). The encoder uses the ReLU activation function, and the decoder output layer uses the Sigmoid activation function. The training data consists of table detection datasets such as ICDAR Table Detection Dataset and PubLayNet. The training data is normalized, resized, and labeled before use. Binary cross-entropy loss (BCE Loss) and Dice loss, along with the Adam optimizer, are used. The learning rate is 1e-4; the batch size is 10 images / batch; the number of channels in the deconvolutional layer is the same as the number of channels in the corresponding layer of the encoder; the optimization strategy is cosine annealing with a learning rate and a weight decay coefficient of 0.001.

[0122] Example 2:

[0123] This embodiment provides a multi-path hybrid knowledge retrieval enhancement generation system applied to a vertical domain, including a processor and a memory storing a number of computer instructions. When the computer instructions are executed by the processor, they implement the steps of the multi-path hybrid knowledge retrieval enhancement generation method applied to a vertical domain.

[0124] For specific limitations on multi-path hybrid knowledge retrieval enhancement generation systems applied to vertical domains, please refer to the limitations on multi-path hybrid knowledge retrieval enhancement generation methods applied to vertical domains mentioned above, which will not be repeated here.

[0125] The memory and processor are electrically connected directly or indirectly to enable data transmission or interaction. For example, these components can be electrically connected to each other via one or more communication buses or signal lines. The memory stores a computer program that can run on the processor, which implements the method of the present invention by running the computer program stored in the memory.

[0126] The memory may be, but is not limited to, Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), etc. The memory stores the program, and the processor executes the program upon receiving an execution instruction.

[0127] The processor may be an integrated circuit chip with data processing capabilities. The aforementioned processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this invention. The general-purpose processor can be a microprocessor or any conventional processor.

[0128] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0129] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the appended claims.

Claims

1. A multi-path hybrid knowledge retrieval enhancement generation method applied to vertical domains, characterized in that, The multi-path hybrid knowledge retrieval enhancement generation method applied to vertical domains includes: Receive natural language text input from the user as a user query; Based on user queries, text feature recall retrieval method, vector model recall retrieval method and brain-inspired multi-hop retrieval method are used respectively to output preliminary candidate documents; The initial candidate documents are segmented to obtain multiple candidate text fragments. Meta-information of the document in which they belong is added to each candidate text fragment. The user query and the candidate text fragments with meta-information are input into a multi-stage re-ranking model to obtain the final ranking and score of the selected text fragments. Each final selected text fragment is assigned a corresponding weight based on its ranking and score. All final selected text fragments are then weighted and fused based on their weights. The user query and the weighted fused text are then input into the generation model to obtain the answer text corresponding to the user query.

2. The multi-path hybrid knowledge retrieval enhancement generation method applied to vertical fields according to claim 1, characterized in that, The user query is used to generate an optimized query representation for recall using a rewriting strategy based on a large language model. The optimized query representation is then used as input for text feature recall retrieval method, vector model recall retrieval method, and brain-inspired multi-hop retrieval method, respectively.

3. The multi-path hybrid knowledge retrieval enhancement generation method applied to vertical fields according to claim 1, characterized in that, The brain-inspired multi-hop retrieval method includes: The large language model is used to identify entities and relations in user queries and generate original query triples. The entities and relations in the original query triples are expanded using synonyms to obtain the expanded query triples; A query knowledge graph is constructed based on the original query triples and the expanded query triples, and the meta-information of the document in which the node is located is added as an attribute to the node in the query knowledge graph. Using nodes in the query knowledge graph as initial nodes, a personalized PageRank algorithm is used to perform multi-hop retrieval in the knowledge graph to be recalled, which consists of candidate documents. Based on the node importance scores in the knowledge graph to be recalled output by the personalized PageRank algorithm, the top few candidate documents with the highest recall node importance scores are selected as the initial screening candidate documents output by the brain-inspired multi-hop retrieval method.

4. The multi-path hybrid knowledge retrieval enhancement generation method applied to vertical domains according to claim 1, characterized in that, The multi-stage reordering model includes: Based on the similarity to the user query, the sentence order of each candidate text segment is rearranged. The user query and the candidate text fragments after sentence reordering are input into multiple reordering models. The ensemble method is used to integrate the outputs of all reordering models to obtain the ranking and ensemble score of each candidate text fragment. The top few candidate text fragments with the highest ensemble scores are selected as the initial screened text fragments. For the initial screening text fragments containing tabular data, the MRC model is used to parse the tabular data, and the parsing results are used to replace the corresponding tabular data to obtain the final screening text fragments; for the initial screening text fragments that do not contain tabular data, the initial screening text fragments are directly used as the final screening text fragments. The ranking and integrated score of the initial screened text fragments are used as the ranking and score of the corresponding final screened text fragments.

5. The multi-path hybrid knowledge retrieval enhancement generation method applied to vertical domains according to claim 4, characterized in that, The reordering of sentence segments for each candidate text segment includes: Convert sentences from user queries and candidate text fragments into sparse vectors; Calculate the similarity between the sparse vectors of each sentence in the candidate text fragment and the sparse vector of the user query; The sentences in the candidate text fragments are reordered according to their similarity from high to low to obtain the candidate text fragments with rearranged sentence order.

6. The multi-path hybrid knowledge retrieval enhancement generation method applied to vertical domains according to claim 4, characterized in that, The process of parsing tabular data using the MRC model includes: A table detection algorithm is used to scan and initially screen text fragments, identifying the location and range of all tables. Extract the table content based on its location and range, and concatenate the content of each cell in the table with the corresponding row and column headers to convert the table content into a text sequence; Input the user query and text sequence into the MRC model to obtain the start and end positions in the text sequence output by the MRC model; Based on the start and end positions, extract the corresponding text content from the text sequence as the parsing result.

7. The multi-path hybrid knowledge retrieval enhancement generation method applied to vertical domains according to claim 1, characterized in that, The process involves assigning a corresponding weight to each final selected text segment based on its ranking and score, and then performing a weighted fusion of all final selected text segments based on these weights, including: The final score of the selected text segment will be used as the relevance score; The source reliability score is assigned based on the metadata of the final selected text fragments; Calculate the mean semantic similarity between the final selected text fragment and other final selected text fragments, and use it as the context consistency score; Based on relevance score, source reliability score, and context consistency score, a weighted average algorithm or graph attention mechanism is used for dynamic fusion to obtain weighted fused text.

8. A multi-path hybrid knowledge retrieval enhancement generation system applied in a vertical field, comprising a processor and a memory storing a plurality of computer instructions, characterized in that, When the computer instructions are executed by the processor, they implement the steps of the multi-path hybrid knowledge retrieval enhancement generation method for vertical domains as described in any one of claims 1 to 7.

Images