Multi-source information fusion query method, device and equipment based on RAG Fusion

Abstract

The invention discloses a multi-source information fusion query method based on RAG Fusion, which comprises the following steps: receiving original query data input by a user, and carrying out preprocessing and expansion processing on the original query data to obtain an expanded query set; retrieving each extended query in the extended query set in a preset vector library by using a plurality of retrieval strategies to obtain a document set corresponding to each retrieval strategy, and reordering all documents in the document set to obtain a document list; dynamic context construction is carried out on documents in the document list, and a context document set with weights is generated; performing sentence-level segmentation on each document in the context document set, extracting fragments related to the original query data from segmented sentences by utilizing a pre-trained question and answer model, and performing duplicate removal and combination on the extracted fragments to generate an information unit set; and inputting the original query data, the context document set and the information unit set into a pre-trained large-scale language model to generate a target answer.

Description

Technical Field

[0001] The present invention relates to the field of information retrieval technology, and in particular to a multi-source intelligence fusion query method, device and equipment based on RAG Fusion. Background Art

[0002] In the era of information explosion, the rapid and accurate retrieval and understanding of intelligence information has become a vital need. Traditional intelligence information retrieval technology mainly relies on keyword matching algorithms, which simply compare keywords in titles or article content to filter relevant information. However, this method has significant limitations: low retrieval efficiency, making it difficult to quickly locate the information that users really need in massive data; lack of content understanding ability, unable to effectively summarize or refine the retrieved information, resulting in users spending a lot of time on self-screening and analysis.

[0003] In order to overcome these shortcomings, a retrieval technology based on Retrieval Augmented Generation (RAG) was introduced. The RAG model combines the retrieval module and the generation module to retrieve relevant information from a large-scale text library and use this information to assist in generating more accurate and rich responses. This technology significantly improves retrieval efficiency. It can not only quickly find relevant articles, documents, and even multimedia resources such as videos, but also summarize the retrieved content to a certain extent, greatly improving the user experience.

[0004] However, although RAG technology has made significant progress in the field of intelligence information retrieval, it still has some limitations. RAG models often rely on a single retrieval strategy, which limits their performance in complex query scenarios. Due to the diversity of query intent and the differences in document representation, a single retrieval strategy is often difficult to fully cover all possible matching situations, resulting in the omission of some relevant information. In addition, the RAG model is also insufficient in its ability to integrate intelligence sources. In practical applications, intelligence information often comes from multiple different data sources, and there may be large differences in the format, vocabulary, expression, etc. between these data sources. When processing this type of cross-source information, traditional RAG models often lack an effective fusion mechanism and find it difficult to fully utilize the advantages of each data source, thus affecting the overall retrieval effect. Summary of the invention

[0005] In view of this, the purpose of the present invention is to propose a multi-source intelligence fusion query method, device and equipment based on RAG Fusion, aiming to solve the problems of traditional RAG technology adopting a single retrieval strategy and insufficient intelligence source fusion capability.

[0006] To achieve the above object, the present invention provides a multi-source intelligence fusion query method based on RAG Fusion, the method comprising:

[0007] Receiving original query data input by a user, performing preprocessing and expansion processing on the original query data to obtain an expanded query set;

[0008] Using multiple search strategies to search each extended query in the extended query set in a preset vector library, obtaining a document set corresponding to each search strategy, and reordering all documents in the document set to obtain a document list;

[0009] Dynamically constructing contexts for the documents in the document list to generate a context document set with weights;

[0010] Segment each document in the context document set at the sentence level, extract segments related to the original query data from the segmented sentences using a pre-trained question-answering model, deduplicate and merge the extracted segments, and generate an information unit set;

[0011] The original query data, the context document set, and the information unit set are input into a pre-trained large-scale language model to generate a target answer.

[0012] Preferably, the preprocessing and expansion processing of the original query data to obtain an expanded query set includes:

[0013] Using natural language processing technology to pre-process the original query data including word segmentation, part-of-speech tagging and entity recognition to obtain multiple word segmentation results;

[0014] Performing at least one expansion process including format, vocabulary, and expression on the word segmentation result to obtain multiple variant results, wherein the vocabulary expansion process includes synonym replacement, hyponym expansion variant results;

[0015] Based on the multiple variant results, the original query data is input into a pre-trained large language model to generate the expanded query set.

[0016] Preferably, the method of using multiple search strategies to search each extended query in the extended query set in a preset vector library to obtain a document set corresponding to each search strategy includes:

[0017] Match each extended query in a preset vector library by vector similarity to obtain a first document;

[0018] The BM25 algorithm is used to calculate the relevance score between each extended query and the documents in the preset vector library, and the first N documents with the highest relevance score are used as the second document;

[0019] Perform entity recognition and linking on each extended query, determine the relevant knowledge graph nodes, and search the knowledge graph for nodes and information associated with the entity to obtain a third document, and use the first document, the second document, and the third document as the document set.

[0020] Preferably, reordering all documents in the document set to obtain a document list includes:

[0021] Calculate the relevance score of each document in the document collection to the original query data using a cross encoder;

[0022] Counting the number of times each document in the document collection appears in different extended queries and different retrieval strategies;

[0023] Based on the relevance score and the number of times, a weighted sum algorithm is used to calculate to obtain a score corresponding to each document, and all documents are reordered according to the score to obtain the document list.

[0024] Preferably, the dynamically constructing contexts of the documents in the document list to generate a weighted context document set includes:

[0025] Selecting the first N documents from the document list, and constructing a relationship network for the selected first N documents using a graph model to obtain a document relationship grid;

[0026] Calculate each document in the document relationship grid using the PageRank algorithm to obtain a PageRank value corresponding to each document, and determine a key document among all documents according to the PageRank value;

[0027] The weight of each document in the context is dynamically adjusted according to the importance between the key documents and the relevance of the documents in the document set to obtain the context document set.

[0028] Preferably, the step of inputting the original query data, the context document set, and the information unit set into a pre-trained large-scale language model to generate a target answer includes:

[0029] Inputting the original query data, the context document set, and the information unit set into a pre-trained large-scale language model to obtain an initial answer;

[0030] Based on the initial answer, inputting the constructed prompt word into a large-scale language model to generate a to-be-determined answer that references information in the context document set;

[0031] The information unit set is used to perform a fact check on the answer to be determined to obtain the target answer.

[0032] Preferably, the method further comprises:

[0033] The original query data and the target answer are used as query-answer pairs, the query-answer pairs are vectorized to be stored in the preset vector library, and the document index is updated.

[0034] To achieve the above object, the present invention also provides a multi-source intelligence fusion query device based on RAG Fusion, the device comprising:

[0035] A data processing unit, configured to receive original query data input by a user, and perform preprocessing and expansion processing on the original query data to obtain an expanded query set;

[0036] A document retrieval unit, configured to use a plurality of retrieval strategies to retrieve each extended query in the extended query set in a preset vector library, obtain a document set corresponding to each retrieval strategy, and reorder all documents in the document set to obtain a document list;

[0037] A context construction unit, used to dynamically construct contexts for the documents in the document list to generate a context document set with weights;

[0038] An information generation unit, configured to segment each document in the context document set at a sentence level, extract segments related to the original query data from the segmented sentences using a pre-trained question-answering model, deduplicate and merge the extracted segments, and generate an information unit set;

[0039] The answer generation unit is used to input the original query data, the context document set and the information unit set into a pre-trained large-scale language model to generate a target answer.

[0040] In order to achieve the above-mentioned objectives, the present invention also proposes a multi-source intelligence fusion query device based on RAG Fusion, including a processor, a memory, and a computer program stored in the memory, and the computer program is executed by the processor to implement the steps of a multi-source intelligence fusion query method based on RAG Fusion as described in the above embodiment.

[0041] In order to achieve the above-mentioned objectives, the present invention also proposes a computer-readable storage medium, on which a computer program is stored, and the computer program is executed by a processor to implement the steps of a multi-source intelligence fusion query method based on RAG Fusion as described in the above embodiment.

[0042] In order to achieve the above objectives, the present invention also proposes a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of a multi-source intelligence fusion query method based on RAG Fusion as described in the above embodiment.

[0043] Beneficial effects:

[0044] The above scheme increases the breadth and depth of the query by preprocessing and expanding the original query, which helps to discover more relevant documents. By adopting multiple retrieval strategies, retrieval can be performed from different angles and dimensions, improving the flexibility and efficiency of retrieval. Building dynamic context based on the relationship between documents helps to generate more accurate and coherent answers; using pre-trained large-scale language models and question-answering models, the accuracy and readability of answer generation are improved.

[0045] The above scheme can simultaneously process and fuse multiple types of intelligence sources such as structured data, unstructured text, images, and videos, greatly improving the comprehensiveness and integrity of information. By adopting multiple fusion strategies (such as RRF) and dynamically adjusting the fusion weight according to the original query and intelligence source characteristics, the fusion results are more accurate and flexible. By identifying the original query intent and key entities, more targeted retrieval results can be provided.

[0046] The above scheme improves the accuracy and efficiency of query processing by performing steps such as word segmentation, part-of-speech tagging and entity recognition on the original query data, and increases the semantic coverage of the query by performing synonym replacement and hyponym expansion, which helps to discover more relevant variants; and uses a large language model to generate an extended query set, further enhancing the generalization ability of the query.

[0047] The above scheme can quickly filter documents similar to the query through vector similarity matching, thereby improving retrieval efficiency; accurately calculate the relevance score between the document and the query through the BM25 algorithm to ensure that the most relevant documents are retrieved; and realize entity recognition and linking through knowledge graph retrieval, find relevant information in the knowledge graph, and enrich the retrieval results.

[0048] In the above scheme, the cross encoder calculation can accurately evaluate the relevance of documents and queries and improve the accuracy of sorting; the important documents can be further screened by combining the performance of documents in different queries and strategies; the weighted sum algorithm can comprehensively consider multiple factors and sort documents more reasonably.

[0049] The above scheme builds a relational network through a graph model to clearly represent the relationship between documents, which helps to understand the connection and context between documents. It uses the PageRank algorithm to automatically identify key documents based on the importance and relationship of documents, and dynamically adjusts the weights according to the relevance between key documents, making the context document collection more reasonable and useful.

[0050] The above scheme stores query and answer pairs in vector form, which is convenient for subsequent retrieval and reuse. By updating the document index, the timeliness and accuracy of the vector library are maintained. By continuously updating the vector library, a feedback loop of queries and answers is formed, which improves the self-optimization ability of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0052] Figure 1 A flowchart of a multi-source intelligence fusion query method based on RAG Fusion is provided for one embodiment of the present invention.

[0053] Figure 2 A schematic diagram of the structure of a multi-source intelligence fusion query device based on RAG Fusion provided by one embodiment of the present invention.

[0054] The realization of the purpose, functional features and advantages of the invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the invention claimed for protection, but merely represents the selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0056] In the description of the present invention, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features.

[0057] The content of the present invention is described in detail below with reference to embodiments.

[0058] Reference Figure 1 The figure is a flow chart of a multi-source intelligence fusion query method based on RAG Fusion provided by an embodiment of the present invention.

[0059] This method can process and fuse different types of data sources (i.e., multi-source intelligence data), including structured data, unstructured text, images, and videos. For example, vector similarity matching: applicable to vectorized text or multimodal data, including unstructured text, pictures, videos, audio and other multimodal data; knowledge graph retrieval: querying structured relational data. The system retrieves relevant information or related documents from multiple data sources (such as vector libraries, knowledge graphs, text indexes, etc.), which means that the initial retrieval results already include data from different sources, and then adapt the retrieval strategy based on different data.

[0060] In this embodiment, the method includes:

[0061] S11, receiving original query data input by a user, performing pre-processing and expansion processing on the original query data to obtain an expanded query set.

[0062] Furthermore, in step S11, the original query data is preprocessed and extended to obtain an extended query set, including:

[0063] S11-1, using natural language processing technology to perform preprocessing including word segmentation, part-of-speech tagging and entity recognition on the original query data to obtain multiple word segmentation results;

[0064] S11-2, performing at least one expansion process including format, vocabulary, and expression on the word segmentation result to obtain multiple variant results, wherein the vocabulary expansion process includes synonym replacement, hyponym expansion variant results;

[0065] S11-3, based on multiple variant results, input the original query data into a pre-trained large language model to generate the expanded query set.

[0066] In this embodiment, the original query data is decomposed into separate words or tags by using natural language processing technology. For example, the original query data "research on radar absorbing technology of stealth fighter" is segmented to obtain "stealth", "fighter", "radar", "absorbing", "technology", and "research". Each word is annotated with its part of speech: ("stealth", adjective), ("fighter", noun), ("radar", noun), ("absorbing", noun), ("technology", noun), ("research", noun), and the named entities in the original query data are identified: equipment entity: ["fighter"], technical field entity: ["radar absorbing"], research direction entity: ["stealth technology"].

[0067] By adopting the expansion of synonym replacement, hyponym expansion, word vector model, multilingual query, abbreviation and full name conversion, knowledge graph reasoning, etc., multiple variant results are generated, specifically:

[0068] Synonym replacement: For example, using a synonym dictionary, "wave absorption" may be replaced by "anti-interference", "radar stealth" or "electromagnetic stealth";

[0069] Expansion of hypernyms and hyponyms: For example, "fighter jet" may be expanded to the hypernym "military aircraft", or the hyponym "fifth-generation fighter jet" or "stealth fighter jet";

[0070] Word embedding model: By finding words with similar semantics, such as "invisible" may be related to "stealth", "low detectability" or "electromagnetic invisibility".

[0071] Based on multiple variant results, the original query data input by the user is input into a pre-trained large language model (such as T5 or GPT), and the original query data is encoded into a vector representation through the model. A decoder is used to generate multiple extended queries with similar semantics but different expressions, that is, a set of extended query sets Q = {q1, q2, ..., qn} is obtained. Among them, the model can be fine-tuned on a specific task dataset to improve the quality of the rewriting. Furthermore, the diversity of the generated results can be increased by using strategies such as beam search and sampling. The generated multiple extended queries are screened and sorted to select the optimal extended query. The original query data "Research on radar wave absorption technology for stealth fighters" may be expanded to, for example: "Progress in radar reflection wave control technology for stealth fighters", "New breakthroughs in stealth fighter counter-radar technology", "Research on radar wave absorption technology for stealth fighter jets", "Radar signature management in stealth aircraft development", "Electromagnetic stealth technology in fifth-generation fighter aircraft".

[0072] The large language model is a typical pre-training model structure, including: input layer, word embeddings, positional encodings, segment encodings (for some models); Transformer layer, multi-head self-attention mechanism, feed-forward neural network, normalization layer, residual connection; output layer, depending on the task, may include language model head, classification head, etc.

[0073] S12, using multiple search strategies to search each extended query in the extended query set in a preset vector library, obtaining a document set corresponding to each search strategy, and reordering all documents in the document set to obtain a document list.

[0074] In step S12, the plurality of search strategies are used to search each extended query in the extended query set in a preset vector library to obtain a document set corresponding to each search strategy, including:

[0075] S12-1, matching each extended query in a preset vector library by vector similarity to obtain a first document;

[0076] S12-2, using the BM25 algorithm to calculate the relevance score between each extended query and the documents in the preset vector library, and taking the first N documents with the highest relevance score as the second document;

[0077] S12-3, perform entity recognition and linking for each extended query, determine the relevant knowledge graph nodes, and search the nodes and information associated with the entity in the knowledge graph to obtain the third document, and take the first document, the second document and the third document as the document set.

[0078] In this embodiment, the following search strategies are applied to each extended query qi to search in the vector library, including: semantic search based on dense vectors, keyword search based on sparse vectors (such as BM25), and entity association search based on knowledge graphs; each search strategy returns the Top-k related documents to obtain the document set D i,j , where i represents the extended query and j represents the retrieval strategy. Among them, dense vector semantic retrieval is to match the extended query by vector similarity (such as cosine similarity), and find the most relevant documents or information fragments in the vector library as the retrieval result. Sparse vector keyword retrieval is to calculate the relevance score between the extended query and the documents in the vector library by applying BM25 or similar algorithms, and obtain documents with high scores as retrieval results. Entity association retrieval of knowledge graph is to identify and link entities of the extended query, determine the relevant knowledge graph nodes, find other nodes and information associated with these entities in the knowledge graph, and return highly associated content as retrieval results. Furthermore, when the extended query contains clear keywords or phrases and the importance of the keywords is high, sparse vector-based keyword retrieval is used; when the extended query involves specific entities (such as names of people, places, organization names, etc.) or needs to find answers in complex entity relationships, knowledge graph-based entity association retrieval is used.

[0079] The construction of the vector library mainly includes steps such as data preprocessing, text segmentation, feature extraction and vectorization. These data can be text or pictures, images, or audio, satellite positioning and other intelligence data. First, the original intelligence data is cleaned and formatted, and then the long text is segmented into segments of appropriate size. Then, the pre-trained language model is used to extract the semantic features of the text and convert these features into a vector representation of fixed dimensions. Then, an index structure is built to support fast retrieval, and necessary metadata is associated with each vector. Finally, quality control and performance optimization are performed to ensure that the vector library can accurately and efficiently represent and retrieve the semantic information of the original data. This process converts the original intelligence data into a vector form that can be efficiently searched, providing basic information retrieval capabilities for the multi-source intelligence fusion query system.

[0080] Furthermore, in step S12, all documents in the document set are reordered to obtain a document list, including:

[0081] S12-4, using a cross encoder to calculate a relevance score between each document in the document set and the original query data;

[0082] S12-5, counting the number of times each document in the document collection appears in different extended queries and different retrieval strategies;

[0083] S12-6, based on the relevance score and the number of times, a weighted sum algorithm is used to calculate to obtain a score corresponding to each document, and all documents are reordered according to the score to obtain the document list.

[0084] In this embodiment, a cross encoder (such as BERT) is used to calculate the relevance score of each document in the document collection with the original query data, and a voting mechanism is applied to count the number of times each document appears in different extended queries and retrieval strategies. The relevance score and the number of occurrences are combined, and a weighted sum method is used to calculate the final score of the document. All documents are re-ranked according to the final score to obtain a comprehensively ranked document list D_ranked.

[0085] In the multi-source intelligence fusion query system, the voting mechanism is an important result integration method. This mechanism is mainly used to integrate multiple retrieval or ranking results to improve the accuracy and reliability of the final answer. The following is a brief description of the voting mechanism in the query process: The voting mechanism is mainly reflected in the query process as the integration of different retrieval strategies or model results. The system uses a variety of retrieval strategies (such as vector similarity, BM25 algorithm, knowledge graph retrieval, etc.) to retrieve relevant information, and each retrieval strategy will produce a candidate result. The voting mechanism determines the final result ranking by counting the frequency and ranking of these candidate results in different retrieval strategies. Specifically, the voting mechanism may include the following steps:

[0086] 1) Result summary: Collect candidate results of all search strategies, i.e., document collection;

[0087] 2) Frequency calculation: count the number of times each candidate result appears in different strategies;

[0088] 3) Ranking Fusion: Consider the ranking position of each candidate solution in each search strategy, using methods such as Borda counting;

[0089] 4) Weight assignment: assign different weights to the results of different search strategies according to their reliability or applicability;

[0090] 5) Threshold screening: set a minimum number of votes or score threshold to filter out low-relevance results;

[0091] 6) Final sorting: The results are sorted based on the comprehensive scores.

[0092] This voting mechanism can effectively integrate the advantages of multiple retrieval strategies, reduce the bias that may be caused by a single method, and thus provide more comprehensive and reliable query results. It is particularly suitable for processing complex multi-source intelligence queries, and can highlight the most relevant and credible information while ensuring the diversity of results.

[0093] S13, dynamically constructing contexts for the documents in the document list to generate a context document set with weights.

[0094] Furthermore, in step S13, the dynamic context construction is performed on the documents in the document list to generate a context document set with weights, including:

[0095] S13-1, selecting the first N documents from the document list, and constructing a relationship network for the selected first N documents using a graph model to obtain a document relationship grid;

[0096] S13-2, using the PageRank algorithm to calculate each document in the document relationship grid to obtain a PageRank value corresponding to each document, and determining a key document among all documents according to the PageRank value;

[0097] S13-3, dynamically adjusting the weight of each document in the context according to the importance between the key documents and the relevance of the documents in the document set, to obtain the context document set.

[0098] In this embodiment, the top-N documents are selected from the document list D_ranked, and a relational network between documents is constructed using a graph model, and the edge weight represents the document similarity. Among them, the construction process of the relational network between documents using a graph model includes: converting each document into a dense vector representation, the vector can capture the semantic features of the document, and these features will be used for subsequent similarity calculations; constructing a graph model, each document (or a fragment of a document) as a node in the graph, creating edges between nodes, and the existence of edges indicates that there is a certain relationship or similarity between documents; calculating the weight of the edge, calculating the cosine similarity between document vectors, the higher the similarity, the closer the semantics between the documents, and the calculated similarity value is used as the weight of the edge. The higher the similarity value, the greater the weight of the edge, indicating that the relationship between the two documents is stronger. The data structure of a graph constructed in this way represents the relational network between documents.

[0099] Apply the PageRank algorithm to the relational network built using the graph model, calculate the PageRank value of each document, and use the PageRank algorithm to identify key documents. According to the calculated PageRank value corresponding to each document, all documents are sorted according to the PageRank value. Documents with high PageRank values ​​are considered key documents because they have more links in the relational network or are linked from important documents. According to the importance between key documents and the relevance of documents in the document collection (referring to the similarity between multiple result documents in the document collection), the weight of each document in the context is dynamically adjusted to generate a weighted context document set D. context .

[0100] (1) Build a document relationship diagram:

[0101] Define the basic elements of the graph model, V: document collection, containing n documents {d1, d2, ..., d n}, E: edge set, representing the link relationship between documents. Using text similarity metrics (such as cosine similarity, Jaccard similarity, etc.), edges can be established between document pairs with similarity above a certain threshold: E = {(d i , d j )|S(di , d j )>threshold}, where S(d i , d j ) is the document (d i , d j ) is a similarity measure between .

[0102] (2) The PageRank algorithm formula is:

[0103] Among them, PR(d i ) indicates the document (d i )’s PageRank value, d represents the damping coefficient, which is usually 0.85, L(d j ) represents all documents d pointing to document di j The set of C(d j ) indicates document d j The number of outgoing links, i.e., from d j The number of links to other documents.

[0104] (3) Calculate the initial PageRank value:

[0105] Initially, the PageRank values ​​of all documents can be set to the same value, that is, 1.

[0106] (4) Iterative calculation:

[0107] Iterate according to the above PageRank formula until convergence, convergence conditions:

[0108] Among them, ∈ is a small threshold indicating the accuracy of convergence, and t is the number of iterations.

[0109] (5) Document sorting:

[0110] Sort all documents according to the calculated PageRank value:

[0111] Sorting result = sort(PR(d1), PR(d2), ..., PR(d n )).

[0112] (6) Dynamically adjust document weights:

[0113] Dynamically adjust the document weight according to the importance and relevance between key documents, which can be expressed by the following formula:

[0114] W(d i )=αα·PR(d i )+β·S(d i );

[0115] Where: W(d i ): Document d i The weighted value, S(d i ): Document d i The similarity with other documents can usually be calculated by cosine similarity, etc. a and β: weight coefficients, ensuring that a+β=1.

[0116] (7) Generate a weighted context document set:

[0117] Finally, a weighted context document set D is generated context :

[0118]

[0119] S14, performing sentence-level segmentation on each document in the context document set, and using a pre-trained question-answering model to extract fragments related to the original query data from the segmented sentences, de-duplicating and merging the extracted fragments to generate a set of information units.

[0120] In this embodiment, each document in the context document set D_context is segmented at the sentence level, and a pre-trained question-answering model (such as BERT-QA; this question-answering model is used as a component in the retrieval-augmented generation (RAG) system) is used to extract fragments related to the original query data from each sentence. The extracted fragments are deduplicated and merged to obtain a set of information units I = {i1, i2, ..., im}.

[0121] S15, inputting the original query data, the context document set, and the information unit set into a pre-trained large-scale language model to generate a target answer.

[0122] Furthermore, in step S15, the inputting of the original query data, the context document set, and the information unit set into a pre-trained large-scale language model to generate a target answer includes:

[0123] S15-1, inputting the original query data, the context document set, and the information unit set into a pre-trained large-scale language model to obtain an initial answer;

[0124] S15-2, based on the initial answer, inputting the constructed prompt word into a large-scale language model to generate an answer to be determined that references information in the context document set;

[0125] S15-3, using the information unit set to perform a fact check on the answer to be determined to obtain the target answer.

[0126] In this embodiment, the original query data, the context document set D context and the information unit set I are taken as input, and a pre-trained large-scale language model (such as GPT) is used to generate a preliminary answer; a controlled generation technology is applied (i.e., specific prompt words or templates are introduced to guide the generation, and post-processing rules are applied to filter or modify the generated content, so as to provide a more accurate and well-founded answer to the original query data) to ensure that the generated answer accurately refers to the information in the context document set D_context; the information unit set I is used to perform factual checks (such as errors in dates, numbers, names, historical sequences, scientific facts, logical errors, irrelevant answers, etc.), correct possible errors, and generate the final target answer A.

[0127] In another embodiment, the method further comprises:

[0128] S16, taking the original query data and the target answer as a query-answer pair, vectorizing the query-answer pair to store it in the preset vector library and updating the document index.

[0129] In this embodiment, by recording user feedback and query-answer pairs, the document index and embedding model are updated regularly. This process involves collecting and analyzing the interaction data between the user and the system. By recording the intelligence query input by the user, the system's answer and the user's feedback (such as clicks, dwell time, explicit ratings, etc.), the system can accumulate a large amount of data for subsequent optimization. In order to maintain the timeliness and accuracy of the system, it is necessary to regularly update the document index: ensure that the retrieval library contains the latest and most relevant information, and update the embedding model: update the semantic understanding ability to better handle the emerging language patterns and concepts, and store intelligence source data such as information, pictures, videos, etc. through the embedded model (Embedding). It represents the original complex data as a point in a low-dimensional vector space, so that the data can be processed and stored more efficiently in the vector space. The role of the embedding model here is to convert the original unstructured data into a vector form through the model, which is convenient for subsequent retrieval, analysis and processing. In this way, different types of data (text, pictures, videos, etc.) can be represented by a unified vector, thereby realizing cross-modal query and information processing.

[0130] We also use reinforcement learning to optimize the parameters of the retrieval strategy and fusion algorithm. This is a continuous optimization process that uses reinforcement learning to adjust the core components of the system, including: retrieval strategy: optimize the method of document retrieval and improve the recall rate of relevant documents; fusion algorithm: adjust how to effectively combine information from different sources. By continuously learning and adjusting these parameters, the system can gradually improve its performance.

[0131] Adapt to new domains and query types through few-sample learning methods. For emerging professional fields or special types of intelligence queries, the system can quickly learn and adapt through a small number of samples. This enables the RAG Fusion system to flexibly respond to changing user needs and emerging knowledge areas.

[0132] Reference Figure 2 The figure is a schematic diagram of the structure of a multi-source intelligence fusion query device based on RAG Fusion provided by an embodiment of the present invention.

[0133] In this embodiment, the device 20 includes:

[0134] The data processing unit 21 is used to receive the original query data input by the user, and perform pre-processing and expansion processing on the original query data to obtain an expanded query set;

[0135] The document retrieval unit 22 is used to use multiple retrieval strategies to search each extended query in the extended query set in a preset vector library to obtain a document set corresponding to each retrieval strategy, and reorder all documents in the document set to obtain a document list;

[0136] A context construction unit 23, configured to construct a dynamic context for the documents in the document list to generate a context document set with weights;

[0137] An information generating unit 24 is used to segment each document in the context document set at the sentence level, extract segments related to the original query data from the segmented sentences using a pre-trained question-answering model, deduplicate and merge the extracted segments, and generate an information unit set;

[0138] The answer generation unit 25 is used to input the original query data, the context document set and the information unit set into a pre-trained large-scale language model to generate a target answer.

[0139] The device 20 also includes:

[0140] An updating unit is used to take the original query data and the target answer as a query-answer pair, vectorize the query-answer pair to store it in the preset vector library and update the document index.

[0141] Each unit module of the device 20 can respectively execute the corresponding steps in the above method embodiment, so each unit module will not be described in detail here. Please refer to the description of the above corresponding steps for details.

[0142] The embodiment of the present invention further provides a multi-source intelligence fusion query device based on RAG Fusion, the device comprising the multi-source intelligence fusion query device based on RAG Fusion as described above, wherein the multi-source intelligence fusion query device based on RAG Fusion can adopt Figure 2 The structure of the embodiment can be executed accordingly. Figure 1 The technical solution of the method embodiment shown has similar implementation principles and technical effects. For details, please refer to the relevant records in the above embodiments, which will not be repeated here.

[0143] The device includes: a mobile phone, a digital camera, a tablet computer, or other devices with a camera function, or a device with an image processing function, or a device with an image display function. The device may include components such as a memory, a processor, an input unit, a display unit, and a power supply.

[0144] Among them, the memory can be used to store software programs and modules, and the processor executes various functional applications and data processing by running the software programs and modules stored in the memory. The memory may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as an image playback function, etc.), etc.; the data storage area may store data created according to the use of the device, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory may also include a memory controller to provide the processor and the input unit with access to the memory.

[0145] The input unit can be used to receive input digital or character or image information, and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control. Specifically, the input unit of this embodiment includes not only a camera, but also a touch-sensitive surface (such as a touch display) and other input devices.

[0146] The display unit can be used to display information input by the user or information provided to the user and various graphical user interfaces of the device, which can be composed of graphics, text, icons, videos and any combination thereof. The display unit may include a display panel, and optionally, the display panel may be configured in the form of LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode), etc. Further, the touch-sensitive surface may cover the display panel, and when the touch-sensitive surface detects a touch operation on or near it, it is transmitted to the processor to determine the type of touch event, and then the processor provides a corresponding visual output on the display panel according to the type of touch event.

[0147] The embodiment of the present invention further provides a computer-readable storage medium, which may be a computer-readable storage medium included in the memory in the above embodiment; or a computer-readable storage medium that exists independently and is not installed in a device. The computer-readable storage medium stores at least one instruction, which is loaded and executed by a processor to implement Figure 1 The multi-source intelligence fusion query method based on RAG Fusion is shown. The computer-readable storage medium may be a read-only memory, a disk or an optical disk, etc.

[0148] The embodiment of the present invention also provides a computer program product, including a computer program / instruction, which is loaded and executed by a processor to implement Figure 1 A multi-source intelligence fusion query method based on RAG Fusion is shown.

[0149] It should be noted that each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the embodiments can be referred to each other. For the device embodiment, equipment embodiment and storage medium embodiment, since they are basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0150] Furthermore, in this document, the terms "comprises," "comprising," or any other variation thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or device that includes the element.

[0151] The above description shows and describes the preferred embodiments of the present invention. It should be understood that the present invention is not limited to the form disclosed herein, and should not be regarded as excluding other embodiments, but can be used in various other combinations, modifications and environments, and can be modified within the scope of the invention, through the above teachings or the technology or knowledge of the relevant field. The changes and modifications made by those skilled in the art shall not depart from the spirit and scope of the present invention, and shall be within the scope of protection of the claims attached to the present invention.

Claims

1. A multi-source intelligence fusion query method based on RAG Fusion, characterized in that: The method comprises: Receiving original query data input by a user, performing preprocessing and expansion processing on the original query data to obtain an expanded query set; Using multiple search strategies to search each extended query in the extended query set in a preset vector library, obtaining a document set corresponding to each search strategy, and reordering all documents in the document set to obtain a document list; Dynamically constructing contexts for the documents in the document list to generate a context document set with weights; Segment each document in the context document set at the sentence level, extract segments related to the original query data from the segmented sentences using a pre-trained question-answering model, deduplicate and merge the extracted segments, and generate an information unit set; The original query data, the context document set, and the information unit set are input into a pre-trained large-scale language model to generate a target answer.

2. According to the multi-source intelligence fusion query method based on RAG Fusion in claim 1, it is characterized in that: The preprocessing and expansion processing of the original query data to obtain an expanded query set includes: Using natural language processing technology to pre-process the original query data including word segmentation, part-of-speech tagging and entity recognition to obtain multiple word segmentation results; Performing at least one expansion process including format, vocabulary, and expression on the word segmentation result to obtain multiple variant results, wherein the vocabulary expansion process includes synonym replacement and hyponym expansion; Based on the multiple variant results, the original query data is input into a pre-trained large language model to generate the expanded query set.

3. The multi-source intelligence fusion query method based on RAG Fusion according to claim 1 is characterized in that: The method of using multiple search strategies to search each extended query in the extended query set in a preset vector library to obtain a document set corresponding to each search strategy includes: Match each extended query in a preset vector library by vector similarity to obtain a first document; The BM25 algorithm is used to calculate the relevance score between each extended query and the documents in the preset vector library, and the first N documents with the highest relevance score are used as the second document; Perform entity recognition and linking on each extended query, determine the relevant knowledge graph nodes, and search the knowledge graph for nodes and information associated with the entity to obtain a third document, and use the first document, the second document, and the third document as the document set.

4. The multi-source intelligence fusion query method based on RAG Fusion according to claim 1 is characterized in that: The step of reordering all documents in the document set to obtain a document list includes: Calculate the relevance score of each document in the document collection to the original query data using a cross encoder; Counting the number of times each document in the document collection appears in different extended queries and different retrieval strategies; Based on the relevance score and the number of times, a weighted sum algorithm is used to calculate to obtain a score corresponding to each document, and all documents are reordered according to the score to obtain the document list.

5. The multi-source intelligence fusion query method based on RAG Fusion according to claim 1 is characterized in that: The dynamically constructing contexts of the documents in the document list to generate a context document set with weights includes: Selecting the first N documents from the document list, and constructing a relationship network for the selected first N documents using a graph model to obtain a document relationship grid; Calculate each document in the document relationship grid using the PageRank algorithm to obtain a PageRank value corresponding to each document, and determine a key document among all documents according to the PageRank value; The weight of each document in the context is dynamically adjusted according to the importance between the key documents and the relevance of the documents in the document set to obtain the context document set.

6. The multi-source intelligence fusion query method based on RAG Fusion according to claim 1 is characterized in that: The step of inputting the original query data, the context document set, and the information unit set into a pre-trained large-scale language model to generate a target answer includes: Inputting the original query data, the context document set, and the information unit set into a pre-trained large-scale language model to obtain an initial answer; Based on the initial answer, inputting the constructed prompt word into a large-scale language model to generate a to-be-determined answer that references information in the context document set; The information unit set is used to perform a fact check on the answer to be determined to obtain the target answer.

7. The multi-source intelligence fusion query method based on RAG Fusion according to claim 1 is characterized in that: The method further comprises: The original query data and the target answer are used as query-answer pairs, the query-answer pairs are vectorized to be stored in the preset vector library, and the document index is updated.

8. A multi-source intelligence fusion query device based on RAG Fusion, characterized in that: The device comprises: A data processing unit, configured to receive original query data input by a user, and perform preprocessing and expansion processing on the original query data to obtain an expanded query set; A document retrieval unit, configured to use a plurality of retrieval strategies to retrieve each extended query in the extended query set in a preset vector library, obtain a document set corresponding to each retrieval strategy, and reorder all documents in the document set to obtain a document list; A context construction unit, used to dynamically construct contexts for the documents in the document list to generate a context document set with weights; An information generation unit, configured to segment each document in the context document set at a sentence level, extract segments related to the original query data from the segmented sentences using a pre-trained question-answering model, deduplicate and merge the extracted segments, and generate an information unit set; The answer generation unit is used to input the original query data, the context document set and the information unit set into a pre-trained large-scale language model to generate a target answer.

9. A multi-source intelligence fusion query device based on RAG Fusion, characterized in that: It includes a processor, a memory and a computer program stored in the memory, wherein the computer program is executed by the processor to implement the steps of a multi-source intelligence fusion query method based on RAG Fusion as described in any one of claims 1 to 7.

10. A computer program product, comprising a computer program / instruction, wherein the computer program / instruction is executed by a processor to implement the steps of a multi-source intelligence fusion query method based on RAG Fusion as described in any one of claims 1 to 7.

Images