A context-enhanced query fusion system

Abstract

The present invention discloses a context-enhanced query fusion system, comprising: an original query processing module, the original query processing module is used to receive the original query Q input by the user and pre-process it; a contextualized query generation module, which is connected to the original query processing module in communication, and is used to receive the pre-processed original query Q to generate multiple contextualized queries Qgen; a query re-ranking module, which is connected to the contextualized query generation module in communication; a document retrieval module, which is connected to the query re-ranking module in communication; and a document fusion module, which is connected to the document retrieval module in communication. The context-enhanced query fusion system of the present invention meets the needs of engineers, customers and account managers to quickly obtain product information through the context-enhanced query fusion system.

Description

Technical Field

[0001] The present invention relates to the fields of artificial intelligence (AI) and natural language processing (NLP), and more specifically to a context-enhanced query fusion system. Background Art

[0002] At present, industries such as e-commerce, manufacturing, and retail often come into contact with a large number of products in the course of their work. Correspondingly, a large number of products will have a large amount of product information. For this reason, they generally choose to set up a database to store a large amount of product information, and then set up a query system to facilitate the query of product information. However, most of the query systems in the existing technology currently adopt a one-to-one query method, that is, after the user inputs relevant information, the query system queries the product information based on the relevant information. However, after receiving the relevant information input by the user, the existing query system does not use a simple information matching method to complete the query work. Generally speaking, the relevant information input by the user is often not accurate enough. Therefore, the answers provided by the existing query system are not accurate and comprehensive enough. Summary of the invention

[0003] In view of the deficiencies in the prior art, the purpose of the present invention is to provide a context-enhanced query fusion system (CEQFS), which aims to provide users with more accurate, comprehensive and relevant answers by generating multiple contextualized queries, re-ranking and document fusion.

[0004] To achieve the above object, the present invention provides the following technical solution: a context-enhanced query fusion system, characterized in that it includes:

[0005] An original query processing module, which is used to receive an original query Q input by a user and pre-process it;

[0006] A contextualized query generation module, which is in communication with the original query processing module and is used to receive the pre-processed original query Q and generate multiple contextualized queries Qgen;

[0007] A query re-ranking module, which is in communication with the contextualized query generation module, is used to receive the generated multiple contextualized queries Qgen, and sort the generated multiple contextualized queries Qgen using a re-ranking function RRF;

[0008] A document retrieval module, in communication with the query re-ranking module, for receiving the ranked query Qgen contextualized query Qgen set, and retrieving relevant documents from the document set D;

[0009] The document fusion module is in communication with the document retrieval module and is used to fuse the retrieved related documents using the document fusion function F to generate a final answer.

[0010] As a further improvement of the present invention, the preprocessing of the original query processing module includes operations such as word segmentation, removal of stop words, and part-of-speech tagging, so as to provide a basis for subsequent query generation and processing.

[0011] As a further improvement of the present invention, the specific steps of the contextual query generation module generating multiple contextual queries Qgen are as follows:

[0012] Step 1: Use the word vector model to generate queries that are semantically similar to the original query for semantic expansion;

[0013] Step 2: Combine domain knowledge base or context information to supplement the missing information in the original query to complete context supplement;

[0014] Step three, interpret and expand the original query from different angles to generate a variety of different query forms to fully cover the user's query intention.

[0015] As a further improvement of the present invention, the specific steps of the query re-ranking module to sort the generated multiple contextualized queries Qgen are as follows:

[0016] Step 4: Calculate the similarity score between each generated contextualized query Qgen and the original query Q;

[0017] Step 5: Sort the generated multiple contextualized queries Qgen according to the similarity scores calculated in step 4 to obtain a similarity score set S.

[0018] As a further improvement of the present invention, the specific steps of the document fusion module fusing related documents are as follows:

[0019] Step 6: Remove duplicate documents;

[0020] Step 7: merge the documents retrieved by different queries and re-rank them according to the similarity scores;

[0021] Step 8: Extract key information from the fused documents and generate the final answer.

[0022] As a further improvement of the present invention, it also includes:

[0023] The context similarity matching module is connected with the contextualized query generation module and the query re-ranking module. It is used to always consider the semantic similarity between the contextualized query Qgen and the original query Q during the query generation and re-ranking process to avoid the generated query from deviating from the topic.

[0024] As a further improvement of the present invention, the specific steps for the context similarity matching module to always consider the semantic similarity between the contextualized query Qgen and the original query Q are as follows: first, a pre-trained language model is used to generate semantic embeddings of the contextualized query Qgen and the original query Q, and their similarity is calculated. Then, when generating the contextualized query Qgen, it is checked whether the contextualized query Qgen contains context information consistent with the original query Q, to ensure that the generated query can accurately reflect the user's intention.

[0025] The beneficial effect of the present invention is that, through the setting of the original query processing module, the original query Q can be preprocessed, and through the setting of the contextualized query generation module, multiple contextualized queries Qgen can be generated based on the original query Q, and then the query re-ranking module uses the re-ranking function RRF to sort the generated multiple contextualized queries Qgen, and then through the setting of the document retrieval module, relevant documents can be effectively retrieved from the document set D, and finally the document fusion module uses the document fusion function F to fuse the retrieved relevant documents to generate the final answer. Compared with the query system in the prior art, the above process provides users with more accurate, comprehensive and relevant answers by generating multiple contextualized queries, re-ranking and document fusion, so as to provide efficient, accurate and comprehensive information retrieval services to meet the needs of users to quickly obtain product information. Through the context-enhanced query fusion system, the quality and relevance of information retrieval results are improved. Bring innovation to the fields of artificial intelligence and natural language processing, and promote application transformation in the context of multiple industries around the world. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 A schematic diagram of modules in a context-enhanced query fusion system of the present invention. DETAILED DESCRIPTION

[0027] The present invention will be further described in detail with reference to the given embodiments below.

[0028] A context-enhanced query fusion system according to this embodiment includes:

[0029] 1. Original query processing module

[0030] The original query processing module receives the original query Q input by the user and preprocesses it. The preprocessing includes operations such as word segmentation, removal of stop words, and part-of-speech tagging, so as to provide a basis for subsequent query generation and processing.

[0031] 2. Contextualized query generation module

[0032] Based on the original query processing, the contextualized query generation module generates multiple contextualized queries Qgen. These queries are generated in the following ways:

[0033] Semantic expansion: Use word vector models (such as Word2Vec, BERT, etc.) to generate queries that are semantically similar to the original query Q.

[0034] Contextual supplementation: Combine domain knowledge base or contextual information to supplement the missing information in the original query. For example, for queries in the technical field, relevant technical terms or background information can be added.

[0035] Multi-angle perspective: The original query is interpreted and expanded from different angles to generate a variety of different query forms to fully cover the user's query intention.

[0036] 3. Query re-ranking module

[0037] There may be a large number of generated contextual queries Qgen. In order to improve the relevance and accuracy of the query results, the query re-ranking module uses the re-ranking function RRF to sort the generated contextual queries Qgen. The specific steps are as follows:

[0038] Similarity calculation: Calculate the similarity score between each contextualized query Qgen and the original query Q. Similarity calculation can be based on methods such as cosine similarity and Euclidean distance.

[0039] Re-ranking: Sort the generated multiple contextualized queries Qgen according to the similarity scores to obtain a similarity score set S.

[0040] 4. Document Retrieval Module

[0041] The document retrieval module uses the sorted similarity score set S of the contextualized query Qgen to retrieve relevant documents from the document set D. Each contextualized query Qgen retrieves a batch of relevant documents and preliminarily sorts them according to the similarity score.

[0042] 5. Document fusion module

[0043] The document fusion module uses the document fusion function F to fuse the retrieved documents and generate the final answer. The steps of document fusion include:

[0044] Document deduplication: Remove duplicate documents to ensure diversity of results.

[0045] Document merging: Merge documents retrieved from different queries and re-sort them according to similarity scores to ensure that high-quality documents are displayed first.

[0046] Answer generation: Extract key information from the fused documents to generate the final answer. The extraction method can include summary generation, key sentence extraction and other technologies.

[0047] 6. Contextual Similarity Matching Mechanism

[0048] In order to ensure the relevance of the generated contextualized query Qgen to the original query Q, the system introduces a contextual similarity matching mechanism. This mechanism always considers the semantic similarity between the contextualized query Qgen and the original query Q during the generation and re-ranking of the contextualized query Qgen, and avoids the generated contextualized query Qgen from deviating from the topic. The specific methods include:

[0049] Semantic embedding matching: Use pre-trained language models (such as BERT, GPT, etc.) to generate semantic embeddings of the contextualized query Qgen and the original query Q, and calculate their similarity.

[0050] Context consistency check: When generating the contextualized query Qgen, check whether the contextualized query Qgen contains context information that is consistent with the original query Q, to ensure that the generated contextualized query Qgen can accurately reflect the user's intention.

[0051] In this embodiment, the system uses the following variables and formulas: Q: original query, Qgen: generated contextualized query set, D: document set, S: similarity score set, RRF: re-ranking function, F:

[0052] Document fusion function, the system workflow is as follows:

[0053] Multiple contextualized queries Qgen are generated based on different contexts to enrich the meaning of the original query Q.

[0054] The generated query set is sorted using the re-ranking function RRF to obtain a similarity score set S.

[0055] The document set D is fused using the document fusion function F, and the similarity score set S is considered to generate the final answer.

[0056] The above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

Claims

1. A context-enhanced query fusion system, characterized by: include: An original query processing module, which is used to receive an original query Q input by a user and pre-process it; A contextualized query generation module, which is in communication with the original query processing module and is used to receive the pre-processed original query Q and generate multiple contextualized queries Qgen; A query re-ranking module, which is in communication with the contextualized query generation module, is used to receive the generated multiple contextualized queries Qgen, and sort the generated multiple contextualized queries Qgen using a re-ranking function RRF; A document retrieval module, in communication with the query re-ranking module, for receiving the ranked query Qgen contextualized query Qgen set, and retrieving relevant documents from the document set D; The document fusion module is in communication with the document retrieval module and is used to fuse the retrieved related documents using the document fusion function F to generate a final answer.

2. The context-enhanced query fusion system according to claim 1, characterized in that: The preprocessing of the original query processing module includes operations such as word segmentation, removal of stop words, and part-of-speech tagging, so as to provide a basis for subsequent query generation and processing.

3. The context-enhanced query fusion system according to claim 1 or 2, characterized in that: The specific steps of the contextual query generation module generating multiple contextual queries Qgen are as follows: Step 1: Use the word vector model to generate queries that are semantically similar to the original query for semantic expansion; Step 2: Combine domain knowledge base or context information to supplement the missing information in the original query to complete context supplement; Step three, interpret and expand the original query from different angles to generate a variety of different query forms to fully cover the user's query intention.

4. The context-enhanced query fusion system according to claim 1 or 2, characterized in that: The specific steps of the query re-ranking module to sort the generated multiple contextualized queries Qgen are as follows: Step 4: Calculate the similarity score between each generated contextualized query Qgen and the original query Q; Step 5: Sort the generated multiple contextualized queries Qgen according to the similarity scores calculated in step 4 to obtain a similarity score set S.

5. The context-enhanced query fusion system according to claim 1 or 2, characterized in that: The specific steps of the document fusion module to fuse related documents are as follows: Step 6: Remove duplicate documents; Step 7: merge the documents retrieved by different queries and re-rank them according to the similarity scores; Step 8: Extract key information from the fused documents and generate the final answer.

6. The context-enhanced query fusion system according to claim 1 or 2, characterized in that: Also includes: The context similarity matching module is connected with the contextualized query generation module and the query re-ranking module. It is used to always consider the semantic similarity between the contextualized query Qgen and the original query Q during the query generation and re-ranking process to avoid the generated query from deviating from the topic.

7. The context-enhanced query fusion system according to claim 6, characterized in that: The specific steps for the context similarity matching module to always consider the semantic similarity between the contextualized query Qgen and the original query Q are as follows: first, a pre-trained language model is used to generate semantic embeddings of the contextualized query Qgen and the original query Q, and their similarity is calculated. Then, when generating the contextualized query Qgen, it is checked whether the contextualized query Qgen contains context information consistent with the original query Q, to ensure that the generated query can accurately reflect the user's intention.

Images