Double-retrieval model generated based on extension problem

Abstract

The invention relates to the technical field of language models, in particular to a double-retrieval model generated based on extension questions.The double-retrieval model comprises the following steps that S1, query standardization is conducted on query of a user through a 5W1H framework; s2, a sentence converter encodes each standard and extended query as an embedding, and the embedding is combined into a single group embedding by using mean pooling. According to the double-retrieval model generated based on the extended questions, the retrieval precision of question answers based on LLM (Large Language Model) is improved and the illusion is reduced through the HIYO-Encoder double-retrieval model, and the HIYO-Encoder is combined with an element to overcome the limitation of a traditional RAG model, namely, double-retrieval design: the retrieval between direct search and extended search is balanced, and the retrieval efficiency is improved. In one embodiment, the method comprises the following steps of: establishing a database using context and query embedding so as to enhance response relevance and reduce hallusion, enriching the retrieval through 5W1H sub-problems so as to capture additional semantic depth in a user search, constructing the database using context and query embedding, and thus allowing multi-dimensional retrieval alignment to obtain maximum accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of language models, and in particular to a dual retrieval model generated based on extended questions. Background Art

[0002] Using the Retrieval-Augmented Generation (RAG) technique, the performance of large language models has shown promise.

[0003] However, traditional RAG methods mostly rely on users' direct search, ignoring the complex semantic connection between query and retrieval text, which reduces the accuracy of retrieval and increases the risk of hallucination. Summary of the invention

[0004] The purpose of the present invention is to provide a dual retrieval model based on extended question generation to solve the problem that the traditional RAG method proposed in the above background technology mostly relies on the direct search of the user, ignoring the complex semantic connection between the query and the retrieval text, thereby reducing the accuracy of the retrieval and increasing the risk of hallucination. To achieve the above purpose, the present invention provides the following technical solution: A dual retrieval model based on extended question generation, comprising the following steps:

[0005] S1: User queries are standardized using the 5W1H framework;

[0006] S2: The sentence transformer encodes each standard and expanded query into an embedding, which is merged into a single group embedding by using mean pooling;

[0007] S3: HIYO-Encoder dual retrieval model uses a hierarchical database structure with an additional generative query layer for retrieval;

[0008] The HIYO-Encoder dual retrieval model pipeline simplifies user queries into short words that convey core topics and important concepts. This stage separates basic ideas and removes irrelevant information.

[0009] The 5W1H framework is used to capture a range of semantic nuances;

[0010] The HIYO-Encoder dual retrieval model includes content embedding and query embedding, wherein the content embedding is used to represent the content of the document, and the query embedding is used to encode the sub-queries generated by 5w1h connected to each document to expand the searchable materials.

[0011] Compared with the prior art, the present invention has the following beneficial effects:

[0012] In the present invention, the retrieval accuracy of LLM (large language model) based question answering is improved and hallucinations are reduced through the HIYO-Encoder dual retrieval model. The HIYO-Encoder combines an element to overcome the limitations of the traditional RAG model, namely the dual retrieval design: balancing the retrieval between direct and extended search to enhance response relevance and reduce hallucinations, enriching the retrieval through 5W1H sub-questions to capture additional semantic depth in user searches, and building a database using context and query embeddings, thereby allowing multi-dimensional retrieval alignment for maximum accuracy.

[0013] In this paper, the F1 and ExactMatch indicators show that the HIYO-Encoder dual retrieval model outperforms the baseline model in retrieval accuracy and hallucination reduction on multiple QA datasets, in which query expansion and dual retrieval components play a key role. Its computational efficiency also makes it suitable for real-time applications. Further research will focus on enhancing its dual retrieval system with adaptive techniques to support high-precision, context-aware QA capabilities. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 It is a flowchart of the HIYO-Encoder dual retrieval model of the present invention;

[0015] Figure 2 The test result table of the first embodiment of the present invention is Figure 1 ;

[0016] Figure 3 This is a bar graph of test results of Example 1 of the present invention;

[0017] Figure 4 The test result table of the first embodiment of the present invention is Figure 2 ;

[0018] Figure 5 This is a test result curve diagram of Example 1 of the present invention. DETAILED DESCRIPTION

[0019] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technical personnel in this field without creative work are within the scope of protection of the present invention.

[0020] Embodiment 1

[0021] Test equipment

[0022] The setting includes different baseline models, carefully selected datasets, and standard evaluation metrics to ensure a thorough performance evaluation. For baseline comparison, the HIYO-Encoder dual retrieval model is benchmarked by several state-of-the-art retrieval methods, including BM25, Contriever, DPR, FAISS, and random baselines, and three widely adopted question-answering datasets (TriviaQA, HotpotQA, and SQuADv2). All datasets are standardized to the SQuAD format. The generative model used in conjunction with the retrieval method is gpt40-mini.

[0023] Testing Procedure

[0024] The baseline RAG model serves as a control group, a HIYO-Encoder dual retrieval model variant without the group retrieval component to study the impact of the mechanism, another version without the query retrieval module to analyze its key role, and the full HIYO-Encoder dual retrieval model approach with all components;

[0025] The test results of the HIYO-Encoder dual retrieval model can be used to know the performance of the HIYO-Encoder dual retrieval model against the baseline retrieval method on three datasets, using F1 and exact match (EM) scores as indicators;

[0026] Depend on Figure 2 and Figure 3 It is known that on the TriviaQA dataset, the HIYO-Encoder dual retrieval model achieved the highest F1 score of 89.74 and EM score of 81.53, which is significantly better than other methods. Similarly, in the HotpotQA dataset, the HIYO-Encoder dual retrieval model achieved an F1 score of 79.59 and an EM score of 67.68. For the SQuADv2 dataset, the HIYO-Encoder dual retrieval model once again topped the list with an F1 score of 77.34 and an EM score of 62.31, indicating its robustness in different types of QA tasks, even in datasets that require complex or multi-hop reasoning, such as TirviaQA and HotpotQA, it shows a particularly increased effect, which shows that the HIYO-Encoder dual retrieval model is able to capture subtle query features;

[0027] The ablation results show that the contribution of the HIYO-Encoder dual retrieval model mainly comes from:

[0028] Query expansion: The HIYO-Encoder dual retrieval model improves retrieval alignment with user intent by creating expanded searches that capture additional viewpoints;

[0029] Dual retrieval mechanism: Through hierarchical encoding and balanced retrieval between extended search and original search, the incidence of hallucinations is greatly reduced, and the responses produced are more consistent with the expected context. Although the HIYO-Encoder dual retrieval model adds more query processing stages, the retrieval time of all models is kept within a reasonable range, thus supporting their relevance in real-time QA environments without sacrificing computational efficiency;

[0030] In-depth analysis of comparative experiments: The F1 score and EM score of the HIYO-Encoder dual retrieval model on the three datasets are significantly higher than those of other methods such as BM25, Contriever, and DPR, which shows that the HIYO-Encoder dual retrieval model has a stronger ability to understand questions, retrieve relevant documents, and generate answers. It is worth noting that the HIYO-Encoder dual retrieval model still maintains high performance on complex datasets (such as hot pot), which shows that this method is not only suitable for simple QA tasks, but also can handle complex problems that require multi-step reasoning. Compared with traditional retrieval methods such as BM25, the HIYO-Encoder dual retrieval model demonstrates the advantages of deep learning models in semantic understanding;

[0031] In-depth analysis of ablation experiments: The ablation experiments clearly illustrate the importance of each module in the HIYO-Encoder dual retrieval model. For example, removing the query retrieval module leads to a performance drop on all datasets, which indicates that query retrieval is a key component of the HIYO-Encoder dual retrieval model. Default-RAG outperforms any variant that removes a specific module, proving that there is a positive combination effect between the modules of the HIYO-Encoder dual retrieval model. Each module contributes uniquely to the overall performance, and they enhance each other when working together.

[0032] The above shows and describes the basic principles, main features and advantages of the present invention. Technical personnel in this industry should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only preferred examples of the present invention and are not used to limit the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A dual retrieval model based on extended question generation, characterized in that: The steps include: S1: User queries are standardized using the 5W1H framework; S2: The sentence transformer encodes each standard and expanded query into an embedding, which is merged into a single group embedding by using mean pooling; S3: HIYO-Encoder dual retrieval model uses a hierarchical database structure with an additional generative query layer for retrieval; The HIYO-Encoder dual retrieval model pipeline simplifies user queries into short words that convey core topics and important concepts. This stage separates basic ideas and removes irrelevant information. The 5W1H framework is used to capture a range of semantic nuances; The HIYO-Encoder dual retrieval model includes content embedding and query embedding, wherein the content embedding is used to represent the content of the document, and the query embedding is used to encode the sub-queries generated by 5w1h connected to each document to expand the searchable materials.

Images