Query method, system, device and medium based on iterative retrieval generation verification

Abstract

The present disclosure provides a query method, system, device and medium based on iterative retrieval generation verification, which belongs to the field of data verification query technology. The method includes: based on the user query statement, using a large language model to obtain an initial expanded query. Based on the initial expanded query and the retrieval model, an initial retrieval document set is obtained from the corpus. The user query statement and the initial retrieval document set are filled into a text prompt template to obtain text prompt information. Based on the text prompt information, a target generation document set is obtained using a large language model. Based on the target generation document set and the retrieval model, a target retrieval document set is obtained from the corpus. The user query statement is expanded based on the target retrieval document set and the target generation document set to obtain a target expanded query. The present disclosure can optimize and screen the expanded documents to prevent excessive expansion, so as to improve the quality of query expansion.

Description

Technical Field

[0001] The present disclosure belongs to the field of data verification query technology, and more specifically, to a query method, system, device and medium based on iterative retrieval generation verification. Background Art

[0002] In the fields of information retrieval and natural language processing, query expansion refers to the process of semantically enriching an original user query by applying a specific mapping function. This process aims to improve retrieval results by adding additional contextual information to the initial query, thereby enhancing the user's expressed information needs and generating an expanded query with potentially higher retrieval performance.

[0003] Existing query expansion methods can be broadly categorized into three categories: retrieval-based, generation-based, and ensemble-based. Retrieval-based query expansion methods, namely pseudo-relevance feedback (PRF), assume that the top-k documents retrieved are relevant to the query. The most relevant documents are then extracted from the target corpus as the basis for expansion. However, this approach exhibits significant limitations when dealing with short or ambiguous queries, as the documents retrieved from the original query may not fully align with the actual information required. Generation-based methods, namely generative relevance feedback (GRF), utilize advanced generative models, such as large language models (LLMs), as external knowledge bases to generate contextual documents. However, this approach also presents challenges when applied to specific corpora. In particular, when directly using existing LLMs for few-shot or zero-shot generation, the model struggles to align with the specific corpus. Furthermore, the generated content may contain irrelevant or hallucinatory information, reducing the effectiveness of the expansion. In order to combine the advantages of retrieval methods and generation methods, an integration-based method, namely Integrated Relevance Feedback (IRF), has been developed. This type of method uses both retrieved documents and generated documents as expansion basis.

[0004] However, the latest integrated relevance feedback methods have shortcomings when performing query expansion. They often introduce a lot of noise information that is irrelevant to user intent or deviates from the topic, resulting in excessive expansion, causing retrieval results to deviate from user intent and low query expansion quality. Summary of the Invention

[0005] The purpose of the present disclosure is to provide a query method, system, device and medium based on iterative retrieval generation verification, so as to optimize and screen extended documents, prevent over-extension and improve the quality of query expansion.

[0006] A first aspect of the embodiments of the present disclosure provides a query method based on iterative retrieval generation verification, comprising:

[0007] Based on the user query, an initial expanded query is obtained using a large language model; based on the initial expanded query and the retrieval model, an initial retrieval document set is obtained from the corpus, where the initial retrieval document set includes multiple retrieval documents whose similarity scores with the initial expanded query are greater than or equal to a similarity threshold;

[0008] Fill the user query statement and the initial retrieval document set into the text prompt template to obtain text prompt information; based on the text prompt information, use the large language model to obtain the target generated document set; based on the target generated document set and the retrieval model, obtain the target retrieval document set from the corpus;

[0009] The user query statement is expanded based on the target retrieval document set and the target generation document set to obtain the target expanded query.

[0010] A second aspect of the embodiments of the present disclosure provides a query system based on iterative retrieval generation verification, comprising:

[0011] An initial search generation module is used to obtain an initial expanded query based on a user query statement using a large language model; based on the initial expanded query and the retrieval model, an initial search document set is obtained from the corpus, where the initial search document set includes multiple search documents whose similarity scores with the initial expanded query are greater than or equal to a similarity threshold;

[0012] The iterative retrieval generation module is used to fill the user query statement and the initial retrieval document set into the text prompt template to obtain text prompt information; based on the text prompt information, the target generation document set is obtained using the large language model; based on the target generation document set and the retrieval model, the target retrieval document set is obtained from the corpus;

[0013] The document rearrangement and screening module is used to expand the user query statement based on the target retrieval document set and the target generation document set to obtain a target expanded query.

[0014] According to a third aspect of an embodiment of the present disclosure, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, wherein when the processor executes the computer program, the steps of the above-mentioned query method based on iterative retrieval generation verification are implemented.

[0015] According to a fourth aspect of the embodiments of the present disclosure, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the query method based on iterative retrieval generation verification are implemented.

[0016] The query method, system, device, and medium based on iterative retrieval generation verification provided by the embodiments of the present disclosure have the following beneficial effects:

[0017] On the one hand, the present disclosure leverages the deep understanding capabilities of large-scale language models to further enrich information dimensions through multiple rounds of iterative generation and retrieval. At the same time, each round screens out more relevant documents, reduces interference from irrelevant information, and improves recall and precision.

[0018] On the other hand, the present disclosure derives a target expanded query based on the user query, the target retrieval document set, and the target generated document set, achieving mutual verification among the three. By calculating the semantic relevance between documents and performing rearrangement and screening, off-topic and irrelevant information is filtered out, ensuring that the expanded query is both rich and accurate, preventing search results from deviating from user needs. In information retrieval scenarios, this approach can provide users with more valuable information and optimize their search experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0020] Figure 1 A flowchart of a query method based on iterative retrieval generation and verification provided in one embodiment of the present disclosure;

[0021] Figure 2 Schematic diagram of initial prompt construction (a) and subsequent prompt construction (b) provided in an embodiment of the present disclosure;

[0022] Figure 3 A flowchart of a query method based on iterative retrieval generation verification provided by another embodiment of the present disclosure;

[0023] Figure 4 A flowchart of a query method for a document rearrangement and screening module provided in one embodiment of the present disclosure;

[0024] Figure 5 A flowchart of a traditional query method provided in one embodiment of the present disclosure;

[0025] Figure 6 A flowchart of a query method based on iterative retrieval generation and verification provided in yet another embodiment of the present disclosure;

[0026] Figure 7 A flowchart of a query method based on iterative retrieval generation verification provided in yet another embodiment of the present disclosure;

[0027] Figure 8 A structural block diagram of a query system based on iterative retrieval generation and verification provided by an embodiment of the present disclosure;

[0028] Figure 9 A schematic block diagram of an electronic device provided in one embodiment of the present disclosure. DETAILED DESCRIPTION

[0029] In the following description, specific details such as specific system structures and techniques are provided for purposes of illustration rather than limitation to facilitate a thorough understanding of the embodiments of the present disclosure. However, it will be apparent to those skilled in the art that the present disclosure may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obscuring the description of the present disclosure with unnecessary detail.

[0030] In order to make the purpose, technical solutions and advantages of the present disclosure more clear, specific embodiments will be described below with reference to the accompanying drawings.

[0031] Please refer to Figure 1 , Figure 1 This is a flowchart of a query method based on iterative retrieval generation verification provided by an embodiment of the present disclosure. The method may include S101 to S103.

[0032] S101: Based on the user query, an initial expanded query is obtained using a large language model. An initial retrieval document set is obtained from a corpus based on the initial expanded query and the retrieval model. The initial retrieval document set includes multiple retrieval documents whose similarity scores with the initial expanded query are greater than or equal to a similarity threshold.

[0033] In this embodiment, the user query statement refers to the original information requirement entered by the user and serves as the starting point for query expansion. It often suffers from semantic ambiguity or insufficient information. The initial expanded query is a query that is expanded by combining the user query statement with content generated by a large language model. This provides the retrieval model with richer semantic input and resolves the ambiguity of the original query.

[0034] A corpus is a collection of documents that serves as a data source for retrieval. A retrieval model (RM) is a model used to retrieve documents relevant to a query from a corpus, including both sparse and dense retrieval. The initial retrieval document set refers to the set of documents returned by the retrieval model that have a similarity with the initial expanded query that exceeds a threshold. This set of documents originates from the corpus and serves as the basis for subsequent generation and verification, providing real-world contextual constraints. The similarity threshold is a preset critical value used to determine the relevance of retrieved documents to the query.

[0035] For example, in the fields of information retrieval and natural language processing, query expansion refers to the process of applying a specific mapping function , query the original user The process of semantic enrichment is to generate an expanded query with potentially higher retrieval efficiency by adding additional relevant contextual information to the initial query. This transformation can be formally described as Query expansion methods can improve the relevance of search results, increase recall and precision, and thus optimize the user's search experience.

[0036] To optimize information retrieval, the goal of the query expansion task can be formally defined as solving the following optimization problem:

[0037]

[0038] Among them, q represents the user query statement, Indicates the query after the user query statement is expanded. represents evaluation metrics, such as recall, normalized discounted cumulative gain, etc., Represents the retrieval model. The goal of optimization is to find the best parameters , so that the expanded query In the given evaluation index The best performance is achieved under

[0039] For example, the generation capability of LLM is used to derive relevant semantic dimensions from user query statements. For example, "sleeping environment", "work and rest schedule", "stress management" and other related information are generated based on "insomnia", and the user query statement is expanded to solve the vocabulary sparsity problem of the original query.

[0040] The expanded query is input into the retrieval model to obtain more comprehensive relevant documents through lexical matching (sparse retrieval) or semantic matching (dense retrieval), avoiding the limitations of traditional retrieval that relies only on the original query.

[0041] S102: The user query and the initial search document set are entered into a text prompt template to obtain text prompt information. Based on the text prompt information, a target generated document set is obtained using a large language model. Based on the target generated document set and the search model, a target search document set is obtained from the corpus.

[0042] In this embodiment, based on the text prompt information, a target generated document set is obtained using a large language model, and a target retrieved document set is obtained from a corpus based on the target generated document set and a retrieval model, including:

[0043] Step 1: Based on the text prompt information, use the large language model to obtain the intermediate generated document set.

[0044] Step 2: Obtain an intermediate retrieval document set from the corpus based on the intermediate generated document set and the retrieval model.

[0045] Step 3: Fill the user query and the intermediate retrieved document set into the text prompt template to obtain the intermediate text prompt information. Based on the intermediate text prompt information, the large language model is used to obtain the updated intermediate generated document set.

[0046] Step 1, step 2, and step 3 are iteratively executed based on the number of iterations, and the obtained updated intermediate generated document set is used as the target generated document set.

[0047] A target retrieval document set is obtained from the corpus based on the target generation document set and the similarity function of the query-document pair in the retrieval model.

[0048] In this embodiment, the text prompt template is a structured text framework used to guide the large language model to generate content that meets the requirements, including input placeholders for user query statements and retrieved documents. The text prompt template can combine user intent (original query) with real corpus information (initial retrieved documents) to constrain LLMs to generate content aligned with the target corpus. The target generated document set is a collection of related documents generated by LLMs based on text prompt information, which is used to supplement the potential intent of the user query. The target retrieval document set is a high-similarity document obtained by the retrieval model from the corpus based on the target generated document set, which is more in line with user intent after iterative optimization. The target retrieval document set serves as real data feedback to further correct the deviation of the generated content (target generated document set), forming a "generation-retrieval" closed loop.

[0049] For example, Figure 2 As shown, sub-figures (a) and (b) show the text prompt construction templates at two different stages, which are used to prompt LLM to generate extended content.

[0050] Sub-Figure (a): Initial prompt construction. This part shows a basic prompt template, which contains the original query as input and requires the generation of corresponding answer paragraphs for subsequent query expansion.

[0051] Sub-Figure (b): Subsequent prompt construction. This section shows a more advanced prompt template. The input includes not only the original query but also the results returned by the retrieval model, and requires the generation of corresponding answer paragraphs for subsequent query expansion.

[0052] For example, Figure 3 As shown, Figure 3This section describes the workflow of the Iterative Retrieval Generation Collaborative Module (IRG). This module innovatively combines the real-time retrieval capabilities of the retrieval model (RM) with the semantic understanding advantages of the Large Language Model (LLM). The retrieval model used in this module is the BM25. Through the bidirectional interaction between "Generate Enhanced Retrieval (GAR)" and "Retrieval Enhanced Generation (RAG)," this example first uses the LLM to generate documents to optimize the retrieval process, and then uses the retrieval results to guide the LLM in generating more accurate documents. Figure 3 The dynamic verification process (GVR / RVG) indicated by the short and medium dashed lines continuously screens documents that align the query intent and the corpus, and ultimately outputs high-quality extended documents that have been iteratively optimized. By complementing pseudo-relevant feedback documents with generative relevance feedback documents, the limitations of a single method are overcome.

[0053] BM25 evaluates the importance of each term in a document using term frequency (TF) and inverse document frequency (IDF), and normalizes the results based on the document length. The relevant formula is as follows:

[0054]

[0055]

[0056] in, Expressive words In the documentation The word frequency in Indicates the total number of documents, Indicates the inclusion of words The number of documents, Represents a document length, represents the average length of the document collection, and represents a hyperparameter, usually and .

[0057] For example, a specific text prompt template is constructed based on the initial search document set and the desired document content direction. The initial search document set and the user query statement are integrated into the text prompt template to form a knowledge-intensive input and generate text prompt information. The text prompt information construction function is:

[0058]

[0059] in, Indicates in The document collection generated by LLMs in the round, Indicates the The nth document in the document collection generated in the round, where n is the index of the generated document in the collection and N is the total number of generated documents. Indicates that LLMs are based on input The generated output is, include 、 and , Represents a text prompt template, Indicates in The initial retrieval document set obtained by the retrieval module in the round, Indicates information splicing operation.

[0060] Based on the textual prompt information, LLMs, such as ChatGPT, are invoked. LLMs leverage their text understanding and generation capabilities to generate an intermediate generated document set. Based on this intermediate generated document set, the retrieval model is again used to retrieve an intermediate retrieval document set from the corpus. Based on this newly obtained intermediate retrieval document set, the textual prompt template is reintegrated and input into the large language model to generate an updated intermediate generated document set. This process is repeated for a predetermined number of iterations until the target generated document set is achieved.

[0061] The resulting target document set is used as the new retrieval basis and fed into the retrieval model. The retrieval model searches the corpus based on its own query-document similarity function, calculating the similarity between each document and the target document set. It then selects documents that meet the required similarity and uses them as the target retrieval document set. This process leverages the retrieval results to assist LLMs in making predictions, overcoming the limitations of the LLMs' own knowledge and ensuring that the generated documents are more closely aligned with the user's query intent.

[0062] S103: Expand the user query statement based on the target retrieval document set and the target generation document set to obtain a target expanded query.

[0063] In this embodiment, the target retrieval document set includes a plurality of retrieval documents, and the target generated document set includes a plurality of generated documents;

[0064] The user query statement is expanded based on the target retrieval document set and the target generation document set to obtain the target expanded query, including:

[0065] For each retrieved document in the target retrieved document set: calculating a first relevance score between the retrieved document and the target generated document set and the user query statement;

[0066] For each generated document in the target generated document set: calculating a second relevance score between the generated document and the target retrieved document set and the user query statement;

[0067] All generated documents and retrieved documents are sorted and filtered based on the first relevance scores of the plurality of retrieved documents and the second relevance scores of the plurality of generated documents to obtain a target expanded query.

[0068] In this embodiment, the first relevance score is a comprehensive score of the semantic relevance between the retrieved document, the user's query, and the generated documents, reflecting the alignment between the retrieved document and the user's intent. The first relevance score is calculated based on the following dimensions: query-document relevance (the semantic match between the retrieved document and the original query); and document-document relevance (the semantic consistency between the retrieved document and all generated documents).

[0069] The second relevance score is a comprehensive score of the semantic relevance between the generated document and the user's query and retrieved documents, reflecting the alignment of the generated document with the real corpus. The second relevance score is calculated based on the following dimensions: query-document relevance (the semantic match between the generated document and the original query); and document-document relevance (the semantic consistency between the generated document and all retrieved documents).

[0070] This embodiment uses a bidirectional relevance constraint, wherein the retrieved document must not only be relevant to the user query but also be semantically consistent with the generated document, to avoid irrelevant retrieval and "pseudo-relevant" documents that deviate from the user's intent.

[0071] Generated documents must not only be relevant to the user query but also semantically aligned with the retrieved documents to avoid off-topic generation and “hallucination” content that is detached from the real corpus.

[0072] The scoring calculation formula may include calculating the semantic relevance of "query-document" and "document-document" based on cosine similarity, forming a comprehensive score through weighted summation, and filtering low-relevance documents.

[0073] For example, Figure 4 As shown in the figure, it describes the workflow of the "reranking and filtering module DRF", which is used to optimize the quality of retrieved documents and generated documents for subsequent query expansion. The reranking and filtering module receives three inputs: the embedding representation of the original query, the embedding representation of the retrieved document set, and the embedding representation of the generated document set. The reranking and filtering module assigns a comprehensive score to each document by calculating the cosine similarity between document-document pairs and query-document pairs: for each retrieved document, its score is the aggregate value of the semantic similarity between it and the original query and all generated documents, quantifying its degree of consistency with the user's query intent; for each generated document, its score is the aggregate value of the semantic similarity between it and the original query and all retrieved documents, quantifying its degree of alignment with the target corpus. Finally, the documents are re-ranked and filtered according to the score, and the retrieved document set and generated document set with the highest score are selected as the optimized context document set. Through this pairwise verification process among the three, the re-ranking and filtering module can select retrieval documents that are more aligned with the user's search intent, generate documents that are more aligned with the target corpus, and filter out irrelevant retrieval documents and off-topic generated documents, thereby ensuring the overall quality of the query basis.

[0074] For example, Figure 5 As shown, Figure 5 This paper describes three existing query expansion methods: pseudo relevance feedback (PRF), generative relevance feedback (GRF), and integrated relevance feedback (IRF). Pseudo relevance feedback uses a retrieval model (RM) to extract the most relevant documents from the target corpus as the basis for expansion. Generative relevance feedback directly utilizes the internal parameter knowledge of a large language model (LLM) to generate expansion content without accessing additional data sources, making it independent of external expansion. Integrated relevance feedback combines pseudo relevance feedback with generative relevance feedback, utilizing both retrieved and generated documents for query expansion.

[0075] Exemplarily, traditional integrated relevance feedback includes: internally expanding the user query using the LLM to obtain generative relevance feedback; externally expanding the user query using the knowledge base to obtain pseudo relevance feedback; and combining the generative relevance feedback and pseudo relevance feedback as the final expanded query.

[0076] For example, the query method based on iterative retrieval generation verification provided in this embodiment is applied to the IRGV-QE model. The workflow of IRGV-QE is as follows: Figure 6 As shown, Figure 6 The IRGV-QE framework describes a three-phase process: First, the original query is initially generated using the LLM. This phase then enters the core iterative retrieval generation and verification phase, which consists of two modules: The first is the Iterative Retrieval-Generation (IRG) collaboration, which aims to integrate the advantages of pseudo-relevance feedback and generative relevance feedback. During the iterative process, retrieval and generation alternate, with generated content used to improve retrieval, and retrieval results used to further inform construction, forming an iterative optimization process that integrates the advantages of pseudo-relevance feedback and generative relevance feedback. The second is the Document Re-ranking and Filtering (DRF), which aims to prevent over-expansion. By introducing a validation and filtering mechanism for query-document and document-document pairs, DRF selects generated documents that are more aligned with the target corpus and retrieved documents that are more aligned with the user's search intent during the iterative retrieval generation process. It also filters out off-topic generated documents and irrelevant retrieved documents, thereby avoiding over-expansion. IRG focuses on optimizing the breadth of expanded documents and pursuing comprehensiveness, while DRF focuses on improving the relevance of content and emphasizing accuracy. Finally, the retrieved document set and generated document set obtained after M rounds of iterative retrieval generation and verification are used as the basis for expansion and spliced ​​with the original query to obtain the expanded query.

[0077] For example, Figure 7 As shown, Figure 7 This paper describes the interrelationships among the various concepts of IRGV-QE. IRGV-QE performs query expansion by combining pseudo-relevance feedback and generative relevance feedback. In its implementation, each iteration of IRGV-QE is divided into four phases: Generation-Augmented Retrieval (GAR), Generation-Verified Retrieval (GVR), Retrieval-Augmented Generation (RAG), and Retrieval-Verified Generation (RVG). GAR and RAG form the Iterative Retrieval Generation (IRG) module, which integrates the advantages of pseudo-relevance feedback and generative feedback to overcome the limitations of each method. GVR and RVG form the Document Re-ranking (DRE) module, which prevents over-expansion.

[0078] Exemplarily, (1) generating enhanced search GAR:

[0079] When a user submits a query statement to the retrieval system , the user query can be a question, a keyword or a combination of both. RM can process user query statements , and based on the similarity function of the query-document pair Retrieve a set of documents from a corpus , to expand the original query. Ideally, the set Should contain extended query All the necessary information required. However, since the queries entered by users are usually short, this has significant limitations in accurately capturing the user's actual information needs. Especially when faced with vague or brief queries, it is difficult to accurately reflect the user's query intent, thus affecting the quality of the expanded documents. In addition, the effectiveness of query expansion depends on the accuracy of the initial search results. If the initial search fails to provide sufficient relevant information, the query expression cannot be effectively enhanced, which may weaken the overall retrieval performance.

[0080] Based on this, the IRGV-QE framework model introduces LLMs in the previous round ( The generated document set obtained in each (round) iteration is used to enrich the query content. Specifically, each element in the generated document set is repeatedly linked to the query, and the similarity between the corpus documents and the expanded query is calculated. The resulting documents are more accurately aligned with the user's search intent than traditional pseudo-relevance feedback documents.

[0081] (2) Retrieval enhancement generation RAG:

[0082] By building a specific text prompt template , LLMs can be called to generate documents. LLMs perform well in text generation, but the generation method that relies on the intrinsic parameter knowledge of LLMs has many limitations, such as hallucinations, difficulty in knowledge updating, and insufficient coverage of long-tail knowledge. Based on this, the IRGV-QE framework introduces the retrieval results of RMs As a query reference information to assist LLMs in making more accurate predictions. Specifically, by designing a new text prompt template , the current round of retrieval documents Integrate into text prompt template Through the above method, the input of LLMs is knowledge-intensive, which incorporates Extracted from the query relevant information, thereby helping LLMs understand and respond to queries more accurately.

[0083] From the above, we can conclude that, on the one hand, this embodiment leverages the deep understanding capabilities of a large language model to further enrich information dimensions through multiple rounds of iterative generation and retrieval. At the same time, each round screens out more relevant documents, reduces interference from irrelevant information, and improves recall and precision.

[0084] Furthermore, this embodiment derives a target expanded query based on the user query, the target retrieval document set, and the target generated document set, achieving mutual verification among the three. By calculating the semantic relevance between documents and performing rearrangement and filtering, off-topic and irrelevant information is filtered out, ensuring that the expanded query is both rich and accurate, preventing search results from deviating from user needs. In information retrieval scenarios, this approach can provide users with more valuable information and optimize their search experience.

[0085] In one embodiment of the present disclosure, based on a user query statement, an initial expanded query is obtained using a large language model, including:

[0086] Based on the user query statement, the initial generated document set is obtained using a large language model.

[0087] Each element in the initially generated document set is linked to the user query statement to obtain the initial expanded query.

[0088] In this embodiment, a large language model can perform multiple sampling and expansion of a user query, thereby obtaining an initial generated document set containing multiple generated documents. This initial generated document set serves as an important basis for expanding the query. Each element (generated document) in the initial generated document set is sequentially linked to the user query using a linking function. This repeated linking operation transforms the user query into a knowledge-intensive query, resulting in the initial expanded query. This process allows the query to contain more information, making it more conducive to the retrieval model accurately identifying user needs.

[0089] For example, the IRGV-QE framework introduces LLMs in the previous round ( The generated document set obtained in the iteration To enrich the query Specifically, by generating a document collection Each element in Repeatedly link to query , count the corpus documents The similarity between the query and the expanded query. The similarity function of the retrieval model is:

[0090]

[0091] in, Represents similarity calculation. Indicates in The collection of documents generated by LLMs in the round robin generation. Indicates the The first document in the collection of documents generated by round documents. Indicates that the user query statement Generate documentation with each Expanded query after alternating splicing. Represents the corpus to be searched. Indicates the current iteration round, Indicates the previous round.

[0092] This chaining operation makes the query Becoming a knowledge-intensive query helps RM focus on user queries more accurately Search intent, improve the accuracy and relevance of retrieval. In the (round) retrieval, the documents obtained are more accurate in matching the user's search intent than traditional pseudo-relevant feedback documents. The retrieval result representation formula is:

[0093]

[0094] in, Indicates the The retrieved document set obtained by the round retrieval, Indicates the The first round of search results documents, and K represents the total number of documents returned in each round of retrieval, that is, the size of the retrieval result set.

[0095] This implementation uses a large language model to generate an initial document set through multiple sampling. Its elements are then linked to the user query to form a knowledge-intensive query, enriching the query semantics and accurately targeting user needs. Compared to traditional methods, this operation ensures that retrieved documents are more aligned with user intent, improving search accuracy and relevance. It effectively addresses the ambiguity and information deficiency of the original query, laying a high-quality data foundation for subsequent iterative optimization.

[0096] In one embodiment of the present disclosure, calculating a first relevance score between the retrieved document, the target generated document set, and the user query statement includes:

[0097] Calculating a first semantic relevance index between the retrieved document and the user query statement based on a similarity function;

[0098] Calculate the second semantic relevance index between the retrieved document and all generated documents in the target generated document set based on the similarity function;

[0099] A first relevance score of the retrieved document is calculated based on the first semantic relevance index and the second semantic relevance index; the first relevance score is an aggregate value of semantic similarities between the retrieved document and the user query statement and all generated documents.

[0100] In this embodiment, calculating the second relevance score between the generated document, the target retrieval document set, and the user query statement includes:

[0101] Calculate a third semantic relevance index between the generated document and the user query statement based on a similarity function;

[0102] Calculate the fourth semantic relevance index between the generated document and all the retrieved documents in the target retrieval document set based on the similarity function;

[0103] A second relevance score of the generated document is calculated based on the third semantic relevance index and the fourth semantic relevance index; the second relevance score is an aggregate value of semantic similarities between the generated document and the user query statement and all retrieved documents.

[0104] In this embodiment, the first semantic relevance index refers to the direct match between the retrieved document and the user query (query-document relevance); the second semantic relevance index refers to the semantic consistency between the retrieved document and all generated documents (document-document relevance). The third semantic relevance index refers to the direct match between the generated document and the user query (query-document relevance). The fourth semantic relevance index refers to the semantic consistency between the generated document and all retrieved documents (document-document relevance).

[0105] For example, using the encoder of a dense retrieval model, the following is converted into a vector:

[0106] The original query (i.e., user query statement), the retrieved document set, and the generated document set are respectively denoted as 、 、 .

[0107]

[0108]

[0109]

[0110] Cosine similarity is used to calculate the semantic relevance between document-document pairs and query-document pairs, and an overall score is assigned to each document.

[0111] For each generated document , its second relevance score is the aggregated value of the semantic similarity between it and the original query and all retrieved documents. Therefore, the second relevance score can be interpreted as the generated document The degree of alignment with the target corpus. The second relevance score is calculated as:

[0112]

[0113] in, Indicates generated documentation ’s relevance score, i.e., the second relevance score. represents the similarity function, Indicates generated documentation The dense embedding representation of Indicates the Retrieved documents The dense embedding representation of Indicates the total number of retrieved documents. Indicates the maximum similarity value between the generated document and all retrieved documents. Represents the user's original query α represents the weight coefficient, 0≤α≤1. express and The similarity value between .

[0114] For each retrieved document , its first relevance score is the aggregate value of its semantic similarity with the original query and all generated documents. Therefore, the first relevance score reflects the retrieved document The degree of consistency with the user's query intent. The first relevance score calculation formula is:

[0115]

[0116] in, Retrieve documents ’s relevance score, i.e. the first relevance score. Represents the maximum similarity value between the retrieved document and all generated documents. express and The similarity value between .

[0117] All generated documents and retrieved documents are ranked based on the calculated first relevance score for each retrieved document and the calculated second relevance score for each generated document. Several retrieved documents and several generated documents with the highest scores are selected to form the context document set required for the target expanded query, thereby obtaining the target expanded query.

[0118]

[0119] in, Represents the generated document set after rearrangement and filtering, including the highest score Generate documentation. Indicates the filter condition, that is, only select the scores in the top Generate documentation for the name. Represents the retrieved document set after rearrangement and filtering, including the one with the highest score Retrieve documents. Indicates the filter condition, that is, only select the scores in the top Search documents by name.

[0120] In this embodiment, the weight coefficient used in calculating the second relevance score is the same as that used in calculating the first relevance score, ensuring consistency in weighting strategies.

[0121] This embodiment achieves accurate filtering of retrieved and generated documents by quantifying semantic relevance and weighted aggregation, ensuring that the target expanded query strikes a balance in user intent alignment, real corpus adaptability, and content accuracy. It is a key technical means to solve the problem of over-expansion.

[0122] In one embodiment of the present disclosure, all generated documents and retrieved documents are sorted and filtered based on the first relevance scores of the plurality of retrieved documents and the second relevance scores of the plurality of generated documents to obtain a target expanded query, including:

[0123] All the retrieved documents are sorted based on the first relevance scores of the multiple retrieved documents, and the target retrieved document with the highest score is selected.

[0124] All generated documents are sorted based on the second relevance scores of the multiple generated documents, and a target generated document with the highest score is selected.

[0125] A target expanded query is obtained based on the target retrieval document and the target generation document.

[0126] In this embodiment, the search document sorting and screening is as follows: all search documents for which the first relevance score has been calculated are sorted from high to low according to the first relevance score. After the sorting is completed, the search documents with the highest scores are selected as target search documents based on the preset number of search documents to be retained.

[0127] Generated Document Sorting and Filtering: All generated documents with calculated second relevance scores are sorted from high to low based on their second relevance scores. Similarly, according to the preset filtering conditions, the generated documents with the highest scores are selected as the target generated documents.

[0128] Obtaining a target expanded query: Integrate the filtered target retrieval documents and target generation documents, combining the key information and relevant content in these documents with the original user query. For example, important words and sentences from the target retrieval documents and target generation documents can be added to the original query, or they can be organized in a logical structure to ultimately form a target expanded query. In this way, the filtered documents are used to optimize the original query, making it more accurately reflect the user's information needs, improving the retrieval system's understanding of user intent, and ultimately enhancing retrieval results.

[0129] For example, in a query expansion method that integrates pseudo-relevance feedback and generative relevance feedback, the expanded documents contain a large amount of information that is irrelevant or off-topic to the user's query requirements, causing the final retrieval results to deviate from the user's intent. Based on this, this embodiment designs a re-ranking and screening module to improve the overall quality of query expansion by combining the complementary advantages of the original query, retrieved documents (aligned with a specific corpus), and generated documents (aligned with the true query intent).

[0130] This embodiment can screen out generated documents that are more aligned with the target corpus, retrieved documents that are more aligned with the user's search intent, and filter out off-topic generated documents and irrelevant retrieved documents, thereby avoiding the problem of over-expansion and ensuring the overall quality of query expansion.

[0131] Corresponding to the query method based on iterative retrieval generation verification in the above embodiment, Figure 8 This is a structural block diagram of a query system based on iterative retrieval generation verification provided by an embodiment of the present disclosure. For ease of explanation, only the parts related to the embodiment of the present disclosure are shown. Figure 8 The query system 20 based on iterative retrieval generation and verification includes: an initial retrieval generation module 21, an iterative retrieval generation module 22 and a document rearrangement and screening module 23.

[0132] The initial search generation module 21 is configured to generate an initial expanded query based on the user query using a large language model. Based on the initial expanded query and the retrieval model, an initial search document set is obtained from the corpus. The initial search document set includes multiple search documents whose similarity scores with the initial expanded query are greater than or equal to a similarity threshold.

[0133] The iterative search generation module 22 is configured to insert the user query and the initial search document set into a text prompt template to generate text prompt information. Based on the text prompt information, the large language model is used to obtain a target generated document set. Based on the target generated document set and the search model, the target search document set is obtained from the corpus.

[0134] The document rearrangement and screening module 23 is configured to expand the user query statement based on the target retrieval document set and the target generation document set to obtain a target expanded query.

[0135] In one embodiment of the present disclosure, the initial search generation module 21 is specifically configured to obtain an initial generated document set based on a user query statement using a large language model.

[0136] Each element in the initially generated document set is linked to the user query statement to obtain the initial expanded query.

[0137] In one embodiment of the present disclosure, the iterative retrieval generation module 22 is specifically used for step 1, obtaining an intermediate generated document set based on text prompt information using a large language model.

[0138] Step 2: Obtain an intermediate retrieval document set from the corpus based on the intermediate generated document set and the retrieval model.

[0139] Step 3: Fill the user query and the intermediate retrieved document set into the text prompt template to obtain the intermediate text prompt information. Based on the intermediate text prompt information, the large language model is used to obtain the updated intermediate generated document set.

[0140] Step 1, step 2, and step 3 are iteratively executed based on the number of iterations, and the obtained updated intermediate generated document set is used as the target generated document set.

[0141] A target retrieval document set is obtained from the corpus based on the target generation document set and the similarity function of the query-document pair in the retrieval model.

[0142] In one embodiment of the present disclosure, the target retrieval document set includes a plurality of retrieval documents, and the target generated document set includes a plurality of generated documents;

[0143] The document rearrangement and screening module 23 is specifically configured to calculate, for each retrieved document in the target retrieved document set, a first relevance score between the retrieved document and the target generated document set and the user query statement;

[0144] For each generated document in the target generated document set: calculating a second relevance score between the generated document and the target retrieved document set and the user query statement;

[0145] All generated documents and retrieved documents are sorted and filtered based on the first relevance scores of the plurality of retrieved documents and the second relevance scores of the plurality of generated documents to obtain a target expanded query.

[0146] In one embodiment of the present disclosure, the document rearrangement and screening module 23 is further configured to calculate a first semantic relevance index between the retrieved document and the user query statement based on a similarity function;

[0147] Calculate the second semantic relevance index between the retrieved document and all generated documents in the target generated document set based on the similarity function;

[0148] A first relevance score of the retrieved document is calculated based on the first semantic relevance index and the second semantic relevance index; the first relevance score is an aggregate value of semantic similarities between the retrieved document and the user query statement and all generated documents.

[0149] In one embodiment of the present disclosure, the document rearrangement and screening module 23 is further configured to calculate a third semantic relevance index between the generated document and the user query statement based on a similarity function;

[0150] Calculate the fourth semantic relevance index between the generated document and all the retrieved documents in the target retrieval document set based on the similarity function;

[0151] A second relevance score of the generated document is calculated based on the third semantic relevance index and the fourth semantic relevance index; the second relevance score is an aggregate value of semantic similarities between the generated document and the user query statement and all retrieved documents.

[0152] In one embodiment of the present disclosure, the document rearrangement and screening module 23 is further configured to sort all retrieved documents based on the first relevance scores of the multiple retrieved documents, and screen out the target retrieved document with the highest score.

[0153] All generated documents are sorted based on the second relevance scores of the multiple generated documents, and a target generated document with the highest score is selected.

[0154] A target expanded query is obtained based on the target retrieval document and the target generation document.

[0155] See also Figure 9 , Figure 9 This is a schematic block diagram of an electronic device provided by an embodiment of the present disclosure. Figure 9 The electronic device 300 in the embodiment shown may include: one or more processors 301, one or more input devices 302, one or more output devices 303, and one or more memories 304. The processors 301, input devices 302, output devices 303, and memories 304 communicate with each other via a communication bus 305. The memory 304 is used to store computer programs, which include program instructions. The processor 301 is used to execute the program instructions stored in the memory 304. The processor 301 is configured to call the program instructions to execute the functions of the modules in the above-mentioned system embodiments, such as Figure 8 The functions of the initial retrieval generation module 21, the iterative retrieval generation module 22 and the document rearrangement and screening module 23 are shown.

[0156] It should be understood that in the embodiments of the present disclosure, the processor 301 may be a central processing unit (CPU), or may be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0157] The input device 302 may include a touchpad, a fingerprint collection sensor (for collecting user fingerprint information and fingerprint direction information), a microphone, etc. The output device 303 may include a display (LCD, etc.), a speaker, etc.

[0158] The memory 304 may include a read-only memory and a random access memory, and provides instructions and data to the processor 301. A portion of the memory 304 may also include a non-volatile random access memory. For example, the memory 304 may also store information about the device type.

[0159] In a specific implementation, the processor 301, input device 302, and output device 303 described in the embodiments of the present disclosure can execute the implementation methods described in the first and second embodiments of the query method based on iterative retrieval generation verification provided in the embodiments of the present disclosure, and can also execute the implementation methods of the electronic device 300 described in the embodiments of the present disclosure, which will not be repeated here.

[0160] In another embodiment of the present disclosure, a computer-readable storage medium is provided. The computer-readable storage medium stores a computer program. The computer program includes program instructions. When the program instructions are executed by a processor, all or part of the process of the method in the above embodiment is implemented. The computer program can also be used to instruct related hardware to complete the process. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of each of the above method embodiments are implemented. The computer program includes computer program code, which can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium can include: any entity or device capable of carrying computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium.

[0161] The computer-readable storage medium can be an internal storage unit of the electronic device in any of the aforementioned embodiments, such as a hard disk or memory of the electronic device. The computer-readable storage medium can also be an external storage device of the electronic device, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a flash memory card, etc. Furthermore, the computer-readable storage medium can include both an internal storage unit of the electronic device and an external storage device. The computer-readable storage medium is used to store computer programs and other programs and data required by the electronic device. The computer-readable storage medium can also be used to temporarily store data that has been output or is about to be output.

[0162] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the composition and steps of each example according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this disclosure.

[0163] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the electronic devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0164] In the several embodiments provided in this application, it should be understood that the disclosed electronic devices and methods can be implemented in other ways. For example, the system embodiments described above are merely schematic. For example, the division of units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces or units, or can be an electrical, mechanical or other form of connection.

[0165] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, i.e., they may be located in one place or distributed across multiple network units. Some or all of these units may be selected based on actual needs to achieve the objectives of the embodiments of the present disclosure.

[0166] In addition, the functional units in the various embodiments of the present disclosure may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0167] The above are only specific embodiments of the present disclosure, but the scope of protection of the present disclosure is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or replacements within the technical scope disclosed in this disclosure, and such modifications or replacements should be included in the scope of protection of the present disclosure. Therefore, the scope of protection of the present disclosure should be based on the scope of protection of the claims.

Claims

1. A query method based on iterative retrieval generation and verification, characterized in that: include: Based on the user query, a large language model is used to obtain the initial generated document set; Linking each element in the initial generated document set with the user query statement to obtain an initial expanded query; obtaining an initial retrieved document set from a corpus based on the initial expanded query and a retrieval model, the initial retrieved document set including a plurality of retrieved documents having a similarity score with the initial expanded query greater than or equal to a similarity threshold; Fill the user query statement and the initial search document set into the text prompt template to obtain text prompt information; Based on the text prompt information, the target generated document set is obtained using a large language model; Obtaining a target retrieval document set from the corpus based on the target generated document set and the retrieval model; The target retrieval document set includes a plurality of retrieval documents, and the target generated document set includes a plurality of generated documents; For each retrieved document in the target retrieved document set: calculating a first relevance score between the retrieved document and the target generated document set and the user query statement; For each generated document in the target generated document set: calculating a second relevance score between the generated document and the target retrieved document set and the user query statement; sorting and filtering all generated documents and retrieved documents based on the first relevance scores of the plurality of retrieved documents and the second relevance scores of the plurality of generated documents to obtain a target expanded query; Based on the text prompt information, a target generated document set is obtained by using a large language model, and a target retrieved document set is obtained from a corpus based on the target generated document set and a retrieval model. include: Step 1: Based on the text prompt information, use the large language model to obtain the intermediate generated document set; Step 2: obtaining an intermediate retrieval document set from a corpus based on the intermediate generated document set and the retrieval model; Step 3: Fill the user query statement and the intermediate search document set into a text prompt template to obtain intermediate text prompt information; based on the intermediate text prompt information, use a large language model to obtain an updated intermediate generated document set; Iteratively execute step 1, step 2, and step 3 based on the number of iterations, and use the updated intermediate generated document set as the target generated document set; A target retrieval document set is obtained from a corpus based on the target generated document set and a similarity function of query-document pairs in a retrieval model.

2. The query method based on iterative retrieval generation and verification according to claim 1, characterized in that: Calculating a first relevance score between the retrieved document, the target generated document set, and the user query statement, including: Calculating a first semantic relevance index between the retrieved document and the user query statement based on a similarity function; Calculate the second semantic relevance index between the retrieved document and all generated documents in the target generated document set based on the similarity function; A first relevance score of the retrieved document is calculated based on the first semantic relevance index and the second semantic relevance index; the first relevance score is an aggregate value of semantic similarities between the retrieved document and the user query statement and all generated documents.

3. The query method based on iterative retrieval generation and verification according to claim 1, characterized in that: Calculating a second relevance score between the generated document and the target retrieval document set and the user query statement, including: Calculate a third semantic relevance index between the generated document and the user query statement based on a similarity function; Calculate the fourth semantic relevance index between the generated document and all the retrieved documents in the target retrieval document set based on the similarity function; A second relevance score of the generated document is calculated based on the third semantic relevance index and the fourth semantic relevance index; the second relevance score is an aggregate value of semantic similarities between the generated document and the user query statement and all retrieved documents.

4. The query method based on iterative retrieval generation and verification according to claim 1, characterized in that: All generated documents and retrieved documents are sorted and filtered based on the first relevance scores of the plurality of retrieved documents and the second relevance scores of the plurality of generated documents to obtain a target expanded query, including: sorting all the retrieved documents based on the first relevance scores of the multiple retrieved documents, and selecting the target retrieved document with the highest score; sorting all generated documents based on the second relevance scores of the multiple generated documents, and selecting the target generated document with the highest score; A target expansion query is obtained based on the target retrieval document and the target generation document.

5. A query system based on iterative retrieval generation and verification, characterized in that: include: The initial retrieval generation module is used to obtain the initial generated document set based on the user query statement using a large language model; Linking each element in the initial generated document set with the user query statement to obtain an initial expanded query; obtaining an initial retrieved document set from a corpus based on the initial expanded query and a retrieval model, the initial retrieved document set including a plurality of retrieved documents having a similarity score with the initial expanded query greater than or equal to a similarity threshold; Iterative retrieval generation module, used to fill the user query statement and the initial retrieval document set into the text prompt template to obtain text prompt information; Based on the text prompt information, the target generated document set is obtained using a large language model; Acquire a target retrieval document set from a corpus based on a target generated document set and a retrieval model; the target retrieval document set includes a plurality of retrieval documents, and the target generated document set includes a plurality of generated documents; The iterative retrieval generation module is specifically used in step 1 to obtain an intermediate generated document set based on text prompt information using a large language model; Step 2: obtaining an intermediate retrieval document set from a corpus based on the intermediate generated document set and the retrieval model; Step 3: Fill the user query statement and the intermediate search document set into a text prompt template to obtain intermediate text prompt information; based on the intermediate text prompt information, use a large language model to obtain an updated intermediate generated document set; Iteratively execute step 1, step 2, and step 3 based on the number of iterations, and use the updated intermediate generated document set as the target generated document set; Obtaining a target retrieval document set from a corpus based on the target generated document set and a similarity function of query-document pairs in a retrieval model; The document rearrangement and screening module is configured to calculate, for each retrieved document in the target retrieved document set, a first relevance score between the retrieved document and the target generated document set and the user query statement; and, for each generated document in the target generated document set, a second relevance score between the generated document and the target retrieved document set and the user query statement; All generated documents and retrieved documents are sorted and filtered based on the first relevance scores of the plurality of retrieved documents and the second relevance scores of the plurality of generated documents to obtain a target expanded query.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 4 are implemented.

7. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images