Query method, system and equipment based on iterative retrieval generation verification and medium

Abstract

The invention provides a query method, system and device based on iterative retrieval generation verification and a medium, and belongs to the technical field of data verification query. The method comprises the steps that based on a user query statement, an initial expansion query is obtained through a large language model; an initial retrieval document set is obtained from a corpus based on an initial expansion query and a retrieval model. And filling the user query statement and the initial retrieval document set into the text prompt template to obtain text prompt information. And based on the text prompt information, utilizing a large language model to obtain a target generation document set. And obtaining a target retrieval document set from the corpus based on the target generation document set and the retrieval model. And expanding the user query statement based on the target retrieval document set and the target generation document set to obtain a target expansion query. According to the method and the device, the extended document can be optimized and screened, the phenomenon of excessive extension is prevented, and the query extension quality is improved.

Description

Technical Field

[0001] The present disclosure belongs to the technical field of data verification query, and more specifically, to a query method, system, device and medium based on iterative retrieval generation verification. Background Art

[0002] In the field of information retrieval and natural language processing, query expansion refers to the process of semantically enriching the original user query by applying a specific mapping function. This process aims to improve the retrieval effect by adding additional relevant contextual information to the initial query term, thereby improving the information needs expressed by the user and generating an expanded query with potentially higher retrieval efficiency.

[0003] Existing query expansion methods can be mainly divided into three categories: retrieval-based methods, generation-based methods, and integration-based methods. The retrieval-based query expansion method, namely Pseudo-Relevance Feedback (PRF), assumes that the top-k documents retrieved initially are relevant to the query, and then extracts the most relevant documents from the target corpus as the basis for expansion. However, this method shows obvious limitations when dealing with short or ambiguous queries, because the documents retrieved by the original query may not be completely consistent with the actual information needs. The generation-based method, namely Generative Relevance Feedback (GRF), uses advanced generative models, such as Large Language Models (LLMs), as external knowledge bases to generate context documents. However, this method also has challenges when applied to a specific corpus, especially when directly using existing LLMs for few-shot or zero-shot generation, the model is difficult to align with the specific corpus. In addition, the generated content may contain irrelevant or hallucinatory information, reducing the effectiveness of expansion. In order to combine the advantages of retrieval methods and generation methods, an integration-based method, namely Integrated Relevance Feedback (IRF), has been developed. This type of method uses both retrieval documents and generated documents as expansion basis.

[0004] However, the latest integrated relevance feedback methods have shortcomings when performing query expansion. They often introduce a lot of noise information that is irrelevant to user intent or deviates from the topic, resulting in over-expansion, causing retrieval results to deviate from user intent and low query expansion quality. Summary of the invention

[0005] The purpose of the present disclosure is to provide a query method, system, device and medium based on iterative retrieval generation verification, so as to optimize and screen extended documents, prevent over-extension and improve the quality of query expansion.

[0006] A first aspect of an embodiment of the present disclosure provides a query method based on iterative retrieval generation verification, comprising: Based on the user query statement, an initial expanded query is obtained using a large language model; based on the initial expanded query and the retrieval model, an initial retrieval document set is obtained from the corpus, the initial retrieval document set including multiple retrieval documents whose similarity scores with the initial expanded query are greater than or equal to a similarity threshold; Fill the user query statement and the initial retrieval document set into the text prompt template to obtain text prompt information; based on the text prompt information, use the large language model to obtain the target generated document set; based on the target generated document set and the retrieval model, obtain the target retrieval document set from the corpus; The user query statement is expanded based on the target retrieval document set and the target generation document set to obtain a target expanded query.

[0007] A second aspect of the embodiments of the present disclosure provides a query system based on iterative retrieval generation verification, comprising: An initial search generation module is used to obtain an initial expanded query based on a user query statement using a large language model; based on the initial expanded query and the retrieval model, an initial search document set is obtained from a corpus, the initial search document set including multiple search documents whose similarity scores with the initial expanded query are greater than or equal to a similarity threshold; The iterative retrieval generation module is used to fill the user query statement and the initial retrieval document set into the text prompt template to obtain text prompt information; based on the text prompt information, the target generation document set is obtained by using the large language model; based on the target generation document set and the retrieval model, the target retrieval document set is obtained from the corpus; The document rearrangement and screening module is used to expand the user query statement based on the target retrieval document set and the target generation document set to obtain the target expanded query.

[0008] According to a third aspect of an embodiment of the present disclosure, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor implements the steps of the above-mentioned query method based on iterative retrieval generation verification when executing the computer program.

[0009] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the query method based on iterative retrieval generation verification are implemented.

[0010] The query method, system, device and medium based on iterative retrieval generation verification provided by the embodiments of the present disclosure have the following beneficial effects: On the one hand, the present disclosure uses the deep understanding ability of large language models to further enrich the information dimension through multiple rounds of iterative generation and retrieval. At the same time, each round screens out more relevant documents, reduces interference from irrelevant information, and improves recall and precision.

[0011] On the other hand, the present disclosure obtains a target extended query based on the user query statement, the target retrieval document set and the target generated document set, and realizes mutual verification among the three. By calculating the semantic relevance between documents, the documents are rearranged and screened, and off-topic and irrelevant information is filtered out, so as to ensure that the extended query is both rich and accurate, and avoid the retrieval results from deviating from the user's needs. In the information retrieval scenario, it can provide users with more valuable information and optimize the user's retrieval experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0013] Figure 1 A flowchart of a query method based on iterative retrieval generation verification provided by an embodiment of the present disclosure; Figure 2 A schematic diagram of an initial prompt construction (a) and a schematic diagram of a subsequent prompt construction (b) provided in an embodiment of the present disclosure; Figure 3 A flowchart of a query method based on iterative retrieval generation verification provided by another embodiment of the present disclosure; Figure 4 A flowchart of a query method of a document rearrangement and screening module provided in an embodiment of the present disclosure; Figure 5 A flowchart of a traditional query method provided by an embodiment of the present disclosure; Figure 6 A flowchart of a query method based on iterative retrieval generation verification provided by another embodiment of the present disclosure; Figure 7 A flowchart of a query method based on iterative retrieval generation verification provided in yet another embodiment of the present disclosure; Figure 8 A structural block diagram of a query system based on iterative retrieval generation verification provided by an embodiment of the present disclosure; Fig. 9 A schematic block diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0014] In the following description, specific details such as specific system structures and technologies are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present disclosure. However, it should be clear to those skilled in the art that the present disclosure may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obstructing the description of the present disclosure with unnecessary details.

[0015] In order to make the purpose, technical solutions and advantages of the present disclosure more clear, specific embodiments will be described below in conjunction with the accompanying drawings.

[0016] Please refer to Figure 1 , Figure 1 A flowchart of a query method based on iterative retrieval generation verification provided in an embodiment of the present disclosure, the method may include S101 to S103.

[0017] S101: Based on the user query, an initial expanded query is obtained using a large language model. Based on the initial expanded query and the retrieval model, an initial retrieval document set is obtained from a corpus, the initial retrieval document set including multiple retrieval documents whose similarity scores with the initial expanded query are greater than or equal to a similarity threshold.

[0018] In this embodiment, the user query statement refers to the original information demand input by the user, which is the starting point for triggering query expansion, and usually has semantic ambiguity or insufficient information. The initial expanded query refers to the query that is spliced ​​and expanded based on the user query statement and combined with the content generated by the large language model, which can provide richer semantic input for the retrieval model and resolve the ambiguity of the original query.

[0019] A corpus is a collection of a large number of documents and is a data source for retrieval. A retrieval model (RM) is a model used to retrieve documents related to a query from a corpus, including sparse retrieval and dense retrieval. The initial retrieval document set refers to a set of documents returned by the retrieval model with a similarity ≥ threshold to the initial expanded query. It comes from the corpus and can be used as the basic data for subsequent generation and verification, providing real context constraints. The similarity threshold is a preset critical value used to determine the relevance of the retrieved document to the query.

[0020] For example, in the fields of information retrieval and natural language processing, query expansion refers to the process of applying a specific mapping function to , query the original user The process of semantic enrichment is to generate an expanded query with potentially higher retrieval performance by adding additional relevant contextual information to the initial query. This transformation can be formally described as Query expansion methods can improve the relevance of search results, increase recall and precision, and thus optimize the user's search experience.

[0021] In order to optimize information retrieval, the goal of the query expansion task can be formally defined as solving the following optimization problem:

[0022] Among them, q represents the user query statement, Indicates the query after the user query statement is expanded. represents evaluation metrics, such as recall, normalized discounted cumulative gain, etc., Represents the retrieval model. The optimization goal is to find the best parameters , so that the expanded query In the given evaluation index The best performance is achieved under .

[0023] Exemplarily, the generation capability of LLM is used to derive relevant semantic dimensions from user query statements, such as generating relevant information such as "sleeping environment", "work and rest schedule", and "stress management" based on "insomnia", thereby expanding the user query statement and solving the vocabulary sparsity problem of the original query.

[0024] The expanded query is input into the retrieval model to obtain more comprehensive relevant documents through lexical matching (sparse retrieval) or semantic matching (dense retrieval), avoiding the limitation of traditional retrieval that only relies on the original query.

[0025] S102: Fill the user query statement and the initial search document set into the text prompt template to obtain text prompt information. Based on the text prompt information, use the large language model to obtain the target generated document set. Based on the target generated document set and the search model, obtain the target search document set from the corpus.

[0026] In this embodiment, based on the text prompt information, a target generated document set is obtained by using a large language model, and a target retrieved document set is obtained from a corpus based on the target generated document set and a retrieval model, including: Step 1: Based on the text prompt information, use the large language model to obtain the intermediate generated document set.

[0027] Step 2: Obtain an intermediate retrieval document set from the corpus based on the intermediate generated document set and the retrieval model.

[0028] Step 3: Fill the user query statement and the intermediate search document set into the text prompt template to obtain the intermediate text prompt information. Based on the intermediate text prompt information, the updated intermediate generated document set is obtained using a large language model.

[0029] Step 1, step 2 and step 3 are iteratively executed based on the number of iterations, and the obtained updated intermediate generated document set is used as the target generated document set.

[0030] A target retrieval document set is obtained from the corpus based on the target generation document set and the similarity function of the query-document pair in the retrieval model.

[0031] In this embodiment, the text prompt template is a structured text framework used to guide the large language model to generate content that meets the requirements, including input placeholders for user query statements and retrieved documents. The text prompt template can combine user intent (original query) with real corpus information (initial retrieval document) to constrain LLMs to generate content aligned with the target corpus. The target generated document set is a collection of related documents generated by LLMs based on text prompt information, which is used to supplement the potential intent of the user query. The target retrieval document set is a high-similarity document obtained by the retrieval model from the corpus based on the target generated document set, which is more in line with user intent after iterative optimization. The target retrieval document set is used as real data feedback to further correct the deviation of the generated content (target generated document set) to form a "generation-retrieval" closed loop.

[0032] For example, Figure 2 As shown, sub-figures (a) and (b) show the text prompt construction templates at two different stages, which are used to prompt LLM to generate extended content.

[0033] Sub-figure (a): Initial prompt construction. This part shows a basic prompt template, which contains the original query as input and requires the generation of the corresponding answer paragraph for subsequent query expansion.

[0034] Sub-figure (b): Subsequent prompt construction. This part shows a more advanced prompt template, where the input includes not only the original query but also the results returned by the retrieval model, and requires the generation of corresponding answer paragraphs for subsequent query expansion.

[0035] For example, Figure 3 As shown, Figure 3 Used to describe the workflow of the "Iterative Retrieval Generation Collaborative Module IRG", which innovatively combines the real-time retrieval capability of the retrieval model RM and the semantic understanding advantage of the large language model LLM. The retrieval model used in this module is BM25. Through the two-way interactive mechanism of "Generate Enhanced Retrieval GAR" and "Retrieval Enhanced Generation RAG", this embodiment first uses LLM to generate documents to optimize the retrieval process, and then guides LLM to generate more accurate documents through the retrieval results. Figure 3The dynamic verification link (GVR / RVG) indicated by the short and medium dashed lines continuously screens documents that align the query intent and the corpus, and finally outputs high-quality extended documents that have been iteratively optimized. It overcomes the limitations of a single method through the complementarity of pseudo-relevant feedback documents and generative relevant feedback documents.

[0036] Among them, BM25 evaluates the importance of each term in the document through term frequency (TF) and inverse document frequency (IDF), and normalizes it based on the document length. The relevant formula is as follows:

[0037]

[0038] in, Expressive words In the documentation The word frequency in Represents the total number of documents, Indicates that it contains words The number of documents, Representation Document Length, represents the average length of a document collection, and represents a hyperparameter, usually and .

[0039] Exemplarily, a specific text prompt template is constructed based on the initial search document set and the document content direction expected to be generated. The initial search document set and the user query statement are integrated into the text prompt template to form a knowledge-intensive input and generate text prompt information. The text prompt information construction function is:

[0040] in, Indicated in The collection of documents generated using LLMs in the round, Indicates The nth document in the document collection generated in the round, n represents the index of the generated document in the collection, and N represents the total number of generated documents. Indicates that LLMs are based on input The generated output is, include , and , Represents a text prompt template. Indicated in The initial set of retrieved documents obtained by the retrieval module in the round, Represents information concatenation operation.

[0041] Based on the text prompt information, LLMs, such as ChatGPT, are called to generate an intermediate generated document set through the text understanding and generation capabilities of LLMs. Based on the intermediate generated document set, the retrieval model is used again to obtain the intermediate retrieval document set from the corpus. Based on the newly obtained intermediate retrieval document set, the text prompt template is reintegrated and input into the large language model to generate an updated intermediate generated document set. According to the preset number of iterations, the above steps are repeated continuously to finally obtain the target generated document set.

[0042] The target generated document set is used as a new retrieval basis and input into the retrieval model. The retrieval model searches the corpus based on the query-document similarity function set by itself, calculates the similarity between each document and the target generated document set, selects documents with similarity that meet the requirements, and uses these documents as the target retrieval document set. This process uses the retrieval results to assist LLMs in prediction, making up for the limitations of LLMs' own knowledge and making the generated documents more in line with the user's query intent.

[0043] S103: Expand the user query statement based on the target retrieval document set and the target generation document set to obtain a target expanded query.

[0044] In this embodiment, the target retrieval document set includes a plurality of retrieval documents, and the target generated document set includes a plurality of generated documents; The user query statement is expanded based on the target retrieval document set and the target generation document set to obtain a target expanded query, including: For each retrieved document in the target retrieved document set: calculating a first relevance score between the retrieved document and the target generated document set and the user query statement; For each generated document in the target generated document set: calculating a second relevance score between the generated document and the target retrieved document set and the user query statement; All generated documents and retrieved documents are sorted and screened based on the first relevance scores of the plurality of retrieved documents and the second relevance scores of the plurality of generated documents to obtain a target expanded query.

[0045] In this embodiment, the first relevance score refers to the comprehensive score of the semantic relevance between the retrieved document and the user query statement and the generated document, reflecting the alignment between the retrieved document and the user's intention. The calculation dimensions of the first relevance score include: query-document relevance: the semantic match between the retrieved document and the original query; document-document relevance: the semantic consistency between the retrieved document and all generated documents.

[0046] The second relevance score refers to the comprehensive score of the semantic relevance between the generated document and the user query and retrieved documents, reflecting the alignment between the generated document and the real corpus. The calculation dimensions of the second relevance score include: query-document relevance: the semantic match between the generated document and the original query; document-document relevance: the semantic consistency between the generated document and all retrieved documents.

[0047] This embodiment adopts a bidirectional constraint of relevance, wherein the retrieved document must not only be relevant to the user query, but also be semantically consistent with the generated document, so as to avoid irrelevant retrieval and "pseudo-relevant" documents that deviate from the user's intention.

[0048] The generated documents need to be not only relevant to the user query, but also semantically aligned with the retrieved documents to avoid off-topic generation and “hallucination” content that is divorced from the real corpus.

[0049] The scoring calculation formula may include calculating the semantic relevance of "query-document" and "document-document" based on cosine similarity, forming a comprehensive score through weighted summation, and filtering low-relevance documents.

[0050] For example, Figure 4 As shown in the figure, the workflow of the "reranking and filtering module DRF" is described, which is used to optimize the quality of retrieved documents and generated documents for subsequent query expansion. The reranking and filtering module receives three inputs: the embedding representation of the original query, the embedding representation of the retrieved document set, and the embedding representation of the generated document set. The reranking and filtering module assigns a comprehensive score to each document by calculating the cosine similarity between document-document pairs and query-document pairs: for each retrieved document, its score is the aggregate value of its semantic similarity with the original query and all generated documents, quantifying its degree of consistency with the user's query intent; for each generated document, its score is the aggregate value of its semantic similarity with the original query and all retrieved documents, quantifying its degree of alignment with the target corpus. Finally, the documents are re-ranked and filtered according to the score, and the retrieved document set and generated document set with the highest score are selected as the optimized context document set. Through this pairwise verification process among the three, the rearrangement and screening module can screen out retrieval documents that are more aligned with the user's search intent, generated documents that are more aligned with the target corpus, and filter out irrelevant retrieval documents and off-topic generated documents, thereby ensuring the overall quality of the query basis.

[0051] For example, Figure 5 As shown, Figure 5Three existing query expansion methods are described: pseudo relevance feedback (PRF), generative relevance feedback (GRF), and integrated relevance feedback (IRF). Pseudo relevance feedback uses the retrieval model RM to extract the most relevant documents from the target corpus as the basis for expansion; generative relevance feedback directly uses the internal parameter knowledge of the large language model LLM to generate expansion content without accessing additional data sources, that is, it is independent of external expansion. Integrated relevance feedback combines pseudo relevance feedback with generative relevance feedback, that is, it uses both retrieved documents and generated documents for query expansion.

[0052] Exemplarily, the traditional integrated relevance feedback includes: based on the user query, using LLM for internal expansion to obtain generative relevance feedback. Based on the user query, using the knowledge base for external expansion to obtain pseudo relevance feedback. The generative relevance feedback and the pseudo relevance feedback are used as the final expanded query.

[0053] Exemplarily, the query method based on iterative retrieval generation verification provided in this embodiment is applied to the IRGV-QE model. The workflow of IRGV-QE is as follows: Figure 6 As shown, Figure 6 The three-stage process of the IRGV-QE framework is described: First, the original query is initially generated through LLM. Then it enters the core iterative retrieval generation verification stage, which includes two modules: the first is iterative retrieval-generation (IRG), which aims to integrate the advantages of pseudo-relevance feedback and generative relevance feedback. In the iterative process, retrieval and generation are carried out alternately, the generated content is used to improve the retrieval, and the retrieval results are used to further prompt the construction, forming an iterative optimization process, which aims to integrate the advantages of pseudo-relevance feedback and generative relevance feedback; the second is document re-ranking and filtering (DRF), which aims to prevent over-expansion. DRF introduces the verification and filtering mechanism of query-document pairs and document-document pairs to filter out generated documents that are more aligned with the target corpus and retrieved documents that are more aligned with the user's search intent during the iterative retrieval generation process, and filters out off-topic generated documents and irrelevant retrieved documents, thereby avoiding the problem of over-expansion. IRG is committed to optimizing the breadth of extended documents and pursuing comprehensiveness; DRF focuses on improving the relevance of content and emphasizes accuracy. Finally, the retrieved document set and generated document set obtained after M rounds of iterative retrieval generation and verification are used as the basis for expansion and concatenated with the original query to obtain the expanded query.

[0054] For example, Figure 7 As shown, Figure 7The relationship between the concepts of IRGV-QE is described. IRGV-QE performs query expansion by combining pseudo-relevance feedback and generative relevance feedback. In terms of specific implementation, each round of iteration of IRGV-QE is divided into four stages: Generation-Augmented Retrieval (GAR), Generation-Verified Retrieval (GVR), Retrieval-Augmented Generation (RAG), and Retrieval-Verified Generation (RVG). Among them, GAR and RAG constitute the Iterative Retrieval Generation (IRG) module, which is used to integrate the advantages of pseudo-relevance feedback and generative feedback to overcome the limitations of a single method; GVR and RVG constitute the Document Rearrangement Filter (DRE) module, which is used to prevent over-expansion.

[0055] Exemplarily, (1) generating an enhanced search GAR: When a user submits a query statement to the retrieval system , the user query can be a question, a keyword, or a combination of both. RM can process user query statements , and based on the similarity function of the query-document pair Retrieve a set of documents from a corpus , which can be used to expand the original query. Ideally, the collection Should contain extended query All the necessary information required. However, since the queries entered by users are usually short, this has significant limitations in accurately capturing the users' actual information needs, especially when facing vague or brief queries, it is difficult to accurately reflect the users' query intentions, thus affecting the quality of the expanded documents. In addition, the effectiveness of query expansion depends on the accuracy of the initial search results. If the initial search fails to provide sufficient relevant information, the query expression cannot be effectively enhanced, which may weaken the overall retrieval performance.

[0056] Based on this, the IRGV-QE framework model introduces LLMs in the previous round ( The generated document set obtained in the (round) iteration is used to enrich the query content. Specifically, each element in the generated document set is repeatedly linked to the query, and the similarity between the corpus document and the expanded query is calculated. The final obtained document is more accurate in matching the user's search intent than the traditional pseudo-relevant feedback document.

[0057] (2) Retrieval enhancement generation RAG: By building a specific text prompt template , LLMs can be called to generate documents. LLMs perform well in text generation, but the generation method that relies on the intrinsic parameter knowledge of LLMs has many limitations, such as hallucinations, difficulty in knowledge updating, and insufficient coverage of long-tail knowledge. Based on this, the IRGV-QE framework introduces the retrieval results of RMs As a query To assist LLMs in making more accurate predictions, we design a new text prompt template. , the current round of retrieval documents Integrate into text prompt template Through the above method, the input of LLMs is knowledge-intensive, which incorporates Extracted from the query relevant information, thereby helping LLMs understand and respond to queries more accurately.

[0058] From the above, it can be concluded that, on the one hand, this embodiment uses the deep understanding ability of the large language model to further enrich the information dimension through multiple rounds of iterative generation and retrieval. At the same time, each round screens out more relevant documents, reduces interference from irrelevant information, and improves recall and precision.

[0059] On the other hand, this embodiment obtains the target extended query based on the user query statement, the target retrieval document set and the target generated document set, and realizes mutual verification among the three. By calculating the semantic relevance between documents, rearrangement and screening are performed, and off-topic and irrelevant information is filtered out, so as to ensure that the extended query is both rich and accurate, and avoid the retrieval results deviating from user needs. In the information retrieval scenario, it can provide users with more valuable information and optimize the user retrieval experience.

[0060] In one embodiment of the present disclosure, based on a user query statement, an initial expanded query is obtained using a large language model, including: Based on the user query, a large language model is used to obtain the initial generated document set.

[0061] Each element in the initially generated document set is linked to the user query statement to obtain an initial expanded query.

[0062] In this embodiment, the large language model can perform multiple sampling and expansion on the user query statement, thereby obtaining an initial generated document set containing multiple generated documents, and the initial generated document set will serve as an important basis for the extended query. According to the link function, each element (generated document) in the initial generated document set is linked to the user query statement in sequence. Through this repeated linking operation, the user query is transformed into a knowledge-intensive query, that is, the initial expanded query is obtained. This process allows the query to contain more information, which is more conducive to the retrieval model to accurately locate user needs.

[0063] For example, the IRGV-QE framework introduces LLMs in the previous round ( The generated document set obtained in the iteration To enrich the query Specifically, by generating a document collection Each element in Repeated link to query , count the corpus documents The similarity between the query and the expanded query. The similarity function of the retrieval model is:

[0064] in, Represents similarity calculation. Indicated in The collection of documents generated by LLMs in the round robin generation. Indicates The first document in the collection of documents generated by round documents. Indicates that the user query statement With each generated document Expanded query after alternating concatenation. Represents the corpus to be searched. Indicates the current iteration round, Indicates the previous round.

[0065] This linking operation makes the query Becoming a knowledge-intensive query helps RM focus on user queries more accurately search intent, improve the accuracy and relevance of retrieval. In the retrieval of the first round, the documents obtained are more accurate in matching the user's search intention than the traditional pseudo-relevant feedback documents. The formula for representing the retrieval results is:

[0066] in, Indicates The set of retrieved documents obtained by round retrieval, Indicates The first round of search results documents, and K represents the total number of documents returned in each round of retrieval, that is, the size of the retrieval result set.

[0067] This embodiment obtains the initial generated document set through multiple sampling of a large language model, and links its elements to user queries to form knowledge-intensive queries, which can enrich query semantics and accurately locate user needs. Compared with traditional methods, this operation makes the retrieved documents more consistent with user intent, improves retrieval accuracy and relevance, effectively solves the problems of fuzzy original queries and insufficient information, and lays a high-quality data foundation for subsequent iterative optimization.

[0068] In one embodiment of the present disclosure, calculating a first relevance score between the retrieved document, the target generated document set, and the user query statement includes: Calculate the first semantic relevance index between the retrieved document and the user query statement based on the similarity function; Calculate the second semantic relevance index between the retrieved document and all generated documents in the target generated document set based on the similarity function; A first relevance score of the retrieved document is calculated based on the first semantic relevance index and the second semantic relevance index; the first relevance score is an aggregate value of semantic similarities between the retrieved document and the user query statement and all generated documents.

[0069] In this embodiment, calculating the second relevance score between the generated document and the target retrieval document set and the user query statement includes: Calculate a third semantic relevance index between the generated document and the user query statement based on a similarity function; Calculate the fourth semantic relevance index between the generated document and all the retrieved documents in the target retrieval document set based on the similarity function; A second relevance score of the generated document is calculated based on the third semantic relevance index and the fourth semantic relevance index; the second relevance score is an aggregate value of semantic similarities between the generated document and the user query statement and all retrieved documents.

[0070] In this embodiment, the first semantic relevance index refers to the direct match between the retrieved document and the user query (query-document relevance); the second semantic relevance index refers to the semantic consistency between the retrieved document and all generated documents (document-document relevance). The third semantic relevance index refers to the direct match between the generated document and the user query (query-document relevance). The fourth semantic relevance index refers to the semantic consistency between the generated document and all retrieved documents (document-document relevance).

[0071] For example, using the encoder of a dense retrieval model, the following is converted into a vector: The original query (i.e., user query statement), the retrieved document set, and the generated document set are respectively denoted as , , .

[0072]

[0073]

[0074]

[0075] Cosine similarity is used to calculate the semantic relevance between document-document pairs and query-document pairs, and a comprehensive score is assigned to each document.

[0076] For each generated document , its second relevance score is the aggregate value of its semantic similarity with the original query and all retrieved documents. Therefore, the second relevance score can be interpreted as the generated document The degree of alignment with the target corpus. The second relevance score calculation formula is:

[0077] in, Indicates the generated document , i.e., the second relevance score. represents the similarity function, Indicates the generated document The dense embedding representation of Indicates Search documents The dense embedding representation of Indicates the total number of retrieved documents. Indicates the maximum similarity value between the generated document and all retrieved documents. Represents the user's original query α represents the weight coefficient, 0≤α≤1. express and The similarity value between .

[0078] For each retrieved document , its first relevance score is the aggregate value of its semantic similarity with the original query and all generated documents. Therefore, the first relevance score reflects the retrieved document The degree of consistency with the user's query intent. The first relevance score calculation formula is:

[0079] in, Retrieve documents ’s relevance score, i.e., the first relevance score. Represents the maximum similarity value between the retrieved document and all generated documents. express and The similarity value between .

[0080] According to the calculated first relevance score of each retrieved document and the second relevance score of each generated document, all generated documents and retrieved documents are sorted, and a number of retrieved documents and generated documents with the highest scores are selected to form a context document set required for the target expanded query, thereby obtaining the target expanded query.

[0081]

[0082] in, Represents the generated document set after rearrangement and filtering, including the highest score Generate a document. Indicates the filter condition, that is, only select the scores in the top Generate documentation for the name. Represents the retrieved document set after rearrangement and filtering, including the highest scoring Retrieve documents. Indicates the filter condition, that is, only select the scores in the top Search documents by name.

[0083] In this embodiment, the weight coefficient used in calculating the second correlation score is the same as that used in calculating the first correlation score, ensuring that the weight strategy is consistent.

[0084] This embodiment achieves accurate filtering of retrieved documents and generated documents by quantifying semantic relevance and weighted aggregation, ensuring that the target expanded query strikes a balance in user intent alignment, real corpus adaptability, and content accuracy, and is a key technical means to solve the problem of over-expansion.

[0085] In one embodiment of the present disclosure, all generated documents and retrieved documents are sorted and screened based on the first relevance scores of the plurality of retrieved documents and the second relevance scores of the plurality of generated documents to obtain a target expanded query, including: All the retrieved documents are sorted based on the first relevance scores of the multiple retrieved documents, and the target retrieved document with the highest score is screened out.

[0086] All generated documents are sorted based on the second relevance scores of the multiple generated documents, and the target generated document with the highest score is selected.

[0087] A target expanded query is obtained based on the target retrieval document and the target generation document.

[0088] In this embodiment, the search document sorting and screening: all search documents with calculated first relevance scores are sorted from high to low according to the first relevance scores. After the sorting is completed, according to the preset number of search documents to be retained, several search documents with the highest scores are selected as target search documents.

[0089] Generated document sorting and screening: All generated documents with calculated second relevance scores are sorted from high to low according to the second relevance scores. Also according to the preset screening conditions, select the generated documents with the highest scores as the target generated documents.

[0090] Get target extended query: Integrate the filtered target retrieval documents and target generated documents, and combine the key information and relevant content in these documents with the original user query statement. For example, important words and sentences in the target retrieval documents and target generated documents can be added to the original query statement, or they can be organized in a certain logical structure to finally form a target extended query. In this way, the original query is optimized using the filtered documents, so that it can more accurately reflect the user's information needs, improve the retrieval system's understanding of the user's intentions, and thus improve the retrieval effect.

[0091] For example, in the query expansion method integrating pseudo-relevance feedback and generative relevance feedback, the expanded documents contain a large amount of information that is irrelevant or off-topic to the user's query requirements, which will cause the final retrieval results to deviate from the user's intent. Based on this, this embodiment designs a rearrangement and screening module, which aims to improve the overall quality of query expansion by combining the complementary advantages of the original query, the retrieved documents (aligned with a specific corpus), and the generated documents (aligned with the true intent of the query).

[0092] This embodiment can screen out generated documents that are more aligned with the target corpus, retrieved documents that are more aligned with the user's search intent, and filter out off-topic generated documents and irrelevant retrieved documents, thereby avoiding the problem of over-expansion and ensuring the overall quality of query expansion.

[0093] Corresponding to the query method based on iterative retrieval generation verification in the above embodiment, Figure 8 A structural block diagram of a query system based on iterative retrieval generation verification provided by an embodiment of the present disclosure. For ease of explanation, only the parts related to the embodiment of the present disclosure are shown. Figure 8 The query system 20 based on iterative retrieval generation verification includes: an initial retrieval generation module 21, an iterative retrieval generation module 22 and a document rearrangement screening module 23.

[0094] The initial search generation module 21 is used to obtain an initial expanded query based on the user query sentence using a large language model, and obtain an initial search document set from the corpus based on the initial expanded query and the search model, wherein the initial search document set includes multiple search documents whose similarity scores with the initial expanded query are greater than or equal to a similarity threshold.

[0095] The iterative retrieval generation module 22 is used to fill the user query statement and the initial retrieval document set into the text prompt template to obtain text prompt information. Based on the text prompt information, the target generated document set is obtained using the large language model. Based on the target generated document set and the retrieval model, the target retrieval document set is obtained from the corpus.

[0096] The document rearrangement and screening module 23 is used to expand the user query statement based on the target retrieval document set and the target generation document set to obtain a target expanded query.

[0097] In one embodiment of the present disclosure, the initial search generation module 21 is specifically used to obtain an initial generated document set based on a user query statement using a large language model.

[0098] Each element in the initially generated document set is linked to the user query statement to obtain an initial expanded query.

[0099] In one embodiment of the present disclosure, the iterative retrieval generation module 22 is specifically used for step 1, obtaining an intermediate generated document set based on text prompt information using a large language model.

[0100] Step 2: Obtain an intermediate retrieval document set from the corpus based on the intermediate generated document set and the retrieval model.

[0101] Step 3: Fill the user query statement and the intermediate search document set into the text prompt template to obtain the intermediate text prompt information. Based on the intermediate text prompt information, the updated intermediate generated document set is obtained using a large language model.

[0102] Step 1, step 2 and step 3 are iteratively executed based on the number of iterations, and the obtained updated intermediate generated document set is used as the target generated document set.

[0103] A target retrieval document set is obtained from the corpus based on the target generation document set and the similarity function of the query-document pair in the retrieval model.

[0104] In one embodiment of the present disclosure, the target retrieval document set includes a plurality of retrieval documents, and the target generated document set includes a plurality of generated documents; The document rearrangement and screening module 23 is specifically used to calculate, for each retrieved document in the target retrieved document set: a first relevance score between the retrieved document and the target generated document set and the user query statement; For each generated document in the target generated document set: calculating a second relevance score between the generated document and the target retrieved document set and the user query statement; All generated documents and retrieved documents are sorted and screened based on the first relevance scores of the plurality of retrieved documents and the second relevance scores of the plurality of generated documents to obtain a target expanded query.

[0105] In one embodiment of the present disclosure, the document rearrangement and screening module 23 is further configured to calculate a first semantic relevance index between the retrieved document and the user query statement based on a similarity function; Calculate the second semantic relevance index between the retrieved document and all generated documents in the target generated document set based on the similarity function; A first relevance score of the retrieved document is calculated based on the first semantic relevance index and the second semantic relevance index; the first relevance score is an aggregate value of semantic similarities between the retrieved document and the user query statement and all generated documents.

[0106] In one embodiment of the present disclosure, the document rearrangement and screening module 23 is further configured to calculate a third semantic relevance index between the generated document and the user query statement based on a similarity function; Calculate the fourth semantic relevance index between the generated document and all the retrieved documents in the target retrieval document set based on the similarity function; A second relevance score of the generated document is calculated based on the third semantic relevance index and the fourth semantic relevance index; the second relevance score is an aggregate value of semantic similarities between the generated document and the user query statement and all retrieved documents.

[0107] In one embodiment of the present disclosure, the document re-ranking and screening module 23 is further configured to sort all the retrieved documents based on the first relevance scores of the multiple retrieved documents, and screen out the target retrieved document with the highest score.

[0108] All generated documents are sorted based on the second relevance scores of the multiple generated documents, and the target generated document with the highest score is selected.

[0109] A target expanded query is obtained based on the target retrieval document and the target generation document.

[0110] See also Fig. 9 , Fig. 9 A schematic block diagram of an electronic device provided by an embodiment of the present disclosure. Fig. 9 The electronic device 300 in the embodiment shown may include: one or more processors 301, one or more input devices 302, one or more output devices 303 and one or more memories 304. The processors 301, input devices 302, output devices 303 and memories 304 communicate with each other via a communication bus 305. The memory 304 is used to store computer programs, which include program instructions. The processor 301 is used to execute the program instructions stored in the memory 304. The processor 301 is configured to call the program instructions to execute the functions of the modules in the above-mentioned system embodiments, such as Figure 8The functions of the initial retrieval generation module 21, the iterative retrieval generation module 22 and the document re-ranking and screening module 23 are shown.

[0111] It should be understood that in the embodiment of the present disclosure, the processor 301 may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0112] The input device 302 may include a touch panel, a fingerprint collection sensor (for collecting the user's fingerprint information and fingerprint direction information), a microphone, etc., and the output device 303 may include a display (LCD, etc.), a speaker, etc.

[0113] The memory 304 may include a read-only memory and a random access memory, and provide instructions and data to the processor 301. A portion of the memory 304 may also include a non-volatile random access memory. For example, the memory 304 may also store information about the device type.

[0114] In a specific implementation, the processor 301, input device 302, and output device 303 described in the embodiments of the present disclosure can execute the implementation methods described in the first and second embodiments of the query method based on iterative retrieval generation verification provided in the embodiments of the present disclosure, and can also execute the implementation methods of the electronic device 300 described in the embodiments of the present disclosure, which will not be repeated here.

[0115] In another embodiment of the present disclosure, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by the processor, all or part of the processes in the above-mentioned embodiment method are implemented, and the computer program can also be completed by instructing the relevant hardware through the computer program. The computer program can be stored in a computer-readable storage medium, and when the computer program is executed by the processor, the steps of each of the above-mentioned method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form, etc. The computer-readable medium can include: any entity or device capable of carrying computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium, etc.

[0116] The computer-readable storage medium may be an internal storage unit of the electronic device of any of the aforementioned embodiments, such as a hard disk or memory of the electronic device. The computer-readable storage medium may also be an external storage device of the electronic device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device. Furthermore, the computer-readable storage medium may also include both an internal storage unit of the electronic device and an external storage device. The computer-readable storage medium is used to store computer programs and other programs and data required by the electronic device. The computer-readable storage medium may also be used to temporarily store data that has been output or is to be output.

[0117] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this disclosure.

[0118] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the electronic devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0119] In the several embodiments provided in the present application, it should be understood that the disclosed electronic device and method can be implemented in other ways. For example, the system embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces or units, or it can be an electrical, mechanical or other form of connection.

[0120] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiments of the present disclosure.

[0121] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0122] The above are only specific embodiments of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present disclosure, and these modifications or replacements should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.

Claims

1. A query method based on iterative retrieval generation verification, characterized in that: include: Based on the user query, the initial expanded query is obtained using a large language model; Acquire an initial retrieval document set from the corpus based on the initial expanded query and the retrieval model, the initial retrieval document set including a plurality of retrieval documents having a similarity score with the initial expanded query greater than or equal to a similarity threshold; Fill the user query statement and the initial search document set into the text prompt template to obtain text prompt information; Based on the text prompt information, the target generated document set is obtained using a large language model; Obtaining a target retrieval document set from a corpus based on a target generated document set and a retrieval model; The user query statement is expanded based on the target retrieval document set and the target generation document set to obtain a target expanded query.

2. The query method based on iterative retrieval generation verification as claimed in claim 1, characterized in that: The initial expanded query is obtained based on the user query statement using a large language model, including: Based on the user query, the initial generated document set is obtained using a large language model; Each element in the initially generated document set is linked to the user query statement to obtain an initial expanded query.

3. The query method based on iterative retrieval generation verification as claimed in claim 1, characterized in that: Based on the text prompt information, a target generated document set is obtained by using a large language model, and a target retrieved document set is obtained from a corpus based on the target generated document set and a retrieval model. include: Step 1: Based on the text prompt information, use the large language model to obtain the intermediate generated document set; Step 2: acquiring an intermediate retrieval document set from a corpus based on the intermediate generated document set and the retrieval model; Step 3: Fill the user query statement and the intermediate search document set into a text prompt template to obtain intermediate text prompt information; based on the intermediate text prompt information, use a large language model to obtain an updated intermediate generated document set; Iteratively execute step 1, step 2, and step 3 based on the number of iterations, and use the updated intermediate generated document set as the target generated document set; A target retrieval document set is obtained from a corpus based on the target generated document set and a similarity function of a query-document pair in a retrieval model.

4. The query method based on iterative retrieval generation verification as claimed in claim 1, characterized in that: The target retrieval document set includes a plurality of retrieval documents, and the target generated document set includes a plurality of generated documents; The step of expanding the user query statement based on the target retrieval document set and the target generation document set to obtain a target expanded query includes: For each retrieved document in the target retrieved document set: calculating a first relevance score between the retrieved document and the target generated document set and the user query statement; For each generated document in the target generated document set: calculating a second relevance score between the generated document and the target retrieved document set and the user query statement; All generated documents and retrieved documents are sorted and screened based on the first relevance scores of the plurality of retrieved documents and the second relevance scores of the plurality of generated documents to obtain a target expanded query.

5. The query method based on iterative retrieval generation verification as claimed in claim 4, characterized in that: Calculating a first relevance score between the retrieved document, the target generated document set and the user query statement, including: Calculate the first semantic relevance index between the retrieved document and the user query statement based on the similarity function; Calculate the second semantic relevance index between the retrieved document and all generated documents in the target generated document set based on the similarity function; A first relevance score of the retrieved document is calculated based on the first semantic relevance index and the second semantic relevance index; the first relevance score is an aggregate value of semantic similarities between the retrieved document and the user query statement and all generated documents.

6. The query method based on iterative retrieval generation verification as claimed in claim 4, characterized in that: Calculating a second relevance score between the generated document and the target retrieval document set and the user query statement, including: Calculate a third semantic relevance index between the generated document and the user query statement based on a similarity function; Calculate the fourth semantic relevance index between the generated document and all the retrieved documents in the target retrieval document set based on the similarity function; A second relevance score of the generated document is calculated based on the third semantic relevance index and the fourth semantic relevance index; the second relevance score is an aggregate value of semantic similarities between the generated document and the user query statement and all retrieved documents.

7. The query method based on iterative retrieval generation verification as claimed in claim 4, characterized in that: All generated documents and retrieved documents are sorted and screened based on the first relevance scores of the multiple retrieved documents and the second relevance scores of the multiple generated documents to obtain a target expanded query, including: sorting all the retrieved documents based on the first relevance scores of the multiple retrieved documents, and selecting the target retrieved document with the highest score; sorting all generated documents based on the second relevance scores of the multiple generated documents, and selecting the target generated document with the highest score; A target expansion query is obtained based on the target retrieval document and the target generation document.

8. A query system based on iterative retrieval generation and verification, characterized in that: include: An initial search generation module is used to obtain an initial expanded query based on a user query statement using a large language model; Acquire an initial retrieval document set from the corpus based on the initial expanded query and the retrieval model, the initial retrieval document set including a plurality of retrieval documents having a similarity score with the initial expanded query greater than or equal to a similarity threshold; An iterative retrieval generation module is used to fill the user query statement and the initial retrieval document set into the text prompt template to obtain text prompt information; Based on the text prompt information, the target generated document set is obtained using a large language model; Obtaining a target retrieval document set from a corpus based on a target generated document set and a retrieval model; The document rearrangement and screening module is used to expand the user query statement based on the target retrieval document set and the target generation document set to obtain the target expanded query.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images