Federal retrieval enhancement generation method for multiple data sources

Abstract

The invention provides a federal retrieval enhancement generation method oriented to multiple data sources. The method comprises the following steps: initializing the reliability of federal data sources; dividing a user query field to obtain the reliability of the federal data source to the user query field; selecting a target data source from the federated data source; performing local retrieval on the target data source to obtain candidate answers; estimating the confidence of the candidate answers; according to the confidence coefficient, filtering the candidate answers, and generating a user query answer; and according to the confidence, updating the reliability of the federal data source, and performing confidence estimation on the candidate answers, so that a confidence estimation result has rich semantic information and can be well identified and utilized by a large language model, thereby being beneficial to solving the conflict problem between exogenous knowledge and model endogenous knowledge, and improving the reliability of the model. And through federal data source quality detection of feedback iteration, reliability evaluation of the data source is continuously converged to the domain knowledge accuracy, efficient and accurate detection of the data source quality is realized, and query of invalid data sources and low-quality data sources is effectively reduced.

Description

Technical Field

[0001] This invention relates to the field of data retrieval, and more particularly to a federated retrieval enhancement generation method for multiple data sources. Background Technology

[0002] Search augmentation generation techniques enhance the accuracy and timeliness of large language models by retrieving external data sources based on user queries. Federated searches across multiple data sources can improve the coverage of search content. Current search augmentation generation techniques generally only consider the relevance between the user query and the retrieved content. However, data quality varies across different federated data sources, and some may contain factual errors. Existing methods typically only consider the relevance of the retrieved content, and content relevant to the user query may not provide accurate information, thus misleading the model's output. Furthermore, the data quality and distribution of each federated data source are unknown. Performing quality checks and distribution characteristic analysis on all federated data sources individually would consume significant computational and human resources, leading to inefficiency in scenarios with large datasets. Summary of the Invention

[0003] The purpose of this invention is to provide an enhanced federated retrieval generation method for multiple data sources. The method initializes the reliability of the federated data sources; divides the user query domain to obtain the reliability of the federated data sources for the user query domain; selects a target data source from the federated data sources; performs local retrieval on the target data sources to obtain candidate answers; estimates the confidence of the candidate answers; filters the candidate answers based on the confidence, and generates a user query response; updates the reliability of the federated data sources based on the confidence, and estimates the confidence of the candidate answers. This ensures that the confidence estimation results have rich semantic information, enabling them to be well recognized and utilized by large language models. This helps resolve the conflict between external knowledge and model-internal knowledge. Furthermore, through feedback iterative federated data source quality detection, the reliability assessment of the data sources continuously converges to the domain knowledge accuracy, achieving efficient and accurate detection of data source quality and effectively reducing queries to invalid and low-quality data sources.

[0004] This invention is achieved through the following technical solution: Enhanced generation methods for federated search across multiple data sources include: Initialize the reliability of the federated data source; divide the user query domain to obtain the reliability of the federated data source for the user query domain; Select a target data source from the federated data source; perform a local search on the target data source to obtain candidate answers; Estimate the confidence level of the candidate answers; filter the candidate answers based on the confidence level to generate a response to the user query; The reliability of the federated data source is updated based on the confidence level.

[0005] Optionally, initialize the reliability of the federated data sources, including: For N federated data sources , according to the prior knowledge of the federated data source , initialize the corresponding domain set of the federated data source , where D is the total number of domains; Determine the reliability of the federated data source in each domain . .

[0006] Optionally, divide the user query domain to obtain the reliability of the federated data source for the user query domain, including: For the user query q, set the maximum number of selectable domains from the domain set to be D max ; Use the pre-trained topic model to obtain the score of the user query q in each domain , sort according to the obtained descending order, and perform corresponding sorting on the domain set to obtain a sorted set ; ; According to the sorted set , perform user query domain division to obtain the reliability of the federated data source for the user query domain.

[0007] Optionally, according to the sorted set , perform user query domain division to obtain the reliability of the federated data source for the user query domain, including: Set the score of each domain in the sorted set , where j = 1, 2... D, and use the domain as the initial domain; For the case of 1 < j < D, if , then add the domain to the division result, so as to obtain K divided domain sets , where, is a preset score ratio threshold, ; ; According to the following formula, determine the reliability of the federated data source for the user query domain, ; ​​​ Regarding the reliability of the user query q, Indicates the federal data source In the field The reliability of.

[0008] Optionally, selecting a target data source from the federated data source includes: For N federated data sources When the reliability of the federated data source for the domain of user query q Higher than the preset reliability threshold At that time, the federated data source is assigned to the first candidate data source set. ; The reliability of the domain of user query q Not higher than the preset reliability threshold All federated data sources are subjected to a secondary screening, and the filtered federated data sources are assigned to the second candidate data source set. ; The first candidate data source set and the second candidate data source set Merge to obtain the target data source set ;in, , For the target data source, The total number of target data sources.

[0009] Optionally, a local search is performed on the target data source to obtain candidate answers, including: For the target data source set Based on the user's query 'q', a local search is performed to obtain the retrieved document; Based on the user query q and the retrieved document, from each target data source Extract and generate a set of candidate answers ;in, To obtain from the target data source Extracted candidate answers, This represents the total number of candidate answers.

[0010] Optionally, estimating the confidence level of the candidate answers includes: target data source The corresponding set of candidate answers Divided into M categories Each of the categories is divided into = , The target data source from which each candidate answer comes. For target data source Reliability, To classify The total number of candidate answers, and ; Then determine the classification The confidence level.

[0011] Optionally, determine the classification. The confidence level includes: Using the formula below, based on Bayesian inference, the classification is obtained. confidence level , ； in, , , Target data source , , Reliability, , Classification , Total number of candidate answers.

[0012] Optionally, based on the confidence level, the candidate answers are filtered to generate a user query response, including: Compare all classifications confidence level Determine whether to retain the corresponding classification. Candidate answers, and filtering out other classification categories. The candidate answers; The candidate answers that are retained and classified into categories and their confidence scores are put into the context of the prompt words. The context of the prompt words is processed by a large language model to generate user query questions and answers. Optionally, the reliability of the federated data source is updated based on the confidence level, including: The confidence level for splitting candidate answers is set as follows: For each target data source Its classification category is The classification The confidence level is Then in the domain of user query q Reliability after the update for: ; in, For target data source In the field Reliability, For target data source In the field The next round has already been updated.

[0013] Compared with the prior art, the present invention has the following beneficial effects: This application presents an enhanced federated retrieval generation method for multiple data sources. It employs a Bayesian inference-based candidate answer confidence estimation method, resulting in rich semantic information that can be readily recognized and utilized by large language models, helping to resolve conflicts between external and internal model knowledge. Furthermore, it designs a feedback-iterative federated data source quality detection method, ensuring that the reliability assessment of the data source continuously converges to the domain knowledge accuracy, independent of the initialization rounds and initial reliability. This achieves efficient and accurate detection of data source quality through continuous iterative feedback. Finally, it designs a greedy strategy-based federated data source selection method based on a decay coefficient, gradually converging from federated data source exploration to data source utilization. This enables accurate modeling of the domain reliability of data sources with unknown distributions, effectively reducing queries to irrelevant data sources and low-quality data. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein: Figure 1 This is a flowchart illustrating the enhanced federated retrieval generation method for multiple data sources provided by the present invention.

[0015] Figure 2 This is the implementation process of an embodiment of the federated search enhancement generation method for multiple data sources. Detailed Implementation

[0016] To make the above-mentioned objectives, features, and advantages of this application more apparent and understandable, the specific embodiments of this application will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, it should be noted that, for ease of description, only the parts relevant to this application are shown in the accompanying drawings, not the entire structure. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without inventive effort are within the scope of protection of this application.

[0017] The terms “comprising” and “having”, and any variations thereof, used in this application are intended to cover non-exclusive inclusion. For example, a process, method, product, or apparatus that includes a series of steps or units is not limited to the steps or units listed, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such process, method, product, or apparatus.

[0018] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0019] Please see Figure 1 As shown, an embodiment of this application provides a method for enhancing federated search generation for multiple data sources. This method includes: Initialize the reliability of the federated data source; divide the user query domain to obtain the reliability of the federated data source for the user query domain; Select the target data source from the federated data source; perform a local search on the target data source to obtain candidate answers; Estimate the confidence level of candidate answers; filter candidate answers based on the confidence level and generate a response to the user query; Update the reliability of the federated data source based on the confidence level.

[0020] The beneficial effects of the above embodiments are as follows: This multi-data source-oriented federated retrieval enhancement generation method initializes the reliability of the federated data source; divides the user query domain to obtain the reliability of the federated data source for the user query domain; selects a target data source from the federated data source; performs local retrieval on the target data source to obtain candidate answers; estimates the confidence of the candidate answers; filters the candidate answers based on the confidence, and generates the user query answer; updates the reliability of the federated data source based on the confidence, and estimates the confidence of the candidate answers, so that the confidence estimation results have rich semantic information, which can be well recognized and utilized by large language models, helping to solve the conflict between external knowledge and model internal knowledge. Furthermore, through feedback iterative federated data source quality detection, the reliability assessment of the data source continuously converges to the domain knowledge accuracy, achieving efficient and accurate detection of data source quality, and effectively reducing queries to invalid and low-quality data sources.

[0021] In another embodiment, initializing the reliability of the federated data source includes: For N federated data sources According to federal data sources Prior knowledge to initialize the federated data sources The corresponding domain set , where D is the total number of domains; Determine the federated data sources In each domain Reliability .

[0022] For N federated data sources , considering the problem defined within the D domain sets , according to the prior knowledge of differentiating data types for the data sources, divide the federated data sources into multiple different domains, thus forming a domain set with D domains , where these domains can include three different domains: good domains, bad domains, and trivial domains. The above three domains are well-known in this field and will not be introduced in detail here. Then determine each federated data source In each domain Reliability , the above reliability As the initial reliability of each federated data , serving as the origin of the feedback iteration for the federated data sources later.

[0023] In another embodiment, divide the user query domain to obtain the reliability of the federated data sources for the user query domain, including: For the user query q, set the maximum number of selectable domains from the domain set to be D max ; Use the pre-trained topic model to obtain the score of the user query q in each domain , sort in descending order according to the obtained scores, and perform corresponding sorting on the domain set to obtain the sorted set ; ; According to the sorted set , perform user query domain division to obtain the reliability of the federated data sources for the user query domain.

[0024] In another embodiment, according to the sorted set , perform user query domain division to obtain the reliability of the federated data sources for the user query domain, including: Set the score of each domain in the sorted set , where j = 1, 2... D, and take the domain as the initial domain; ; For the case of 1 < j < D, if , then add the domain to the partitioning result, thereby obtaining K partitioning domain sets , where is a preset score ratio threshold, ; Determine the reliability of the federated data source for the domain of the user query according to the following formula ; In the above formula, represents the reliability of the federated data source for the user query q, represents the reliability of the federated data source in the domain .

[0025] To accurately perform domain partitioning for user queries, an adaptive partitioning method is introduced here to determine multiple domains that may be involved, and the maximum number of selectable domains is set to D max , preset score ratio threshold . First, use the pre-trained topic model to obtain the score of the user query q in each domain , sort them in descending order, and thus reorder all domains into a sorted set . For any two adjacent domains and in the above sorted set (where 1 < j < D), their corresponding scores both satisfy . Select the domain with the highest score as the initial domain. For the case of 1 < j < D, if , then add the domain to the partitioning result, thereby obtaining K partitioning domain sets , which can accurately and comprehensively partition the federated data source at the domain level. Also calculate the reliability of the federated data source in the domain according to the above formula, which can ensure the precise association and matching between the federated data source and the domain.

[0026] In another embodiment, selecting a target data source from the federated data sources includes: For N federated data sources , when the reliability of the federated data source for the domain of the user query q is higher than the preset reliability threshold , partition the federated data source into the first candidate data source set The reliability of the domain of user query q Not higher than the preset reliability threshold All federated data sources are subjected to a secondary screening, and the filtered federated data sources are assigned to the second candidate data source set. ; Set the first candidate data source Second candidate data source set Merge to obtain the target data source set ;in, , For the target data source, The total number of target data sources.

[0027] The goal of selecting federated data sources is to achieve a balance between exploration and utilization, choosing the most relevant data sources that are most likely to provide accurate information. To this end, a decay-based approach is designed. Greedy strategy, for a preset reliability threshold When the reliability of the federated data source for the domain of user query q Higher than the preset reliability threshold At that time, the federated data source is assigned to the first candidate data source set. ; Reliability of the domain for user query q Not higher than the preset reliability threshold All federal data sources still have (where 0 < The exploration probability of <1) was selected and assigned to the second candidate data source set. This corresponds to the selection and allocation to the second candidate data source set mentioned above. The goal is to allow low-scoring data sources to still have a chance of being selected early on, thereby obtaining more accurate reliability feedback during iterations, eliminating the influence of initialization on data source reliability scores, and facilitating convergence from data source exploration to data source utilization by exploring probabilities. The index will continuously decrease as the user queries it, meaning the decay rate is [missing information]. After each selection of a federated data source, the exploration probability is updated to... The final target data source selection is to choose the first set of candidate data sources. Second candidate data source set Merge to obtain the target data source set This ensures that the selected target data source is accurate and reliable. By designing a federated data source selection method based on a greedy strategy using a decay coefficient, the method gradually converges from federated data source exploration to data source utilization, achieving accurate modeling of the reliability of data source domains with unknown distributions, and effectively reducing queries to irrelevant data sources and low-quality data.

[0028] In another embodiment, a local search is performed on the target data source to obtain candidate answers, including: For the target data source collection Based on the user's query 'q', a local search is performed to obtain the retrieved document; Based on the user query q and the retrieved document, from each target data source Extract and generate a set of candidate answers ;in, To obtain from the target data source Extracted candidate answers, This represents the total number of candidate answers.

[0029] In practice, during the local retrieval process based on the user query q, both sparse and dense retrieval methods are supported. For the retrieved files, an extraction model is used to propose a set of candidate answers based on the user query q and the information of the retrieved documents. The above extraction model can be implemented using a large language model to ensure the accuracy of the generated candidate answers.

[0030] In another embodiment, estimating the confidence of candidate answers includes: using the target data source The corresponding set of candidate answers Divided into M categories Each of the categories is divided into = , The target data source from which each candidate answer comes. For target data source Reliability, To classify The total number of candidate answers, and ; Then determine the classification The confidence level.

[0031] In another embodiment, the classification is determined. The confidence level includes: Using the formula below, based on Bayesian inference, the classification is obtained. confidence level , ； in, , , Target data source , , Reliability, , Classification , Total number of candidate answers.

[0032] In practice, the candidate answers from the target data source are first divided into M categories. In the implementation of the specific segmentation method, string matching is used to determine whether different candidate answers belong to the same segmentation category. The data source is the smallest unit between different segmentation categories; that is, candidate answers from the same data source must belong to the same segmentation category. A segmentation category can have candidate answers from multiple data sources, and multiple candidate answers from the same data source can exist, thus enabling confidence estimation of candidate answers for multi-answer questions. By designing a candidate answer confidence estimation method based on Bayesian inference, the confidence estimation results possess rich semantic information, enabling them to be well recognized and utilized by large language models, and helping to resolve the conflict between external knowledge and model-internal knowledge.

[0033] In another embodiment, candidate answers are filtered based on confidence level to generate a user query answer, including: Compare all classifications confidence level Determine whether to retain the corresponding classification. Candidate answers, and filtering out other classification categories. Candidate answers; for example, for each category. confidence level Perform threshold comparison, if confidence level If the value exceeds the preset confidence threshold, the corresponding classification will be retained. Select the candidate answers; otherwise, filter the corresponding categories. The candidate answers; The candidate answers that are retained and classified into categories and their confidence scores are put into the context of the prompt words. The context of the prompt words is processed by a large language model to generate user query questions and answers. In practice, candidate answers need to be filtered before the answer is generated to achieve a dynamic balance between selective information compression and comprehensive information retention, taking into account both computational efficiency and output quality. To this end, an adaptive filtering method is designed: when a candidate answer has a significantly higher confidence level than others, these high-confidence candidate answers are prioritized for retention; when no single candidate answer dominates, all are retained without filtering. The retained candidate answers and their confidence levels are then placed within the context of the prompt words, allowing the large language model to combine its internal knowledge to make a comprehensive judgment and output.

[0034] In another embodiment, updating the reliability of the federated data source based on the confidence level includes: The confidence level for splitting candidate answers is set as follows: For each target data source Its classification category is Classification The confidence level is Then in the domain of user query q Reliability after the update for: ; in, For target data source In the field Reliability, For target data source In the field The next round has already been updated.

[0035] By designing a federated data source quality detection method based on feedback iteration, the reliability assessment of the data source continuously converges to the domain knowledge accuracy, regardless of the initialization round and initial reliability. This enables efficient and accurate detection of data source quality through continuous iterative feedback.

[0036] In summary, this enhanced generation method for federated retrieval across multiple data sources initializes the reliability of the federated data sources; divides the user query domain to obtain the reliability of the federated data sources for the user query domain; selects a target data source from the federated data sources; performs local retrieval on the target data sources to obtain candidate answers; estimates the confidence of the candidate answers; filters the candidate answers based on the confidence, and generates the user query response; updates the reliability of the federated data sources based on the confidence, and estimates the confidence of the candidate answers, ensuring that the confidence estimation results have rich semantic information that can be well recognized and utilized by large language models. This helps resolve the conflict between external knowledge and model-internal knowledge. Furthermore, through feedback iterative federated data source quality detection, the reliability assessment of the data sources continuously converges to the domain knowledge accuracy, achieving efficient and accurate detection of data source quality and effectively reducing queries to invalid and low-quality data sources.

[0037] The above is only one specific embodiment of the present invention, and any improvements made based on the concept of the present invention shall be considered within the scope of protection of the present invention.

Claims

1. A federated search enhancement generation method for multiple data sources, characterized in that, include: Initialize the reliability of the federated data source; The reliability of the federated data source for the user's query domain is obtained by dividing the user's query domain into categories. Select a target data source from the federated data source; A local search is performed on the target data source to obtain candidate answers; Estimate the confidence level of the candidate answers; Based on the confidence level, the candidate answers are filtered to generate a response to the user query; The reliability of the federated data source is updated based on the confidence level.

2. The federated search enhancement generation method for multiple data sources as described in claim 1, characterized in that: Initialize the reliability of the federated data source, including: For N federated data sources According to federal data sources The prior knowledge is used to initialize the federated data source. Corresponding domain set , where D is the total number of domains; Determine the federal data source In every field Reliability .

3. The federated search enhancement generation method for multiple data sources as described in claim 2, characterized in that: The reliability of the federated data source for the user's query domain is obtained by dividing the user's query domain into categories, including: For user query q, set from the set of domains The maximum number of optional domains is D max ; The user query q is obtained in each domain using a pre-trained topic model. Score According to the descending order obtained, the domain set is processed. Perform the corresponding sorting to obtain the sorted set. ; According to the sorted set The user query domain is segmented to obtain the reliability of the federated data source for the user query domain.

4. The federated search enhancement generation method for multiple data sources as described in claim 3, characterized in that: According to the sorted set The system performs user query domain segmentation to obtain the reliability of the federated data source for the user query domain, including: Define the sorted set Each field Score Where j = 1, 2...D, and the domain is... As the initial field; For the case where 1 < j < D, if , then the domain is added to the partitioning result, thereby obtaining K partitioning domain sets , where is a preset score ratio threshold, ; The reliability of the federated data source for the domain queried by the user is determined according to the following formula. ； In the above formula, Indicates the federal data source Regarding the reliability of the user query q, Indicates the federal data source In the field The reliability of.

5. The federated search enhancement generation method for multiple data sources as described in claim 1, characterized in that: Selecting a target data source from the federated data source includes: For N federated data sources When the reliability of the federated data source for the domain of user query q Higher than the preset reliability threshold At that time, the federated data source is assigned to the first candidate data source set. ; The reliability of the domain of user query q Not higher than the preset reliability threshold All federated data sources are subjected to a secondary screening, and the filtered federated data sources are assigned to the second candidate data source set. ; The first candidate data source set and the second candidate data source set Merge to obtain the target data source set ;in, , For the target data source, The total number of target data sources.

6. The federated search enhancement generation method for multiple data sources as described in claim 5, characterized in that: A local search is performed on the target data source to obtain candidate answers, including: For the target data source set Based on the user's query 'q', a local search is performed to obtain the searched document; Based on the user query q and the retrieved document, from each target data source Extract and generate a set of candidate answers ;in, To obtain from the target data source Extracted candidate answers, This represents the total number of candidate answers.

7. The federated search enhancement generation method for multiple data sources as described in claim 1, characterized in that: Estimating the confidence level of the candidate answers includes: target data source The corresponding set of candidate answers Divided into M categories Each of the categories = , The target data source from which each candidate answer comes. For target data source Reliability, To classify The total number of candidate answers, and ; Then determine the classification The confidence level.

8. The federated search enhancement generation method for multiple data sources as described in claim 7, characterized in that: Determine the classification The confidence level includes: Using the formula below, based on Bayesian inference, the classification is obtained. confidence level , ； in, , , Target data source , , Reliability, , They are categorized respectively , Total number of candidate answers.

9. The federated search enhancement generation method for multiple data sources as described in claim 8, characterized in that: Based on the confidence level, the candidate answers are filtered to generate a user query response, including: Compare all classifications confidence level Determine whether to retain the corresponding classification. Candidate answers, and filtering out other classification categories. The candidate answers; The retained candidate answers and their confidence scores are placed into the context of the prompt words, and the context of the prompt words is processed using a large language model to generate user query questions and answers.

10. The federated search enhancement generation method for multiple data sources as described in claim 1, characterized in that: Based on the confidence level, update the reliability of the federated data source, including: The confidence level for splitting candidate answers is set as follows: For each target data source Its classification category is The classification The confidence level is Then in the domain of user query q Reliability after the update for: ； in, For target data source In the field Reliability, For target data source In the field The next round has already been updated.

Images