Retrieval optimization method and retrieval optimization device

Abstract

The invention relates to a retrieval optimization method and a retrieval optimization device. The method comprises the steps of performing plan making processing based on a pre-acquired user problem to obtain a plan list; performing retrieval processing according to the plan list and a pre-established knowledge base, and performing iterative processing according to retrieval process data in the retrieval process to obtain a preliminary retrieval result; and performing uncertainty evaluation processing on the preliminary retrieval result, and determining a target retrieval result according to an uncertainty evaluation result and the preliminary retrieval result. According to the method, the plan list containing the progressive relation retrieval tasks is formulated based on the user problems, and user requirements can be gradually and deeply mined. Iterative processing is carried out according to retrieval process data in the retrieval process, and the retrieval strategy can be dynamically adjusted according to the real-time retrieval condition. And performing uncertainty evaluation on the preliminary retrieval result, and determining a target retrieval result, so that fuzzy or unreliable information in the retrieval process can be effectively identified and processed.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and particularly to a retrieval optimization method and a retrieval optimization device. Background Art

[0002] With the development of LLM (Large Language Model), its application in natural language generation tasks is becoming increasingly widespread. However, general large models have knowledge limitations, hallucination problems, and data security risks in professional scenarios.

[0003] To solve these problems, the RAG (Retrieval Augmented Generation) technology has emerged. It improves the performance of large models in actual business through the "retrieval + generation" mode.

[0004] However, the accuracy of the retrieval results generated by existing RAG technologies is difficult to meet actual requirements. Summary of the Invention

[0005] Based on this, it is necessary to provide a retrieval optimization method and a retrieval optimization device that can solve the above problems in view of the above technical problems.

[0006] In a first aspect, this application provides a retrieval optimization method. The method includes:

[0007] Performing plan formulation processing based on a pre-obtained user question to obtain a plan list; the plan list includes multiple retrieval tasks, and there is a progressive relationship between the retrieval tasks;

[0008] Performing retrieval processing according to the plan list and a pre-established knowledge base, and performing iterative processing according to the retrieval process data during the retrieval process to obtain a preliminary retrieval result;

[0009] Performing uncertainty evaluation processing on the preliminary retrieval result, and determining a target retrieval result according to the uncertainty evaluation result and the preliminary retrieval result.

[0010] In one of the embodiments, the above-mentioned performing retrieval processing according to the plan list and a pre-established knowledge base, and performing iterative processing according to the retrieval process data during the retrieval process to obtain a preliminary retrieval result includes:

[0011] Performing retrieval task extraction processing on the plan list to obtain a target retrieval task;

[0012] Performing retrieval processing on the knowledge base based on the target retrieval task and a preset retrieval strategy to obtain retrieval process data;

[0013] Perform relevance evaluation and integrity evaluation on the retrieval process data to obtain a retrieval evaluation result;

[0014] Adjust the preset retrieval strategy according to the retrieval evaluation result, and re - retrieve according to the adjusted retrieval strategy to obtain a preliminary retrieval result.

[0015] In one embodiment, the above - mentioned retrieval evaluation result includes a relevance evaluation result and an integrity evaluation result; adjusting the preset retrieval strategy according to the retrieval evaluation result, and re - retrieving according to the adjusted retrieval strategy to obtain a preliminary retrieval result, including:

[0016] In the case where the relevance evaluation result does not meet the preset relevance condition, modify the keywords and query logic of the preset retrieval strategy to obtain a relevance adjustment strategy;

[0017] In the case where the integrity evaluation result does not meet the preset integrity condition, expand the retrieval scope and increase the retrieval depth of the preset retrieval strategy to obtain an integrity adjustment strategy;

[0018] Adjust the preset retrieval strategy according to the relevance adjustment strategy and / or the integrity adjustment strategy to obtain an adjusted retrieval strategy;

[0019] Re - perform the retrieval process according to the adjusted retrieval strategy until the relevance evaluation result meets the preset relevance condition and the integrity evaluation result meets the preset integrity condition to obtain a preliminary retrieval result.

[0020] In one embodiment, the above - mentioned method further includes:

[0021] In the case where the retrieval process data does not meet the result generation requirement, update the plan list to obtain an updated plan list;

[0022] Return to the step of performing retrieval task extraction on the plan list to obtain a target retrieval task.

[0023] In one embodiment, perform uncertainty evaluation on the preliminary retrieval result, and determine the target retrieval result according to the uncertainty evaluation result and the preliminary retrieval result. Obtaining the target retrieval result includes:

[0024] Perform uncertainty quantification on the preliminary retrieval result to obtain a quantification result; the quantification result is used to characterize whether the preliminary retrieval result is uncertain data;

[0025] In the case where the quantification result characterizes uncertain data, optimize the preliminary retrieval result to obtain the target retrieval result.

[0026] In one embodiment, the above-mentioned uncertainty quantification processing of the preliminary retrieval results to obtain a quantification result includes:

[0027] Calculating an information quantity index score and a generated content confidence score of the preliminary retrieval results;

[0028] Obtaining a quantification result according to the information quantity index score and the generated content confidence score.

[0029] In one embodiment, the above-mentioned optimization processing of the preliminary retrieval results to obtain a target retrieval result includes:

[0030] Generating a prompt message according to the quantification result, and obtaining feedback information of the user according to the prompt message;

[0031] Adjusting the retrieval query strategy according to the feedback information, and optimizing the preliminary retrieval results based on the adjusted retrieval query strategy to obtain a target retrieval result.

[0032] In one embodiment, the above-mentioned process of formulating a plan based on a pre-obtained user question to obtain a plan list includes:

[0033] Performing knowledge extraction processing on the user question to obtain associated knowledge;

[0034] Performing intention recognition and information enhancement processing on the user question and a pre-stored historical conversation to obtain key information;

[0035] Formulating a plan based on the associated knowledge and the key information to obtain a plan list.

[0036] In one embodiment, the above-mentioned method further includes:

[0037] Performing an explanation generation process on the retrieval process according to the retrieval process data and the target retrieval result to obtain an explanation content; the explanation content is used to represent the basis of each step in the retrieval process, the data source, and the association relationship between the target retrieval result and the user requirement.

[0038] In a second aspect, the present application further provides a retrieval optimization device. The device includes:

[0039] A list formulation module, configured to perform a plan formulation process based on a pre-obtained user question to obtain a plan list; the plan list includes multiple retrieval tasks, and there is a progressive relationship between the retrieval tasks;

[0040] A preliminary retrieval module, configured to perform a retrieval process according to the plan list and a pre-established knowledge base, and perform iterative processing according to the retrieval process data during the retrieval process to obtain a preliminary retrieval result;

[0041] A result evaluation module is used to perform uncertainty evaluation processing on the preliminary retrieval results, and determine the target retrieval results according to the uncertainty evaluation results and the preliminary retrieval results.

[0042] In a third aspect, the present application also provides a computer device. The computer device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the steps of the method in the first aspect are implemented.

[0043] In a fourth aspect, the present application also provides a computer-readable storage medium. On the computer-readable storage medium, a computer program is stored, and when the computer program is executed by a processor, the steps of the method in the first aspect are implemented.

[0044] In a fifth aspect, the present application also provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the steps of the method in the first aspect are implemented.

[0045] The above retrieval optimization method and retrieval optimization device perform plan formulation processing based on a pre-acquired user question to obtain a plan list; perform retrieval processing according to the plan list and a pre-established knowledge base, and perform iterative processing according to the retrieval process data during the retrieval process to obtain preliminary retrieval results; perform uncertainty evaluation processing on the preliminary retrieval results, and determine the target retrieval results according to the uncertainty evaluation results and the preliminary retrieval results. The present application formulates a plan list containing retrieval tasks with a progressive relationship based on the user question, and can gradually dig deeper into the user's needs. During the retrieval process, iterative processing is performed based on the retrieval process data, and the retrieval strategy can be dynamically adjusted according to the real-time retrieval situation. Uncertainty evaluation is performed on the preliminary retrieval results, and the target retrieval results are determined accordingly, which can effectively identify and process fuzzy or unreliable information during the retrieval process. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 It is an application environment diagram of the retrieval optimization method in an embodiment;

[0047] Figure 2 It is a flowchart of the retrieval optimization method in an embodiment;

[0048] Figure 3 It is a flowchart of the retrieval processing and iterative processing in an embodiment;

[0049] Figure 4 It is a flowchart of the retrieval strategy adjustment and re-retrieval in an embodiment;

[0050] Figure 5 It is a flowchart of the update processing of the plan list in an embodiment;

[0051] Figure 6 Schematic diagram of the uncertainty assessment and optimization process in an embodiment;

[0052] Figure 7 Schematic diagram of the uncertainty quantification process in an embodiment;

[0053] Figure 8 Schematic diagram of the optimization process for the preliminary retrieval results in an embodiment;

[0054] Figure 9 Schematic diagram of the process for obtaining the plan list in an embodiment;

[0055] Figure 10 Block diagram of the structure of the retrieval optimization device in an embodiment;

[0056] Figure 11 Internal structure diagram of a computer device in an embodiment. Detailed implementation manners

[0057] In order to make the objectives, technical solutions and advantages of the present application clearer and more understandable, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0058] First, before specifically introducing the technical solutions of the embodiments of the present application, the technical background on which the embodiments of the present application are based will be introduced.

[0059] The wave of large language models (LLMs) has swept through almost all industries. However, when it comes to professional scenarios or industry-specific fields, general LLMs will face the problem of insufficient professional knowledge. Compared with the costly post-training or supervised fine-tuning (SFT), the technical solution based on retrieval augmented generation (RAG) often becomes a better choice. Retrieval Augmented Generation, abbreviated as RAG, is the most popular LLM application solution at present. When applying LLMs to actual business scenarios, it will be found that general basic LLMs basically cannot meet the actual business needs, mainly for the following reasons:

[0060] Limitations of knowledge: The knowledge of the model itself completely comes from its training data, and the training sets of existing mainstream LLMs are basically constructed from publicly available data on the Internet. For some real-time, non-public or offline data, it is impossible to obtain, and this part of knowledge cannot be possessed.

[0061] Hallucination problem: The underlying principle of all AI models is based on mathematical probability, and their model outputs are essentially a series of numerical operations. Large models are no exception. Therefore, they sometimes talk nonsense seriously, especially in scenarios where the large model itself lacks knowledge in a certain aspect or is not good at. And it is relatively difficult to distinguish this hallucination problem because it requires the user to have knowledge in the corresponding field.

[0062] Data security: For any enterprise or institution, data security is crucial. No organization is willing to take the risk of data leakage and upload its private data to a third-party platform for training. This also leads to the application solutions that completely rely on the capabilities of general large models having to make trade-offs between data security and effectiveness.

[0063] And RAG is an effective solution to solve the above problems. However, although the current RAG judges whether the generated result meets the expectations through a reflection mechanism, it does not clearly mention how to handle the uncertainties in the retrieval or generation process. For example, when the retrieved information is insufficient or ambiguous, how the system prompts the user and provides optimization suggestions.

[0064] Based on this, the present application provides a retrieval optimization method and a retrieval optimization device, aiming to solve the above technical problems.

[0065] The retrieval optimization method provided by the embodiments of the present application can be applied to, for example, Figure 1 the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or placed in the cloud or other network servers. The terminal 102 performs plan-making processing based on the pre-obtained user question to obtain a plan list; performs retrieval processing according to the plan list and the pre-established knowledge base, and performs iterative processing according to the retrieval process data during the retrieval process to obtain a preliminary retrieval result; performs uncertainty evaluation processing on the preliminary retrieval result, and determines the target retrieval result according to the uncertainty evaluation result and the preliminary retrieval result. Among them, the terminal 102 can be, but is not limited to, various personal computers, laptop computers, smart phones, and tablet computers. The server 104 can be implemented by an independent server or a server cluster composed of multiple servers.

[0066] In an exemplary embodiment, as Figure 2 shown, the present application provides a retrieval optimization method. Taking the method applied to the Figure 1 terminal 102 in it as an example for description, it includes the following steps:

[0067] S201, perform plan-making processing based on the pre-obtained user question to obtain a plan list.

[0068] Among them, the plan list terminal is a set of a series of retrieval tasks formulated according to the user's question. These tasks revolve around solving the user's question, and there is a progressive relationship among the tasks, aiming to guide the terminal to gradually and deeply obtain relevant information and provide a basis for generating an accurate answer.

[0069] In the embodiment of the present application, after receiving the user's question, the terminal uses natural language processing technology to parse the question. For example, when the user asks "the latest applications of artificial intelligence in medical image diagnosis", the terminal first extracts key information such as "artificial intelligence", "medical image diagnosis", and "latest applications". Then, according to the logical relationship of this information and the structure of the knowledge base, progressive retrieval tasks are formulated. The first retrieval task is to retrieve the basic applications of artificial intelligence in the field of medical image diagnosis to understand the basic situation of this field; the second retrieval task focuses on the new applications of artificial intelligence in medical image diagnosis in the past five years; the third retrieval task is to specifically search for the latest application cases in the past year. In this way, the terminal generates a plan list containing multiple progressive retrieval tasks, providing a clear direction for subsequent retrieval.

[0070] Another embodiment: After obtaining the user's question, the terminal first checks the local cache to see if there is a retrieval record of a similar question. If the user asks "breakthroughs in electric vehicle battery technology" and the terminal finds a retrieval record about "development of battery technology" locally, it uses this as a reference to formulate a plan list. It will first retrieve the current situation of electric vehicle battery technology as basic information; then retrieve the improvement directions that have emerged on the basis of the existing technology; finally, retrieve the achieved breakthrough results. This method can formulate a plan list more efficiently by means of historical retrieval records, and at the same time, it can also combine past experience to make the plan more in line with the user's needs.

[0071] S202, perform a retrieval process according to the plan list and the pre-established knowledge base, and perform iterative processing according to the retrieval process data during the retrieval process to obtain a preliminary retrieval result.

[0072] Among them, the retrieval process data refers to the data generated during the execution of the retrieval task, including information such as the retrieved document content, the relevance score of the retrieval result, the retrieval time, and the data source of the retrieval.

[0073] In the embodiment of the present application, the terminal starts from the first retrieval task according to the plan list and obtains relevant information from the knowledge base. When performing the second retrieval task, if the terminal finds that in the basic application information retrieved previously, a certain algorithm has a new improvement direction in recent years. The terminal can adjust the retrieval keywords according to this retrieval process data, add vocabulary related to the improvement of this algorithm, and retrieve again. After multiple iterations, the terminal integrates all the retrieved information.

[0074] Another embodiment: When the terminal executes a retrieval task, it also adopts a combination of parallel retrieval and iteration.

[0075] S203. Perform an uncertainty evaluation process on the preliminary retrieval results, and determine the target retrieval results according to the uncertainty evaluation results and the preliminary retrieval results.

[0076] In the embodiment of the present application, the terminal first performs an uncertainty evaluation on the preliminary retrieval results from multiple dimensions. First, evaluate the reliability of the information source. For example, if a piece of information about the latest application comes from a non-professional and authoritative data source, its credibility will be reduced. Second, judge the relevance between the information and the user's question, and reduce the weight of the information that only slightly mentions but does not cover the latest application. After evaluation, the terminal filters out the information with high credibility and strong relevance as the target retrieval results.

[0077] Another embodiment: In addition to considering the reliability and relevance of the information itself, the terminal also introduces user preference data for uncertainty evaluation.

[0078] The above retrieval optimization method performs a plan formulation process based on the pre-acquired user question to obtain a plan list; performs a retrieval process according to the plan list and the pre-established knowledge base, and performs an iterative process according to the retrieval process data during the retrieval process to obtain preliminary retrieval results; performs an uncertainty evaluation process on the preliminary retrieval results, and determines the target retrieval results according to the uncertainty evaluation results and the preliminary retrieval results. The present application formulates a plan list containing progressive retrieval tasks based on the user question, which can gradually dig deeper into the user's needs. During the retrieval process, an iterative process is performed based on the retrieval process data, and the retrieval strategy can be dynamically adjusted according to the real-time retrieval situation. An uncertainty evaluation is performed on the preliminary retrieval results, and the target retrieval results are determined accordingly, which can effectively identify and process fuzzy or unreliable information during the retrieval process.

[0079] In an exemplary embodiment, based on the above embodiment, please refer to Figure 3 , the process of the embodiment of the present application for performing a retrieval process according to the plan list and the pre-established knowledge base, and performing an iterative process according to the retrieval process data during the retrieval process to obtain preliminary retrieval results includes the following steps:

[0080] S301. Perform a retrieval task extraction process on the plan list to obtain a target retrieval task.

[0081] Among them, the target retrieval task is the specific retrieval task that needs to be executed currently and is extracted from the plan list.

[0082] In the embodiments of the present application, after receiving the plan list, the terminal can sequentially extract retrieval tasks according to the arrangement order of the tasks in the list. For example, assuming the plan list is formulated for "how to improve the performance of a laptop computer" and includes tasks such as "retrieve the hardware components of a laptop computer", "retrieve the hardware factors affecting the performance of a laptop computer", and "retrieve methods to improve the hardware performance of a laptop computer". The terminal 102 first extracts "retrieve the hardware components of a laptop computer" as the target retrieval task.

[0083] S302, perform a retrieval process on the knowledge base based on the target retrieval task and the preset retrieval strategy to obtain retrieval process data.

[0084] Among them, the preset retrieval strategy is a pre-set retrieval rule and method, including which retrieval algorithm to select, which data sources to retrieve, how to process retrieval keywords, etc., and is the execution basis for performing knowledge base retrieval.

[0085] In the embodiments of the present application, for the target retrieval task, the terminal determines the retrieval algorithm, retrieval keywords, and data sources for retrieval according to the preset retrieval strategy. Use the selected retrieval algorithm to search in the knowledge base, and at the same time record information such as the retrieved document content, the matching degree between the document and the retrieval task, the time spent on retrieval, and the data source. These information together constitute the retrieval process data.

[0086] S303, perform a relevance evaluation and a completeness evaluation on the retrieval process data to obtain a retrieval evaluation result.

[0087] Among them, the relevance evaluation is to judge the degree of closeness of the retrieved data to the target retrieval task, and evaluate whether the data meets the retrieval requirements, usually measured by indicators such as keyword matching degree and semantic similarity. The completeness evaluation is to evaluate the comprehensiveness of the retrieved results covering the information related to the target retrieval task, and check whether key information is missed. The retrieval evaluation result is the result of synthesizing the relevance evaluation and the completeness evaluation, and is used to judge the quality of the current retrieval effect and determine whether the retrieval strategy needs to be adjusted.

[0088] In the embodiments of the present application, the terminal analyzes the retrieval process data. In the relevance evaluation, the degree of association between the retrieved data and the target retrieval task can be judged by methods such as keyword matching degree and semantic similarity; in terms of the completeness evaluation, the retrieved results can be checked whether they cover key information according to the scope of information required by the target retrieval task. Combining the two evaluation results, the retrieval evaluation result is obtained.

[0089] S304, perform a retrieval strategy adjustment process on the preset retrieval strategy according to the retrieval evaluation result, and re-retrieve according to the adjusted retrieval strategy to obtain a preliminary retrieval result.

[0090] In the embodiments of the present application, according to the retrieval evaluation results, if it is found that there are problems with the retrieval results, such as low relevance or insufficient integrity, the terminal can adjust the preset retrieval strategy. The adjustment content includes modifying the retrieval keywords, expanding or narrowing the retrieval scope, replacing the retrieval algorithm, etc. After completing the strategy adjustment, the terminal retrieves again in the knowledge base according to the new retrieval strategy and merges the newly retrieved information with the previous results to obtain the preliminary retrieval results.

[0091] Another implementation method: When adjusting the retrieval strategy, the terminal refers to the historical retrieval data and the successful retrieval strategies of similar tasks. According to the current retrieval evaluation results, the preset retrieval strategy is optimized by combining historical experience, and then retrieved again. The multiple retrieval results are deduplicated and integrated to finally obtain the preliminary retrieval results.

[0092] The steps of the above embodiments effectively improve the accuracy and comprehensiveness of the retrieval by orderly extracting the planned list tasks, retrieving according to the preset strategy, comprehensively evaluating the retrieval process data, and flexibly adjusting the retrieval strategy. During the retrieval process, the retrieval strategy is continuously optimized, enabling the terminal to dynamically adjust the retrieval method according to the actual retrieval situation, avoiding blind retrieval, and improving the retrieval efficiency.

[0093] In an exemplary embodiment, based on the above embodiment, please refer to Figure 4 , the retrieval evaluation results of the embodiments of the present application include the relevance evaluation results and the integrity evaluation results; the embodiments of the present application relate to the process of performing retrieval strategy adjustment processing on the preset retrieval strategy according to the retrieval evaluation results, and retrieving again according to the adjusted retrieval strategy to obtain the preliminary retrieval results, including the following steps:

[0094] S401, in the case where the relevance evaluation result does not meet the preset relevant conditions, perform keyword modification and query logic modification processing on the preset retrieval strategy to obtain a relevance adjustment strategy. In the case where the integrity evaluation result does not meet the preset integrity conditions, perform retrieval scope expansion processing and retrieval depth increase processing on the preset retrieval strategy to obtain an integrity adjustment strategy.

[0095] Among them, the preset relevant conditions are the criteria preset for judging whether the relevance of the retrieval results meets the standard, including specific quantitative indicators or judgment rules such as the keyword matching ratio and the semantic relevance threshold. The preset retrieval strategy is the rules and methods preset for guiding the retrieval operation, including the initial settings in aspects such as determining the retrieval keywords, retrieval logic, retrieval scope, and retrieval depth. The relevance adjustment strategy is the retrieval strategy obtained by modifying the keywords and query logic in the preset retrieval strategy when the relevance evaluation result does not meet the preset relevant conditions, aiming to improve the relevance of the retrieval results.

[0096] Among them, the preset complete condition is a criterion for measuring the completeness of retrieval results determined in advance, which clarifies the scope and degree of the information related to the target retrieval task that the retrieval results should contain. The completeness adjustment strategy is a retrieval strategy formed by adjusting the preset retrieval strategy by expanding the retrieval scope and increasing the retrieval depth when the completeness evaluation result does not meet the preset complete condition, so as to obtain more complete retrieval results.

[0097] In the embodiment of the present application, after the terminal obtains the relevance evaluation result, it compares it with the preset relevance condition. If the condition is not met, first analyze the difference between the retrieval result and the target retrieval task. Starting from the original retrieval keywords, use a thesaurus, semantic expansion tools, etc. to find more accurate or more representative keywords for replacement or supplementation. At the same time, review the original query logic, such as the use of Boolean logical operators, adjust the logical relationship according to the retrieval requirements, recombine the keywords, and form a new query statement, thereby obtaining the relevance adjustment strategy.

[0098] When the terminal determines that the completeness evaluation result does not meet the preset complete condition, it starts to adjust the retrieval scope and depth. In terms of the retrieval scope, it expands from the originally limited specific data sources, such as several knowledge bases or document collections, to more extensive data sources, including professional databases and academic literature databases in related fields. For the retrieval depth, originally only retrieve shallow information such as the title and abstract of the document, now it delves into deeper levels such as the document body and references to obtain more relevant information, and after the adjustment, the completeness adjustment strategy is obtained.

[0099] S402, perform a retrieval strategy adjustment process on the preset retrieval strategy according to the relevance adjustment strategy and / or the completeness adjustment strategy to obtain the adjusted retrieval strategy.

[0100] In the embodiment of the present application, the terminal determines whether there are both relevance and completeness problems at the same time. If there is only a relevance problem, directly use the relevance adjustment strategy as the adjusted retrieval strategy; if there is only a completeness problem, use the completeness adjustment strategy as the adjusted retrieval strategy; if both problems exist, modify the keywords and query logic modification content in the relevance adjustment strategy and the retrieval scope expansion and retrieval depth increase content in the completeness adjustment strategy are integrated and applied to the preset retrieval strategy, thereby obtaining the adjusted retrieval strategy.

[0101] S403, perform a retrieval process again according to the adjusted retrieval strategy until the relevance evaluation result meets the preset relevance condition and the completeness evaluation result meets the preset complete condition, and obtain the preliminary retrieval result.

[0102] In an embodiment of the present application, the terminal re-searches the knowledge base or other data sources based on the adjusted search strategy. After the search is completed, the relevance and completeness evaluation of the new search results is performed again. If the evaluation result still does not meet the preset conditions, the above steps S401-S403 are repeated, and the search strategy is continued to be adjusted and the search is re-searched, and this cycle is repeated until the relevance evaluation result meets the preset relevance conditions and the completeness evaluation result meets the preset complete conditions. The search result at this time is the preliminary search result.

[0103] Through the steps of this embodiment, the terminal can make targeted adjustments to the search strategy based on the search evaluation results, continuously optimizing the search process. Strategy adjustments are made in terms of relevance and completeness, making the search results more tailored to user needs. This not only improves the relevance of the search results to the target task but also ensures the integrity of the information. Multiple iterations of the search are performed until the preset conditions are met, ensuring the high quality of the initial search results.

[0104] In an exemplary embodiment, based on the above embodiment, see Figure 5 , the method of the embodiment of the present application further includes the following steps:

[0105] S501: When the retrieval process data does not meet the result generation requirements, the plan list is updated to obtain an updated plan list.

[0106] Result generation requirements are a set of pre-defined criteria used to measure whether the data from the search process meets the requirements for generating the final results. These criteria include specific indicators and conditions for data accuracy, completeness, and relevance. The updated plan list is a new plan list that is modified, supplemented, or adjusted from the original plan list. It aims to optimize the scheduling of search tasks and obtain search results that better meet the requirements.

[0107] In an embodiment of the present application, the terminal continuously monitors the retrieval process data and compares it with the result generation requirements. Once it is found that the retrieval process data does not meet the requirements, the terminal first analyzes the reasons why the data does not meet the requirements. If it is found that the retrieved information is not relevant enough, more targeted retrieval tasks may be added to the plan list to refine the original retrieval direction; if the data integrity is missing, new retrieval tasks will be added to the plan list to obtain the missing key information. At the same time, the order and priority of the tasks in the original plan list are re-evaluated and adjusted to ensure that the new task arrangement is more reasonable, and finally an updated plan list is generated.

[0108] S502, returning to the step of performing retrieval task extraction processing on the plan list to obtain the target retrieval task.

[0109] In the embodiments of the present application, when the terminal obtains the updated plan list, it immediately returns to the link of extracting tasks from the plan list. According to the order of the updated plan list, the retrieval tasks therein are sequentially extracted as target retrieval tasks. During the extraction process, the terminal will check the integrity and rationality of the tasks again to ensure the smooth progress of subsequent retrieval operations.

[0110] The steps of the above embodiments make the retrieval process of the terminal more flexible and adaptable through the monitoring of retrieval process data and the dynamic update of the plan list. When the retrieval process data does not meet the result generation requirements, adjusting the plan list in a timely manner can effectively make up for the deficiencies in the retrieval process, ensure that subsequent retrieval tasks are more targeted, and thus improve the quality of retrieval results.

[0111] In an exemplary embodiment, based on the above embodiment, please refer to Figure 6 , the embodiments of the present application are related to the process of performing uncertainty evaluation processing on the preliminary retrieval results, determining the target retrieval results according to the uncertainty evaluation results and the preliminary retrieval results, and the process of obtaining the target retrieval results includes the following steps:

[0112] S601, perform uncertainty quantification processing on the preliminary retrieval results to obtain a quantification result.

[0113] Among them, the quantification result is data obtained through uncertainty quantification processing, presented in numerical or other quantified forms, used to clarify whether the preliminary retrieval results belong to uncertain data, and provide a basis for subsequent decisions. Uncertain data are those data in the preliminary retrieval results that have doubts or do not meet the preset standards in terms of accuracy, integrity, and matching degree with user needs. Such data may not be directly output as reliable information.

[0114] In the embodiments of the present application, after the terminal obtains the preliminary retrieval results, it calls a pre-set uncertainty quantification model. This model analyzes the preliminary retrieval results from multiple dimensions, such as evaluating the credibility of each information source in the retrieval results, checking whether the information completely covers the key points related to the retrieval task, and judging the semantic relevance between the information and the user's retrieval needs. Through comprehensive consideration of these dimensions, the model assigns an uncertainty quantification value to each retrieval result item. Summarize and organize the quantification values of all retrieval result items to form an overall quantification result, so as to characterize whether the preliminary retrieval results are uncertain data.

[0115] Another implementation: The terminal evaluates the preliminary search results using multiple quantization metrics. First, calculate the coverage rate of keywords in the search results. The lower the coverage rate, the higher the possible uncertainty. Second, analyze the similarity between the search results and known authoritative information. A low similarity increases the uncertainty. Third, evaluate the logical consistency of the search results. Results with logical confusion have high uncertainty. The terminal sums up the calculation results of these metrics after weighting to obtain a comprehensive quantization value as the quantization result, and uses this to judge the uncertainty level of the preliminary search results.

[0116] S602, in the case where the quantization result represents uncertain data, optimize the preliminary search results to obtain the target search results.

[0117] In the embodiments of the present application, when the terminal determines that the preliminary search results are uncertain data based on the quantization result, an optimization process is started. First, perform data cleaning on the search results to remove duplicate, incorrect, or obviously unreliable information. Then, according to the requirements of the search task and relevant information in the knowledge base, supplement the missing key content. For the remaining information, reorder it according to its relevance and importance to the user's needs, and place the information with high relevance and strong importance in the front, finally forming the target search results.

[0118] Another implementation: The terminal can also optimize the uncertain preliminary search results with the help of machine learning algorithms. The algorithm automatically identifies problems in the preliminary search results based on a large amount of historical search data and optimization experience, and generates corresponding optimization schemes. For example, the algorithm may identify that some information needs semantic correction, or some information needs to be integrated with other relevant information. The terminal processes the preliminary search results according to the optimization scheme generated by the algorithm, and after operations such as correction and integration, obtains the target search results that meet the requirements.

[0119] The steps of the above embodiments significantly improve the quality of the search results through the uncertainty quantization and targeted optimization of the preliminary search results. The uncertainty quantization process helps the terminal accurately identify problems in the preliminary search results and provides a clear direction for subsequent optimization. The optimization process for uncertain data effectively removes unreliable information, supplements missing content, makes the finally obtained target search results more accurate and complete, and has a higher matching degree with the user's needs.

[0120] In an exemplary embodiment, based on the above embodiment, please refer to Figure 7 , the process of the embodiments of the present application for performing uncertainty quantization processing on the preliminary search results to obtain the quantization result includes the following steps:

[0121] S701, calculate the information quantity index score and the generated content confidence score of the preliminary search results.

[0122] In the embodiments of the present application, after the terminal receives the preliminary search results, for the information quantity index score, it first analyzes the quantity and types of key information related to the search task covered by the search results. Using text analysis techniques, keywords and key sentences in the search results are extracted, and compared with the preset key information database for the search task. A quantitative score is given according to the matching degree and the importance of the information, forming the information quantity index score. For the generated content confidence score, the terminal will evaluate the credibility of the source of the search results, such as checking whether the information source is authoritative and whether the data release time is timely, etc. At the same time, a machine learning model is used to evaluate the content quality of the search results, and these factors are comprehensively considered to assign a generated content confidence score to the search results.

[0123] S702, Obtain a quantitative result according to the information quantity index score and the generated content confidence score.

[0124] In the embodiments of the present application, after the terminal obtains the information quantity index score and the generated content confidence score, according to the preset weight assignment scheme, these two scores are weighted and summed. For example, the weight of the information quantity index score is set to 0.6, and the weight of the generated content confidence score is set to 0.4. The two scores are multiplied by their respective weights and then added together to obtain a comprehensive score as the quantitative result. This quantitative result can reflect the overall performance of the preliminary search results in terms of information quantity and credibility, and further reflect its uncertainty degree.

[0125] The steps of the above embodiments provide a quantitative and scientific method for the uncertainty evaluation of the preliminary search results by calculating the information quantity index score and the generated content confidence score and integrating the two. The information quantity index score ensures the consideration of the richness of the content of the search results, and the generated content confidence score ensures the evaluation of the reliability of the search results. Combining the two to obtain the quantitative result can comprehensively and accurately measure the uncertainty degree of the preliminary search results.

[0126] In an exemplary embodiment, based on the above embodiment, please refer to Figure 8 , The embodiments of the present application relate to the process of optimizing the preliminary search results to obtain the target search results, including the following steps:

[0127] S801, Generate prompt information according to the quantitative result, and obtain the feedback information of the user according to the prompt information.

[0128] Among them, the prompt information is generated according to the quantitative result, and is used to display to the user the problems existing in the preliminary search results or information related to the uncertainty, guiding the user to make feedback. These information can include possible incompleteness and inaccuracy of the search results, or content suggesting that the user further clarify the requirements, etc.

[0129] In the embodiments of the present application, after the terminal obtains the quantization result, it analyzes the quantization result. If the quantization result shows that the uncertainty of the preliminary retrieval result is relatively high, such as insufficient information or low confidence, the terminal will generate corresponding prompt information. The terminal will display the prompt information to the user through the interaction interface, for example, presenting it in the form of a pop-up window or a prompt box on the retrieval result page. After that, the terminal waits for the user to view the prompt information and collects the user's feedback. After seeing the prompt information, the user inputs feedback information such as their own ideas, requirements adjustment directions, etc. through the interaction interface (such as input boxes, selection buttons, etc.), and the terminal receives and records these feedback information in real time.

[0130] S802, adjust the retrieval query strategy according to the feedback information, and optimize the preliminary retrieval result based on the adjusted retrieval query strategy to obtain the target retrieval result.

[0131] Among them, the retrieval query strategy is the method and rule adopted by the terminal when performing information retrieval, including strategies in aspects such as determining retrieval keywords, selecting retrieval data sources, and setting retrieval logical relationships, which directly affect the quality of the retrieval result.

[0132] In the embodiments of the present application, after the terminal receives the user's feedback information, it analyzes the feedback content. If the user feedbacks that the relevance of the retrieval result is insufficient, the terminal will extract key information from the feedback, re-determine or adjust the retrieval keywords, and optimize the retrieval logical relationship. For the retrieval data source, if the user feedbacks that more authoritative or more professional data is needed, the terminal will adjust the data source selection strategy and add or replace the data source. After completing the adjustment of the retrieval query strategy, the terminal performs optimization operations such as screening, supplementing, or re-ordering the preliminary retrieval result according to the new strategy. For example, use the new keywords to perform secondary screening on the preliminary retrieval result, remove the content with lower relevance, supplement the newly retrieved relevant information, and re-order the results according to the new relevance standard to finally obtain the target retrieval result.

[0133] The steps of the above embodiments realize the optimization of the preliminary retrieval result through the interaction between the quantization result and the user, improve the performance of the retrieval system and the user experience. The quantization result provides a basis for generating prompt information, enabling users to participate in the retrieval optimization process and enhancing the interactivity between users and the system. Adjusting the retrieval query strategy according to the user feedback and optimizing the preliminary retrieval result can more accurately meet the personalized needs of users and improve the quality and relevance of the retrieval result.

[0134] In an exemplary embodiment, based on the above embodiment, please refer to Figure 9 , the embodiments of the present application relate to the process of formulating a plan based on a pre-obtained user question to obtain a plan list, including the following steps:

[0135] S901. Perform knowledge extraction processing on the user's question to obtain associated knowledge.

[0136] Among them, the associated knowledge refers to the set of relevant knowledge obtained from the user's question through knowledge extraction processing. These knowledge are closely related to the user's question and can assist in the understanding and processing of the question.

[0137] In the embodiments of this application, after the terminal receives the user's question, it can use lexical analysis, syntactic analysis, and semantic analysis technologies in natural language processing to disassemble and understand the user's question. Identify the keywords in the question through lexical analysis, then use syntactic analysis to determine the grammatical relationship between the keywords, and finally combine semantic analysis to dig out the concepts and domain knowledge behind the question. Integrate these extracted keywords, concepts, domain knowledge, etc. to form associated knowledge.

[0138] S902. Perform intention recognition and information enhancement processing based on the user's question and the pre-stored historical conversations to obtain key information.

[0139] Among them, the key information refers to the information that has been refined from the user's question after intention recognition and information enhancement processing and has key guiding significance for formulating a plan list.

[0140] In the embodiments of this application, the terminal first matches and compares the user's question with the pre-stored historical conversations. Calculate the similarity between the questions to find similar historical conversation cases. Then, analyze the interaction process of these historical conversations and the user's final needs, and combine the current user's question to use the intention recognition algorithm to determine the user's true intention. In terms of information enhancement, according to the intention recognition result, extract relevant supplementary information from the knowledge base, expand and refine the user's question, and refine the key information.

[0141] S903. Perform plan formulation processing based on the associated knowledge and the key information to obtain a plan list.

[0142] In the embodiments of this application, the terminal determines the general direction and scope of the retrieval according to the associated knowledge, and then clarifies the specific retrieval tasks and the sequence according to the key information. For example, according to the importance and logical relationship of the information, divide the retrieval tasks into multiple steps, first retrieve the basic information, and then gradually retrieve the detailed information. Arrange these retrieval tasks in order to form a plan list containing multiple ordered tasks, and each task has a clear goal and operation method.

[0143] Through in-depth analysis and processing of user questions, the steps of the above embodiments generate a targeted and efficient list of plans. The knowledge extraction process can accurately mine the key knowledge in user questions and provide a basis for subsequent operations. The combination of intent recognition and information enhancement with historical conversations enables the system to more accurately grasp user needs and enrich question information. The list of plans formulated based on associated knowledge and key information can guide the system to retrieve and process information in an orderly manner, improve the accuracy and comprehensiveness of information acquisition, effectively enhance the performance of the retrieval system and the user experience, and meet the diverse information needs of users.

[0144] In an exemplary embodiment, based on the above embodiment, the method of the embodiment of the present application further includes: generating an explanatory process for the retrieval process based on the retrieval process data and the target retrieval result to obtain explanatory content.

[0145] Among them, the explanatory content can be text, charts or other forms of information obtained through the explanatory generation process, which details the basis for each operation in the retrieval process, the specific source of the data, and the internal connection and matching situation between the target retrieval result and the user's initial needs, enhancing the transparency and interpretability of the retrieval system.

[0146] In the embodiment of the present application, the terminal first collects and organizes various data generated during the retrieval process, including detailed execution records of retrieval tasks, keywords used in each retrieval, and data sources involved. At the same time, clarify the specific content and key information of the target retrieval result. Then, based on the pre-set explanatory template and logical rules, perform an association analysis on the retrieval process data and the target retrieval result. In the order of the retrieval steps, successively elaborate on the purpose and basis of each retrieval step, for example, determining the retrieval direction based on the keywords extracted from the user's needs. For the data source, detail which data source each data item comes from. Finally, analyze how the target retrieval result meets the user's needs, and by comparing the key information in the user's needs and the content of the target retrieval result, elaborate on the association relationship between the two to generate explanatory content.

[0147] Through the above embodiment to perform explanatory generation processing on the retrieval process, it provides users with a detailed description of the retrieval process and results, significantly enhancing the interpretability of the retrieval system. This enables users to clearly understand how the system performs retrieval operations according to their own needs and how the final target retrieval result is generated, enhancing users' trust in the retrieval system. At the same time, the explanatory content also helps users discover possible problems in the retrieval process, further optimize retrieval requirements and strategies, and improve retrieval efficiency and quality.

[0148] In an exemplary embodiment, based on the above embodiment, the method of the embodiment of the present application further includes the following steps:

[0149] Step 1: Perform knowledge extraction processing on the user's question to obtain associated knowledge; perform intent recognition and information enhancement processing based on the user's question and the pre-stored historical conversations to obtain key information; perform plan formulation processing based on the associated knowledge and key information to obtain a plan list;

[0150] Step 2: Perform retrieval task extraction processing on the plan list to obtain target retrieval tasks; perform retrieval processing on the knowledge base based on the target retrieval tasks and the preset retrieval strategies to obtain retrieval process data; perform evaluation processing on the retrieval process data through relevance evaluation and integrity evaluation to obtain retrieval evaluation results;

[0151] Step 3: In the case where the relevance evaluation result does not meet the preset relevant conditions, perform keyword modification and query logic modification processing on the preset retrieval strategies to obtain a relevance adjustment strategy; in the case where the integrity evaluation result does not meet the preset integrity conditions, perform retrieval scope expansion processing and retrieval depth increase processing on the preset retrieval strategies to obtain an integrity adjustment strategy; perform retrieval strategy adjustment processing on the preset retrieval strategies according to the relevance adjustment strategy and / or the integrity adjustment strategy to obtain an adjusted retrieval strategy; perform retrieval processing again according to the adjusted retrieval strategy until the relevance evaluation result meets the preset relevant conditions and the integrity evaluation result meets the preset integrity conditions to obtain a preliminary retrieval result;

[0152] Step 4: Calculate the information quantity index score and the generated content confidence score of the preliminary retrieval result; obtain a quantization result according to the information quantity index score and the generated content confidence score;

[0153] Step 5: In the case where the quantization result represents uncertain data, generate a prompt message according to the quantization result, and obtain the user's feedback information according to the prompt message; perform adjustment processing on the retrieval query strategy according to the feedback information, and perform optimization processing on the preliminary retrieval result based on the adjusted retrieval query strategy to obtain a target retrieval result;

[0154] Step 6: In the case where the retrieval process data does not meet the result generation requirements, perform update processing on the plan list to obtain an updated plan list; return to execute the step of performing retrieval task extraction processing on the plan list to obtain target retrieval tasks as described above;

[0155] Step 7: Perform explanation generation processing on the retrieval processing process according to the retrieval process data and the target retrieval result to obtain explanation content.

[0156] It should be understood that although each step in the flowcharts involved in the above-described embodiments is displayed sequentially as indicated by the arrows, these steps do not necessarily have to be executed sequentially in the order indicated by the arrows. Unless there is a clear indication in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-described embodiments may include multiple steps or multiple stages. These steps or stages do not necessarily have to be completed at the same time, but can be executed at different times. The execution order of these steps or stages does not necessarily have to be sequential, but can be executed alternately or in turn with at least a part of other steps or steps or stages in other steps.

[0157] Based on the same inventive concept, an embodiment of the present application also provides a retrieval optimization device for implementing the retrieval optimization method described above. The solution provided by this device to solve the problem is similar to the solution described in the above method. Therefore, the specific limitations in one or more embodiments of the retrieval optimization device provided below can refer to the limitations on the retrieval optimization method in the above text, and will not be repeated here.

[0158] In one embodiment, as Figure 10 shown, a retrieval optimization device 1000 is provided, including: [[ID=!0]]

[0159] A list formulation module 1001, configured to perform plan formulation processing based on a pre-acquired user problem to obtain a plan list; the plan list includes multiple retrieval tasks, and there is a progressive relationship between the retrieval tasks;

[0160] A preliminary retrieval module 1002, configured to perform retrieval processing according to the plan list and a pre-established knowledge base, and perform iterative processing according to the retrieval process data during the retrieval process to obtain a preliminary retrieval result;

[0161] A result evaluation module 1003, configured to perform uncertainty evaluation processing on the preliminary retrieval result, and determine a target retrieval result according to the uncertainty evaluation result and the preliminary retrieval result.

[0162] [[ID=!0]]In one of the embodiments, the above-mentioned preliminary retrieval module is specifically configured to perform retrieval task extraction processing on the plan list to obtain a target retrieval task; perform retrieval processing on the knowledge base based on the target retrieval task and a preset retrieval strategy to obtain retrieval process data; perform relevance evaluation and integrity evaluation on the retrieval process data to obtain a retrieval evaluation result; perform retrieval strategy adjustment processing on the preset retrieval strategy according to the retrieval evaluation result, and re-retrieve according to the adjusted retrieval strategy to obtain a preliminary retrieval result.

[0163] In one embodiment, the above retrieval evaluation result includes a relevance evaluation result and a completeness evaluation result; the above preliminary retrieval module is further specifically configured to perform retrieval strategy adjustment processing on a preset retrieval strategy according to the retrieval evaluation result, and re-perform retrieval according to the adjusted retrieval strategy to obtain a preliminary retrieval result, including: in the case where the relevance evaluation result does not meet the preset relevance condition, performing keyword modification and query logic modification processing on the preset retrieval strategy to obtain a relevance adjustment strategy; in the case where the completeness evaluation result does not meet the preset completeness condition, performing retrieval scope expansion processing and retrieval depth increase processing on the preset retrieval strategy to obtain a completeness adjustment strategy; performing retrieval strategy adjustment processing on the preset retrieval strategy according to the relevance adjustment strategy and / or the completeness adjustment strategy to obtain an adjusted retrieval strategy; re-performing retrieval processing according to the adjusted retrieval strategy until the relevance evaluation result meets the preset relevance condition and the completeness evaluation result meets the preset completeness condition to obtain a preliminary retrieval result.

[0164] In one embodiment, the above device further includes:

[0165] A list update module, configured to update a planned list to obtain an updated planned list in the case where the retrieval process data does not meet the result generation requirement;

[0166] A return execution module, configured to return to execute the step of performing retrieval task extraction processing on the planned list to obtain a target retrieval task.

[0167] In one embodiment, the above result evaluation module is specifically configured to perform uncertainty quantification processing on the preliminary retrieval result to obtain a quantification result; the quantification result is used to characterize whether the preliminary retrieval result is uncertain data; in the case where the quantification result characterizes uncertain data, perform optimization processing on the preliminary retrieval result to obtain a target retrieval result.

[0168] In one embodiment, the above result evaluation module is further specifically configured to calculate an information quantity index score and a generated content confidence score of the preliminary retrieval result; obtain a quantification result according to the information quantity index score and the generated content confidence score.

[0169] In one embodiment, the above result evaluation module is further specifically configured to generate a prompt message according to the quantification result, and obtain user feedback information according to the prompt message; perform adjustment processing on the retrieval query strategy according to the feedback information, and perform optimization processing on the preliminary retrieval result based on the adjusted retrieval query strategy to obtain a target retrieval result.

[0170] In one embodiment, the above-mentioned list formulation module is specifically configured to perform knowledge extraction processing on the user's question to obtain associated knowledge; perform intent recognition and information enhancement processing based on the user's question and the pre-stored historical conversations to obtain key information; and perform plan formulation processing based on the associated knowledge and the key information to obtain a plan list.

[0171] Each module in the above-mentioned retrieval optimization device can be implemented in whole or in part by software, hardware, and their combination. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or stored in a memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above-mentioned modules.

[0172] In one embodiment, a computer device is provided. The computer device can be a terminal, and its internal structure diagram can be as Figure 11 shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit, and an input device. Among them, the processor, the memory, and the input / output interface are connected through a system bus, and the communication interface, the display unit, and the input device are connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used for exchanging information between the processor and external devices. The communication interface of the computer device is used for communicating with external terminals in a wired or wireless manner. The wireless manner can be implemented through WIFI, a mobile cellular network, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a retrieval optimization method. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer covering the display screen, or a button, a trackball, or a touchpad provided on the housing of the computer device, or an external keyboard, a touchpad, or a mouse, etc.

[0173] Those skilled in the art can understand that Figure 11 the structure shown in

[0174] In one embodiment, a computer device is provided, including a memory and a processor. A computer program is stored in the memory. When the processor executes the computer program, the following steps are implemented:

[0175] Perform planning processing based on a pre-acquired user question to obtain a plan list; the plan list includes multiple retrieval tasks, and there is a progressive relationship between the retrieval tasks;

[0176] Perform retrieval processing according to the plan list and a pre-established knowledge base, and perform iterative processing according to the retrieval process data during the retrieval process to obtain a preliminary retrieval result;

[0177] Perform uncertainty evaluation processing on the preliminary retrieval result, and determine the target retrieval result according to the uncertainty evaluation result and the preliminary retrieval result.

[0178] In one embodiment, when the processor executes the computer program, the following steps are further implemented:

[0179] Perform retrieval task extraction processing on the plan list to obtain a target retrieval task;

[0180] Perform retrieval processing on the knowledge base based on the target retrieval task and a preset retrieval strategy to obtain retrieval process data;

[0181] Perform relevance evaluation and integrity evaluation on the retrieval process data to obtain a retrieval evaluation result;

[0182] Perform retrieval strategy adjustment processing on the preset retrieval strategy according to the retrieval evaluation result, and re-perform retrieval according to the adjusted retrieval strategy to obtain a preliminary retrieval result.

[0183] In one embodiment, when the processor executes the computer program, the following steps are further implemented:

[0184] In the case where the relevance evaluation result does not meet the preset relevance condition, perform keyword modification and query logic modification processing on the preset retrieval strategy to obtain a relevance adjustment strategy;

[0185] In the case where the integrity evaluation result does not meet the preset integrity condition, perform retrieval scope expansion processing and retrieval depth increase processing on the preset retrieval strategy to obtain an integrity adjustment strategy;

[0186] Perform retrieval strategy adjustment processing on the preset retrieval strategy according to the relevance adjustment strategy and / or the integrity adjustment strategy to obtain an adjusted retrieval strategy;

[0187] Re-perform retrieval processing according to the adjusted retrieval strategy until the relevance evaluation result meets the preset relevance condition and the integrity evaluation result meets the preset integrity condition to obtain a preliminary retrieval result.

[0188] In one embodiment, when the processor executes the computer program, the following steps are further implemented:

[0189] In the case where the retrieved process data does not meet the result generation requirements, update the plan list to obtain an updated plan list;

[0190] Return to execute the step of retrieving task extraction processing on the plan list to obtain the target retrieval task.

[0191] In one embodiment, when the processor executes the computer program, the following steps are further implemented:

[0192] Perform uncertainty quantification processing on the preliminary retrieval result to obtain a quantification result; the quantification result is used to characterize whether the preliminary retrieval result is uncertain data;

[0193] In the case where the quantification result characterizes uncertain data, perform optimization processing on the preliminary retrieval result to obtain the target retrieval result.

[0194] In one embodiment, when the processor executes the computer program, the following steps are further implemented:

[0195] Calculate the information quantity index score and the generated content confidence score of the preliminary retrieval result;

[0196] Obtain the quantification result according to the information quantity index score and the generated content confidence score.

[0197] In one embodiment, when the processor executes the computer program, the following steps are further implemented:

[0198] Generate a prompt message according to the quantification result, and obtain the feedback message of the user according to the prompt message;

[0199] Adjust the retrieval query strategy according to the feedback message, and perform optimization processing on the preliminary retrieval result based on the adjusted retrieval query strategy to obtain the target retrieval result.

[0200] In one embodiment, when the processor executes the computer program, the following steps are further implemented:

[0201] Perform knowledge extraction processing on the user question to obtain associated knowledge;

[0202] Perform intention recognition and information enhancement processing according to the user question and the pre-stored historical conversation to obtain key information;

[0203] Perform plan formulation processing according to the associated knowledge and the key information to obtain a plan list.

[0204] According to some embodiments of the present application, there is also provided a computer program product. When the computer program is executed by a processor, the above method can be implemented. The computer program product includes one or more computer instructions. When these computer instructions are loaded and executed on a computer, part or all of the above method can be implemented in accordance with the process or function described in the embodiments of the present application.

[0205] According to some embodiments of the present application, there is also provided a non-transitory computer-readable storage medium including instructions, such as a memory including instructions. The above instructions can be executed by a processor of an electronic device to complete the above method. For example, the non-transitory computer-readable storage medium may be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device, etc.

[0206] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data need to comply with the relevant laws, regulations, and standards of relevant countries and regions.

[0207] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memories. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the embodiments provided in the present application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the embodiments provided in the present application can be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, data processing logics based on quantum computing, etc., without limitation.

[0208] The technical features of the above embodiments can be combined arbitrarily. For the sake of concise description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

[0209] The above-described embodiments only represent several implementation manners of the present application. The description is relatively specific and detailed, but it should not be construed as a limitation on the patent scope of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A retrieval optimization method, characterized in that, The method includes: Performing planning processing based on a pre-obtained user question to obtain a plan list; the plan list includes multiple retrieval tasks, and there is a progressive relationship between the retrieval tasks; Performing retrieval processing according to the plan list and a pre-established knowledge base, and performing iterative processing according to retrieval process data during the retrieval process to obtain a preliminary retrieval result; Performing uncertainty evaluation processing on the preliminary retrieval result, and determining a target retrieval result according to the uncertainty evaluation result and the preliminary retrieval result.

2. The method according to claim 1, characterized in that, The performing retrieval processing according to the plan list and a pre-established knowledge base, and performing iterative processing according to retrieval process data during the retrieval process to obtain a preliminary retrieval result includes: Performing retrieval task extraction processing on the plan list to obtain a target retrieval task; Performing retrieval processing on the knowledge base based on the target retrieval task and a preset retrieval strategy to obtain the retrieval process data; Performing relevance evaluation and integrity evaluation on the retrieval process data to obtain a retrieval evaluation result; Performing retrieval strategy adjustment processing on the preset retrieval strategy according to the retrieval evaluation result, and re-performing retrieval according to the adjusted retrieval strategy to obtain the preliminary retrieval result.

3. The method according to claim 2, wherein The retrieval evaluation result includes a relevance evaluation result and an integrity evaluation result; the performing retrieval strategy adjustment processing on the preset retrieval strategy according to the retrieval evaluation result, and re-performing retrieval according to the adjusted retrieval strategy to obtain the preliminary retrieval result includes: In the case where the relevance evaluation result does not meet the preset relevance condition, performing keyword modification and query logic modification processing on the preset retrieval strategy to obtain a relevance adjustment strategy; In the case where the integrity evaluation result does not meet the preset integrity condition, performing retrieval scope expansion processing and retrieval depth increase processing on the preset retrieval strategy to obtain an integrity adjustment strategy; Performing retrieval strategy adjustment processing on the preset retrieval strategy according to the relevance adjustment strategy and / or the integrity adjustment strategy to obtain the adjusted retrieval strategy; Re-performing retrieval processing according to the adjusted retrieval strategy until the relevance evaluation result meets the preset relevance condition and the integrity evaluation result meets the preset integrity condition to obtain the preliminary retrieval result.

4. The method according to claim 2, wherein The method further includes: In the case where the retrieval process data does not meet the result generation requirement, performing update processing on the plan list to obtain an updated plan list; Returning to execute the step of performing retrieval task extraction processing on the plan list to obtain a target retrieval task.

5. The method according to claim 1, wherein The performing uncertainty evaluation processing on the preliminary retrieval result, and determining a target retrieval result according to the uncertainty evaluation result and the preliminary retrieval result to obtain a target retrieval result includes: Performing uncertainty quantification processing on the preliminary retrieval result to obtain a quantification result; the quantification result is used to represent whether the preliminary retrieval result is uncertain data; In the case where the quantification result represents uncertain data, performing optimization processing on the preliminary retrieval result to obtain the target retrieval result.

6. The method according to claim 5, wherein Performing uncertainty quantification processing on the preliminary search results to obtain a quantification result, including: Calculating the information quantity index score and the generated content confidence score of the preliminary search results; Obtaining the quantification result according to the information quantity index score and the generated content confidence score.

7. The method according to claim 5, wherein Performing optimization processing on the preliminary search results to obtain the target search results, including: Generating a prompt message according to the quantification result, and obtaining the feedback information of the user according to the prompt message; Adjusting the search query strategy according to the feedback information, and performing optimization processing on the preliminary search results based on the adjusted search query strategy to obtain the target search results.

8. The method according to claim 1, characterized in that Performing plan formulation processing based on the pre-obtained user question to obtain a plan list, including: Performing knowledge extraction processing on the user question to obtain associated knowledge; Performing intention recognition and information enhancement processing according to the user question and the pre-stored historical conversation to obtain key information; Performing plan formulation processing according to the associated knowledge and the key information to obtain the plan list.

9. The method according to claim 1, wherein The method further includes: Performing explanation generation processing on the search processing process according to the search process data and the target search results to obtain explanation content; the explanation content is used to represent the basis of each step in the search process, the data source, and the association relationship between the target search results and the user requirements.

10. A retrieval optimization device, characterized in that, The device includes: A list formulation module, configured to perform plan formulation processing based on the pre-obtained user question to obtain a plan list; the plan list includes multiple search tasks, and the search tasks are in a progressive relationship; A preliminary search module, configured to perform search processing according to the plan list and the pre-established knowledge base, and perform iterative processing according to the search process data during the search process to obtain preliminary search results; A result evaluation module, configured to perform uncertainty evaluation processing on the preliminary search results, and determine the target search results according to the uncertainty evaluation results and the preliminary search results.

Images