Retrieval enhancement generation method and device

Abstract

The present invention relates to the field of natural language processing technology, and provides a retrieval enhancement generation method and device. The method provided by the present invention compares the overall score of a document set to be scored with a score threshold. If the comparison result shows that the overall score of the document set is less than the score threshold, a retrieval strategy is generated based on the overall score of the document set and a reinforcement learning network. Based on the retrieval strategy, the next round of updated recall document sets is retrieved. The overall score of the recalled document set and the score threshold are repeatedly compared until the overall score of the document set meets the requirements. An updated recall document set is generated through the final round of updated recall document sets, achieving highly relevant and high-quality document recall. Then, based on the recalled document set with good recall effect, a reply text with high accuracy and reliability is generated.

Description

Technical Field

[0001] The present invention relates to the field of natural language processing technology, and in particular to a retrieval enhancement generation method and device. Background Art

[0002] With the widespread application of large language models in fields such as question-answering and document summarization, retrieval-augmented generation (RAG) based on external knowledge retrieval has become a mainstream framework. RAG matches user queries with external document repositories based on similarity and uses the retrieved documents as contextual input to the model, thereby improving the accuracy and professionalism of responses. Existing RAG systems generally use a static "one-retrieval, one-generation" process. The retrieval stage typically retrieves the retrieved documents based on preset keyword templates or fixed vector search strategies.

[0003] However, the quality of the recalled documents obtained based on static retrieval is poor, and there may be a large amount of redundant and irrelevant information, which in turn leads to a decrease in the accuracy of the response content generated based on the recalled documents. Summary of the Invention

[0004] The present invention provides a retrieval enhancement generation method and device to solve the defect in the prior art that the relevance of recalled documents obtained by static retrieval is low, resulting in a decrease in the accuracy of reply content generated based on the recalled documents.

[0005] The present invention provides a search enhancement generation method, comprising:

[0006] Retrieve the document set to be scored for the current query text;

[0007] Comparing the overall score of the document set to be scored and the score threshold, and if the comparison result shows that the overall score of the document set is less than the score threshold, generating a retrieval strategy based on the overall score of the document set and the reinforcement learning network;

[0008] Based on the retrieval strategy, the next round of updated recalled document set is retrieved;

[0009] The updated recalled document set is used as the recalled document set to be scored, and the overall score of the recalled document set to be scored is repeatedly compared with the scoring threshold until the comparison result is that the overall score of the document set is not less than the scoring threshold. Based on the updated recalled document set of the last round, a query reply text of the current query text is generated.

[0010] According to a retrieval enhancement generation method provided by the present invention, the reinforcement learning network includes a high-order network and a low-order network;

[0011] The generating of the retrieval strategy based on the overall score of the document set and the reinforcement learning network includes:

[0012] generating a high-order search strategy based on the overall score of the document set and the high-order network; the high-order search strategy includes at least one of switching a target search engine, rewriting a query text, and adjusting the number of searches;

[0013] Based on the overall score of the document set and the low-order network, the low-order retrieval parameters in the high-order retrieval strategy are adjusted to obtain modified low-order retrieval parameters; the low-order retrieval parameters include at least one of a retrieval similarity threshold, an expanded word weight, and a vector temperature coefficient.

[0014] According to a retrieval enhancement generation method provided by the present invention, the step of obtaining the overall score of the document set of the recalled document set to be scored includes:

[0015] Based on the current query text and each recalled document in the recalled document set to be scored, calculating a document quality index of each recalled document; the document quality index includes at least one of semantic similarity, keyword coverage, publication time, and retrieval source credibility;

[0016] Calculating a document quality score for each of the recalled documents based on the document quality index of each of the recalled documents;

[0017] Calculating the mutual information of each recalled document based on the semantic similarity of each recalled document and the maximum historical semantic similarity;

[0018] Based on the document quality score and mutual information of each recalled document, an overall score of the document set to be scored is calculated.

[0019] According to a retrieval enhancement generation method provided by the present invention, the overall score of the document set of recalled documents to be scored is calculated based on the document quality score and mutual information of each recalled document, including:

[0020] Based on the mutual information of the recalled documents, a preferred recalled document set is obtained from the recalled document set to be scored;

[0021] Based on the document quality score and mutual information of each preferred recalled document in the preferred recalled document set, the overall score of the document set is calculated.

[0022] According to a retrieval enhancement generation method provided by the present invention, the retrieval obtains a set of recalled documents to be scored for the current query text, including:

[0023] Encode the current query text to obtain a sentence vector;

[0024] If there are historical query text-response pairs, extracting a conversation summary of the historical query text-response pairs, encoding the conversation summary to obtain a context vector;

[0025] Concatenating the sentence vector and the context vector to obtain a query vector, and using the query vector and keywords in the current query text as candidate queries;

[0026] Retrieving initial recall documents matching the candidate query from a target retrieval source corresponding to the candidate query;

[0027] Based on the initial recalled documents, the set of recalled documents to be scored is obtained.

[0028] According to a search enhancement generation method provided by the present invention, the candidate query further includes a user portrait of the user to whom the current query text belongs;

[0029] The step of acquiring the target search source includes:

[0030] Target search sources that match the user profile are screened from the initial search sources.

[0031] According to a retrieval enhancement generation method provided by the present invention, obtaining the set of recalled documents to be scored based on the initial recalled documents includes:

[0032] Determining the search weight of each target search source based on the user profile;

[0033] Calculating the recall priority of each of the initial recalled documents based on the retrieval weight and the semantic similarity of each of the initial recalled documents;

[0034] The initial recalled documents are sorted according to the recall priority to obtain the set of recalled documents to be scored.

[0035] According to a retrieval enhancement generation method provided by the present invention, the network parameters of the reinforcement learning network are optimized based on offline sample data;

[0036] The offline sample data includes the overall score of the document set, the retrieval strategy and the strategy reward of multiple retrieval rounds; the strategy reward includes the overall score of the document set and the response quality score of the query response text.

[0037] The present invention also provides a search enhancement generation device, comprising:

[0038] Retrieval unit, retrieves the document set to be scored for the current query text;

[0039] a strategy adjustment unit, which compares the overall score of the document set to be scored and the score threshold, and generates a retrieval strategy based on the overall score of the document set and the reinforcement learning network when the comparison result shows that the overall score of the document set is less than the score threshold;

[0040] A cyclic retrieval unit, based on the retrieval strategy, retrieves and obtains the next round of updated recalled document set;

[0041] The reply generation unit takes the updated recalled document set as the recalled document set to be scored, repeatedly compares the overall score of the recalled document set to be scored with the score threshold, until the comparison result is that the overall score of the document set is not less than the score threshold, and generates a query reply text for the current query text based on the updated recalled document set of the last round.

[0042] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, any one of the above-described search enhancement generation methods is implemented.

[0043] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-described search enhancement generation methods.

[0044] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any one of the above-mentioned search enhancement generation methods.

[0045] The retrieval enhancement generation method and device provided by the present invention compares the overall score of a recalled document set to be scored with a score threshold. If the comparison result shows that the overall score of the document set is less than the score threshold, a retrieval strategy is generated based on the overall score of the document set and a reinforcement learning network. Based on the retrieval strategy, the next round of updated recalled document sets is retrieved. The overall score of the recalled document set is repeatedly compared with the score threshold until the overall score of the document set meets the requirements. The updated recalled document set is generated through the final round of updated recalled document sets, achieving highly relevant and high-quality document recall. Furthermore, based on the recalled document set with good recall results, a highly accurate and reliable reply text is generated. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction is given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0047] Figure 1 It is a flowchart of the search enhancement generation method provided by the present invention;

[0048] Figure 2 It is a structural diagram of the search enhancement generation device provided by the present invention;

[0049] Figure 3 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0050] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0051] In response to the above problems, the present invention provides a retrieval enhancement generation method to achieve highly relevant and high-quality recall documents, thereby improving the accuracy of question and answer responses. Figure 1 : is a flow chart of the search enhancement generation method provided by the present invention, such as Figure 1 As shown, the method includes:

[0052] Step 110: retrieve and obtain a set of recalled documents to be scored for the current query text.

[0053] The query here refers to the natural language question or request entered by the user. In real-world scenarios, the current query can be the user's first-round query or a non-first-round query, meaning there were multiple rounds of responses before the current query.

[0054] Specifically, first, the current query text can be obtained. Then, by obtaining the semantic vector and keywords of the current query text, similarity matching can be performed using the semantic vector, or keyword matching can be performed to retrieve the corresponding recall documents from the search source. It is understandable that for matching based on semantic vectors or keywords in any search source, the recall documents obtained can include multiple documents. Therefore, the recall documents obtained here are multiple documents. Finally, the recall documents can be spliced ​​and combined to obtain a set of recall documents to be scored.

[0055] Step 120 , comparing the overall score of the document set to be scored and the score threshold, and if the comparison result shows that the overall score of the document set is less than the score threshold, generating a retrieval strategy based on the overall score of the document set and the reinforcement learning network.

[0056] Here, the overall score of the document set refers to the comprehensive quality assessment score of each recalled document in the recalled document set to be scored, which can be used to reflect the quality of the recalled documents retrieved. The scoring threshold here refers to the critical value used to judge whether the current recall result meets the preset conditions, which can be obtained based on manual annotation or based on the summary of historical data. In addition, the retrieval strategy here can include at least one of switching the target retriever, rewriting the query text, and adjusting the number of retrievals. And more detailed retrieval parameter settings. It can be understood that the adjustment of the retrieval strategy can directly affect the document quality of the recalled documents.

[0057] Specifically, first, the overall score of the document set to be scored can be calculated. For example, multi-dimensional key indicators can be calculated using the document set to be scored and the current query text, such as at least one of semantic similarity, keyword coverage, publication time, and retrieval source credibility. The calculated key indicators are then weighted to obtain a score for a single recalled document. The average score of the document set to be scored can then be used as the overall score of the document set.

[0058] Then, the overall score of the recalled document set to be scored can be compared with the scoring threshold. For example, if the document quality score or mutual information in the overall score of the document set is lower than the scoring threshold, the comparison result is considered to be that the overall score of the document set is lower than the scoring threshold. In this case, it means that the document quality of the recalled document set to be scored obtained in the current search round does not meet the preset conditions, and a new retrieval strategy needs to be output to obtain a higher-quality recalled document set.

[0059] For example, the overall score of the document set and the search weights of each search source can be used as state variables. A reinforcement learning network can then be used to output a search strategy based on these state variables, effectively implementing the action output for the current state. It should be noted that the network parameters in the reinforcement learning network can be optimized and determined based on collected offline sample data at a preset time. The reinforcement learning network here can be constructed based on the actor-critic framework.

[0060] In one embodiment, the retrieval strategy here may be to switch the retrieval source and increase the retrieval similarity threshold to retrieve recalled documents with higher semantic similarity from a more appropriate retrieval source, thereby providing relevance of the recalled documents to the current query text.

[0061] It should be noted that when the document quality score or mutual information in the overall score of the document set is not lower than the score threshold, the recalled document set to be scored obtained at this time can be used as contextual knowledge to respond to the current query text to generate a query response text for the current query text.

[0062] Step 130: Based on the retrieval strategy, retrieve and obtain the next round of updated recalled document set.

[0063] Specifically, the adjusted retrieval strategy can be used to re-search based on the current query text to obtain an updated set of recalled documents for the next retrieval round. It should be noted that due to the adjustment of the retrieval strategy, the retriever may be switched, query rewriting may be triggered, retrieval rounds may be increased, or retrieval may be terminated. When switching the retriever, triggering query rewriting, or increasing retrieval rounds, further adjustments to the retrieval parameters may also be triggered, such as adjustments to retrieval parameters such as the retrieval similarity threshold, expansion word weight, and vector temperature coefficient. Therefore, the updated set of recalled documents obtained by retrieval based on the adjusted retrieval strategy has a document content that is different from the document content of the previous round, and the document quality is better than that of the previous round.

[0064] Step 140: Use the updated recall document set as the recall document set to be scored, repeatedly compare the overall score of the recall document set to be scored with the score threshold, until the comparison result is that the overall score of the document set is not less than the score threshold, and generate a query response text for the current query text based on the updated recall document set of the last round.

[0065] Specifically, the updated recall document set can be used as the recall document set to be scored, and the quality assessment of the recall document set to be scored is repeatedly performed to calculate the overall score of the document set of the recall document set to be scored. Then, the overall score of the document set of the recall document set to be scored and the score threshold are repeatedly compared. When the comparison result shows that the overall score of the document set is less than the score threshold, a new retrieval strategy is generated based on the reinforcement learning network and the updated overall score of the document set. And through the new retrieval strategy, the next round of updated recall document set is retrieved, until the comparison result shows that the overall score of the document set is not less than the score threshold, then the query response text of the current query text is generated through the last round of updated recall document set. It should be noted that if the comparison result is that the overall score of the document set is always less than the score threshold, the retrieval can be terminated. Therefore, the maximum number of cyclic retrieval times can be set, for example, 3 times.

[0066] To generate query response text, the updated and recalled document set from the last search round can be used as context for generating the query response text. Using the knowledge provided by the updated and recalled document set, a large language model can be used to generate accurate and reliable query response text. For example, key paragraphs can be extracted from the updated and recalled document set from the last search round, and hint text can be embedded in it according to a preset citation format, such as "Refer to documents (1-n), answer the following question (current query question)." This supplemented hint text can then be fed into the large language model to obtain the query response text output by the large language model.

[0067] It should be noted that the "self-assessment-retrieval strategy update-re-retrieval" mechanism significantly improves the semantic relevance and information completeness of the final document set, enabling the generation of query response text based on more accurate and richer context, and achieves an average improvement of more than 20% in indicators such as BLEU / ROUGE compared to traditional one-time retrieval solutions.

[0068] The method provided by an embodiment of the present invention compares the overall score of a recalled document set to be scored with a score threshold. If the comparison result shows that the overall score of the document set is less than the score threshold, a retrieval strategy is generated based on the overall score of the document set and a reinforcement learning network. Based on the retrieval strategy, the next round of updated recalled document sets is retrieved. The overall score of the recalled document set is repeatedly compared with the score threshold until the overall score of the document set meets the requirements. An updated recalled document set is generated through the final round of updated recalled document sets, achieving highly relevant and high-quality document recall. Furthermore, based on the recalled document set with good recall results, a highly accurate and reliable reply text is generated.

[0069] Based on any of the above embodiments, in step 120, the reinforcement learning network includes a high-order network and a low-order network;

[0070] Generate a retrieval strategy based on the overall score of the document set and the reinforcement learning network, including:

[0071] generating a high-order search strategy based on the overall score of the document set and the high-order network; the high-order search strategy includes at least one of switching a target search engine, rewriting a query text, and adjusting the number of searches;

[0072] Based on the overall score of the document set and the low-order network, the low-order retrieval parameters in the high-order retrieval strategy are adjusted to obtain modified low-order retrieval parameters; the low-order retrieval parameters include at least one of a retrieval similarity threshold, an expanded word weight, and a vector temperature coefficient.

[0073] Specifically, reinforcement learning networks that can be used for retrieval strategy generation can be divided into high-order networks and low-order networks.

[0074] First, the overall score of the document set can be used as the current state quantity and input into the high-order network. The action of generating the current state quantity through the high-order network is to generate a high-order search strategy. The high-order search strategy here includes at least one of switching the target search engine, rewriting the query text, and adjusting the number of searches. For example, it can be switching the target search engine, rewriting the query text, and adjusting the number of searches, or it can be a combination of any two strategies, such as switching the target search engine and rewriting the query text, or it can include all three strategies at the same time, namely switching the target search engine, rewriting the query text, and adjusting the number of searches.

[0075] It should be noted that switching the target retriever refers to selecting a different underlying retrieval model or data source. For example, this could involve switching to a domain-specific retrieval model specific to the domain to which the current query belongs. It is understood that switching the target retriever can provide a more appropriate retrieval source for the recalled document.

[0076] Furthermore, rewriting the query text here refers to semantically expanding or structurally rewriting the current text query through natural language processing technology. For example, multiple candidate rewrites can be generated at once through comparison prompts, and Bandit-UCB can then select the optimal sentence structure within 0.5 milliseconds, effectively solving the problems of keyword fluctuation and query collapse. Rewriting the query text can include synonym replacement, entity enhancement, and question decomposition. It is understood that rewriting the query text can alleviate the low recall rate caused by the discrepancy between keywords and terms in the search source in the previous search round, thereby improving the recall rate of recalled documents.

[0077] Adjusting the search count here refers to adjusting the number of searches performed within the search source for a keyword or semantic vector, either increasing or decreasing it, thereby adjusting the number of documents recalled. It's understandable that increasing the search count can be considered a re-triggering of the search, avoiding local optima and exploring for better recalled documents.

[0078] Furthermore, after generating a high-order search strategy, in order to further improve the search strategy, the low-order search parameters in the high-order search strategy can be adjusted based on the overall score of the document set and the low-order network to obtain modified low-order search parameters. The low-order search parameters include at least one of a search similarity threshold, an expansion word weight, and a vector temperature coefficient.

[0079] It should be noted that the retrieval similarity threshold refers to the minimum similarity score for determining whether the retrieved document is relevant to the current query text. It can be used to control the recall accuracy of recalled documents and reduce redundant content.

[0080] In addition, the term expansion weight here refers to the weight coefficient assigned to the newly added words during query rewriting or keyword expansion. It can be understood that adjusting the term expansion weight can prevent the expanded words from excessively diluting the original query intent, thereby improving the relevance of the recalled documents.

[0081] The vector temperature coefficient (VTC) here refers to a hyperparameter used to adjust the steepness of the Softmax function distribution in semantic vector retrieval, controlling the retrieval granularity. As you can see, a larger VTC results in a search focusing on highly similar documents, suitable for exact matches; a smaller VTC results in a search focusing on recalling more potentially relevant documents, suitable for fuzzy queries.

[0082] The method provided by the embodiment of the present invention divides the reinforcement learning network into a high-order network and a low-order network. The high-order network solves "what to do" and realizes strategy selection, while the low-order network solves "how to do it" and realizes parameter adjustment, thereby avoiding the excessive burden of single network decision-making and realizing a more efficient, precise and accurate retrieval strategy.

[0083] Based on any of the above embodiments, the step of obtaining the overall score of the document set of the recalled document set to be scored includes:

[0084] Based on the current query text and each recalled document in the recalled document set to be scored, calculating a document quality index of each recalled document; the document quality index includes at least one of semantic similarity, keyword coverage, publication time, and retrieval source credibility;

[0085] Calculating a document quality score for each of the recalled documents based on the document quality index of each of the recalled documents;

[0086] Calculating the mutual information of each recalled document based on the semantic similarity of each recalled document and the maximum historical semantic similarity;

[0087] Based on the document quality score and mutual information of each recalled document, an overall score of the document set to be scored is calculated.

[0088] Specifically, first, the document quality index of each recalled document can be calculated by comparing the current query text with each recalled document in the set of recalled documents to be scored. For a single recalled document, the document quality index can be obtained by calculating at least one of the semantic similarity, keyword coverage, release time, and retrieval source credibility between the current query text and the recalled document. Among them, for calculating semantic similarity, the semantic similarity can be obtained by calculating the vector cosine similarity between the query text and the recalled document. For calculating keyword coverage, the keyword coverage can be obtained by counting the proportion of keywords in the current query text in the recalled documents. For calculating release time, it can be calculated by the interval between the release time of the data source to which the recalled document belongs and the current time. In addition, for retrieval source credibility, it can be determined by the pre-set weights of each retrieval source. For example, the retrieval source credibility of academic journals is 0.9, and the retrieval source credibility of news media is 0.7.

[0089] Then, after calculating the document quality index of each recalled document, the document quality score of each recalled document can be calculated using the document quality index of each recalled document. For a single recalled document, the document quality index of the recalled document can be weighted to obtain the document quality score of the recalled document.

[0090] Furthermore, the mutual information of each recalled document can be calculated by the semantic similarity of each recalled document and the maximum historical semantic similarity. Here, the maximum historical semantic similarity refers to the maximum semantic similarity in the historical retrieval of the current query text. Thus, the mutual information of each recalled document is calculated by the semantic similarity of each recalled document obtained by the current retrieval with the current query text and the maximum historical semantic similarity. The mutual information of any recalled document can be calculated by the following formula, as shown below:

[0091]

[0092] Where, Indicates any recall document Mutual information of Represents the current query text; represents the maximum historical semantic similarity, Represents the historical recalled documents. It can be understood that the mutual information of the recalled documents can be used to reflect the marginal contribution of each round of new recalled documents to the knowledge scope.

[0093] Finally, the overall score of the recalled document set can be calculated using the document quality scores and mutual information of each recalled document. For example, the document quality scores and mutual information of all recalled documents can be averaged to obtain the average quality score and average mutual information of the recalled document set. The average quality score and average mutual information are used as the overall score of the recalled document set.

[0094] It should be noted that the document quality score for the recalled documents is calculated by calculating at least one of semantic similarity, keyword coverage, publication time, and retrieval source credibility, ensuring the relevance, accuracy, and reliability of the recalled documents. Furthermore, by calculating the semantic similarity and maximum historical semantic similarity of the recalled documents, the mutual information of the recalled documents is obtained, avoiding repeated recommendations of the same document and increasing the breadth of knowledge covered by the recalled documents.

[0095] In order to further suppress the recall of redundant information, based on any of the above embodiments, the overall score of the document set to be scored is calculated based on the document quality score and mutual information of each recalled document, including:

[0096] Based on the mutual information of the recalled documents, a preferred recalled document set is obtained from the recalled document set to be scored;

[0097] Based on the document quality score and mutual information of each preferred recalled document in the preferred recalled document set, the overall score of the document set is calculated.

[0098] Specifically, the mutual information of each recalled document can be used to filter out the preferred recalled document set from the recalled document set to be scored. For example, a mutual information threshold can be set, and recalled documents with mutual information above the mutual information threshold are selected as the priority recalled document set, while recalled documents with mutual information below the mutual information threshold are filtered out.

[0099] Furthermore, the overall score of the document set can be calculated using the document quality scores and mutual information of each preferred recall document in the preferred recall document set. For example, the average document quality scores and mutual information scores of the preferred recall documents in the preferred recall document set can be calculated separately, and the average document quality score and average mutual information of the preferred recall document set can be used as the preferred recall document set.

[0100] It should be noted that the mutual information of recalled documents can reflect the information gain of recalled documents. Recalled documents with lower gains can be entered into a backoff table, and recalled documents with high gains can be written into a high-priority cache, thus obtaining a set of preferred recalled documents, thereby suppressing redundant recalls from the root and providing a quantitative signal for reinforcement learning reward shaping.

[0101] Based on any of the above embodiments, step 110 includes:

[0102] Encode the current query text to obtain a sentence vector;

[0103] If there are historical query text-response pairs, extracting a conversation summary of the historical query text-response pairs, encoding the conversation summary to obtain a context vector;

[0104] Concatenating the sentence vector and the context vector to obtain a query vector, and using the query vector and keywords in the current query text as candidate queries;

[0105] Retrieving initial recall documents matching the candidate query from a target retrieval source corresponding to the candidate query;

[0106] Based on the initial recalled documents, the set of recalled documents to be scored is obtained.

[0107] Specifically, the current query text can be encoded using the BERT encoder to obtain a 768-dimensional sentence vector. If there are historical query-response pairs, a conversation summary of these pairs is extracted. Here, a historical query-response pair refers to the historical query text and the query-response text corresponding to the historical query text. The conversation summary is semantically encoded using the BERT encoder to obtain a context vector. Furthermore, the sentence vector and the context vector can be concatenated to obtain the query vector.

[0108] It is understandable that when there is a historical conversation about the current query text, the historical query text response can provide contextual information for the current query text, so that the relevance and accuracy of the recalled documents can be significantly improved by searching the query vector obtained by concatenating the sentence vector and the context vector.

[0109] In addition, the keywords in the current query text can be obtained by performing word segmentation, part-of-speech tagging, and named entity recognition on the current query text, and a keyword table can be constructed through one-hot encoding. Then, the concatenated vector and keywords can be used as candidate queries to retrieve initial recall documents that match the candidate query from the target retrieval sources corresponding to the candidate query. For example, the initial recall documents can be retrieved by selecting retrieval sources related to the current query text from the initial retrieval sources and dividing the selected retrieval sources into target retrieval sources for keyword matching and target retrieval sources for vector retrieval.

[0110] It should be noted that embodiments of the present invention support simultaneous searches across multiple search sources. Specifically, the engine supports parallel pipelines for vector searches, BM25 searches, and Boolean keyword searches. During the multi-source search phase, vectors and keywords drive vector recall and Boolean searches, respectively. For real-time APIs, asynchronous callbacks are implemented and timeout fallback policies are set to ensure that latency throughout the recall phase is controlled within hundreds of milliseconds.

[0111] Then, the retrieved initial recall documents can be screened, for example, by sorting the initial recall documents based on the semantic similarity between the initial recall documents and the current query text, and selecting a preset number of initial recall documents with the highest ranking to form a set of recall documents to be scored.

[0112] In order to further improve the accuracy of the search, based on any of the above embodiments, the candidate query also includes a user profile of the user to whom the current query text belongs;

[0113] The step of acquiring the target search source includes:

[0114] Target search sources that match the user profile are screened from the initial search sources.

[0115] Specifically, a user profile can be constructed based on the historical query behavior, user information, and user feedback information of the user of the current query text. It is understandable that the user profile can reflect the domain distribution of the user's historical queries and user preferences, such as preference for academic or popular topics.

[0116] Therefore, when determining target search sources, the initial search sources can be screened to identify those that match the user profile. Specifically, each initial search source can be annotated with tags, such as domain, content depth, and timeliness. Initial search sources can be categorized into enterprise private document repositories, external API knowledge sources, and academic indexes. The degree of match between the user profile and each initial search source can then be calculated, with the top pre-set initial search sources selected as target search sources.

[0117] It's important to note that using user profiles to identify target search sources reduces irrelevant search source queries, improves document recall accuracy, and reduces API call costs. Furthermore, by using both context vectors and user profiles as candidate queries, we achieve both personalization and context enhancement. This allows us to capture both long-term user interests and immediate intent in conversational search, enabling precise document recall. This significantly improves the accuracy of query responses and enhances the user experience.

[0118] Based on any of the above embodiments, obtaining the set of recalled documents to be scored based on the initial recalled documents includes:

[0119] Determining the search weight of each target search source based on the user profile;

[0120] Calculating the recall priority of each of the initial recalled documents based on the retrieval weight and the semantic similarity of each of the initial recalled documents;

[0121] The initial recalled documents are sorted according to the recall priority to obtain the set of recalled documents to be scored.

[0122] Specifically, the matching degree between the user portrait and the label of each initial retrieval source can be calculated, and the calculated matching degree can be used as the retrieval weight of each target retrieval source. Then, the semantic similarity between each initial recall document and the current query text can be calculated. Furthermore, the recall priority of each initial recall document can be calculated by the retrieval weight of the target retrieval source to which each initial recall document belongs and the semantic similarity of each initial recall document. For any initial recall document, the recall priority of the initial recall document can be calculated by multiplying or adding the retrieval weight of the target retrieval source to which the initial recall document belongs and the semantic similarity of the initial recall document. Similarly, the recall priority of the remaining initial recall documents is calculated.

[0123] It can be understood that the recall priority is calculated by the semantic similarity reflecting the recall relevance and the retrieval weight that conforms to the user portrait, so that the recall priority not only considers the semantic relevance, but also considers the content that fits the user's preferences.

[0124] Based on any of the above embodiments, the network parameters of the reinforcement learning network are optimized based on offline sample data;

[0125] The offline sample data includes the overall score of the document set, the retrieval strategy and the strategy reward of multiple retrieval rounds; the strategy reward includes the overall score of the document set and the response quality score of the query response text.

[0126] Specifically, when the accuracy of a query response falls below a certain threshold, the network parameters of the reinforcement learning network can be optimized using offline sample data. Alternatively, the network parameters of the reinforcement learning network can be optimized at pre-set intervals. Furthermore, during the training process of the reinforcement learning network, high-order and low-order networks can share a value network to accelerate convergence.

[0127] Specifically, offline sample data can include the overall score of the document set, search strategies, and strategy rewards from multiple search rounds. For example, the overall score of the document set, search strategies, and strategy rewards from each search round can be stored in a log, and some data can be selected from this log to serve as offline sample data.

[0128] The strategic reward can include the overall score of the document set and the response quality score of the query response text. The response quality score of the query response text can be calculated by calculating the BLEU or ROUGE score of the query response text. For example, the strategic reward can be calculated by weighting the overall score of the document set and the response quality score of the query response text. The weights corresponding to the overall score of the document set and the response quality score of the query response text can be adaptively adjusted.

[0129] It should be noted that the strategy reward is constructed through the overall score of the document set and the response quality score of the query response text, taking into account both generation quality and knowledge diversity, and prompting the retrieval strategy to escape from the local optimum.

[0130] Based on any of the above embodiments, the present invention further provides a multi-source multi-round retrieval enhancement generation method based on hierarchical reinforcement learning self-feedback, the method comprising:

[0131] First, the user enters a question, and the current query text is obtained based on the user input question. Then, the current query text is parsed to generate a query vector, keywords, and user profile tags. Next, input parsing is performed, that is, the query vector, keywords, and user profile tags are input into the search source to retrieve the set of recalled documents to be scored for the current query text. The similarity and information gain (mutual information) of the recalled document set to be scored are calculated as the overall score of the document set. Then, the overall score of the recalled document set to be scored is compared with the scoring threshold. If the comparison result shows that the overall score of the document set is less than the scoring threshold, a retrieval strategy is generated based on the overall score of the document set and the reinforcement learning network to determine whether to re-examine or rewrite the query text. Based on the retrieval strategy, the next round of updated recalled document set is retrieved.

[0132] The updated recall document set is used as the recall document set to be scored, and the overall score of the document set and the score threshold of the recall document set to be scored are repeatedly compared. When the comparison result shows that the overall score of the document set is not less than the score threshold, the query response text of the current query text is generated based on the updated recall document set of the last round. In the query response text generation stage, the key paragraphs in the current query text and the updated recall document set of the last round can be added to the prompt text template through the pre-built prompt text template, and the supplemented prompt text is input into the large language model to generate the query response text through the large language model. Here, the prompt text template is constructed as a "reference template", which inserts the document number into the text and generates a reference. If the confidence of the generated query response text is lower than the threshold, the source document paragraph direct excerpt mode can be triggered to improve verifiability.

[0133] It should be noted that the introduction of a citation control mechanism and confidence judgment logic during the generation phase not only supports structured output of "text + citations" but also automatically falls back to a "paragraph excerpt + refinement" mode when model confidence is low, effectively improving the verifiability and compliance of the content. Furthermore, by introducing a policy memory cache and redundant document backoff mechanism, the system avoids the waste of resources caused by repeated searches and invalid generation, ensuring stability during multiple rounds of reasoning and policy adjustments.

[0134] After generating the query response text, the overall document set score, retrieval strategy, and strategy reward for each round are logged to construct offline sample data. This offline sample data is used to regularly optimize the network parameters of the reinforcement learning network.

[0135] It should be noted that the complete query status, action sequence, document score, and reward function value are recorded during the retrieval and policy process, generating a JSON-Ledger log structure that can be used for result auditing, user review, and policy tuning. Furthermore, the generation module outputs structured reference information, explicitly indicating the specific documents and locations referenced in the answer. This makes the question and answer content verifiable and traceable, significantly improving the relevance of external documents, answer accuracy, and the system's sustainable learning capabilities. This is particularly suitable for industries with high compliance requirements, such as law, finance, and healthcare.

[0136] Furthermore, the method provided by the embodiments of the present invention clearly decouples policy control, search invocation, rewriting guidance, and strategy generation, and offers flexible plug-in configuration capabilities. It also supports access to a variety of search sources, including private enterprise document repositories, open source knowledge bases, and real-time APIs. It can be customized based on industry characteristics, demonstrating broad commercial application and industry-wide promotional potential.

[0137] Based on any of the above embodiments, Figure 2 Schematic diagram of the structure of the search enhancement generation device provided by the present invention. Figure 2 As shown, the device includes:

[0138] Retrieval unit 210 retrieves a set of recalled documents to be scored for the current query text;

[0139] A strategy adjustment unit 220 compares the overall score of the document set to be scored and the score threshold, and generates a retrieval strategy based on the overall score of the document set and the reinforcement learning network if the comparison result shows that the overall score of the document set is less than the score threshold;

[0140] A cyclic retrieval unit 230 retrieves and obtains the next round of updated recalled document sets based on the retrieval strategy;

[0141] The reply generation unit 240 takes the updated recalled document set as the recalled document set to be scored, repeatedly compares the overall score of the recalled document set to be scored with the score threshold, until the comparison result is that the overall score of the document set is not less than the score threshold, and generates a query reply text for the current query text based on the updated recalled document set of the last round.

[0142] The device provided by an embodiment of the present invention compares the overall score of a recalled document set to be scored with a score threshold. If the comparison result shows that the overall score of the document set is less than the score threshold, a retrieval strategy is generated based on the overall score of the document set and a reinforcement learning network. Based on the retrieval strategy, the next round of updated recalled document sets is retrieved. The overall score of the recalled document set is repeatedly compared with the score threshold until the overall score of the document set meets the requirements. An updated recalled document set is generated through the final round of updated recalled document sets, achieving highly relevant and high-quality document recall. Furthermore, based on the recalled document set with good recall results, a highly accurate and reliable reply text is generated.

[0143] Based on any of the above embodiments, the reinforcement learning network includes a high-order network and a low-order network;

[0144] The policy adjustment unit is specifically used to:

[0145] generating a high-order search strategy based on the overall score of the document set and the high-order network; the high-order search strategy includes at least one of switching a target search engine, rewriting a query text, and adjusting the number of searches;

[0146] Based on the overall score of the document set and the low-order network, the low-order retrieval parameters in the high-order retrieval strategy are adjusted to obtain modified low-order retrieval parameters; the low-order retrieval parameters include at least one of a retrieval similarity threshold, an expanded word weight, and a vector temperature coefficient.

[0147] Based on any of the foregoing embodiments, the policy adjustment unit is further specifically configured to:

[0148] Based on the current query text and each recalled document in the recalled document set to be scored, calculating a document quality index of each recalled document; the document quality index includes at least one of semantic similarity, keyword coverage, publication time, and retrieval source credibility;

[0149] Calculating a document quality score for each of the recalled documents based on the document quality index of each of the recalled documents;

[0150] Calculating the mutual information of each recalled document based on the semantic similarity of each recalled document and the maximum historical semantic similarity;

[0151] Based on the document quality score and mutual information of each recalled document, an overall score of the document set to be scored is calculated.

[0152] Based on any of the foregoing embodiments, the policy adjustment unit is further specifically configured to:

[0153] Based on the mutual information of the recalled documents, a preferred recalled document set is obtained from the recalled document set to be scored;

[0154] Based on the document quality score and mutual information of each preferred recalled document in the preferred recalled document set, the overall score of the document set is calculated.

[0155] Based on any of the above embodiments, the retrieval unit is specifically configured to:

[0156] Encode the current query text to obtain a sentence vector;

[0157] If there are historical query text-response pairs, extracting a conversation summary of the historical query text-response pairs, encoding the conversation summary to obtain a context vector;

[0158] Concatenating the sentence vector and the context vector to obtain a query vector, and using the query vector and keywords in the current query text as candidate queries;

[0159] Retrieving initial recall documents matching the candidate query from a target retrieval source corresponding to the candidate query;

[0160] Based on the initial recalled documents, the set of recalled documents to be scored is obtained.

[0161] Based on any of the above embodiments, the candidate query further includes a user portrait of the user to whom the current query text belongs;

[0162] Based on any of the above embodiments, the retrieval unit is further specifically configured to:

[0163] Target search sources that match the user profile are screened from the initial search sources.

[0164] The retrieval unit is further specifically used for:

[0165] Determining the search weight of each target search source based on the user profile;

[0166] Calculating the recall priority of each of the initial recalled documents based on the retrieval weight and the semantic similarity of each of the initial recalled documents;

[0167] The initial recalled documents are sorted according to the recall priority to obtain the set of recalled documents to be scored.

[0168] Based on any of the above embodiments, the network parameters of the reinforcement learning network are optimized based on offline sample data;

[0169] The offline sample data includes the overall score of the document set, the retrieval strategy and the strategy reward of multiple retrieval rounds; the strategy reward includes the overall score of the document set and the response quality score of the query response text.

[0170] Figure 3 An example of a physical structure diagram of an electronic device is shown below. Figure 3 As shown, the electronic device may include: a processor 310 , a communications interface 320 , a memory 330 and a communication bus 340 , wherein the processor 310 , the communications interface 320 and the memory 330 communicate with each other via the communication bus 340 . The processor 310 can call the logic instructions in the memory 330 to execute the retrieval enhancement generation method, which includes: retrieving a set of recall documents to be scored for the current query text; comparing the overall score of the recall document set to be scored and the score threshold, and when the comparison result is that the overall score of the document set is less than the score threshold, generating a retrieval strategy based on the overall score of the document set and the reinforcement learning network; based on the retrieval strategy, retrieving the next round of updated recall document set; using the updated recall document set as the recall document set to be scored, repeatedly comparing the overall score of the recall document set to be scored and the score threshold until the comparison result is that the overall score of the document set is not less than the score threshold, and generating the query reply text of the current query text based on the updated recall document set of the last round.

[0171] Furthermore, the logic instructions in the aforementioned memory 330 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product, stored in a storage medium, includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0172] On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the retrieval enhancement generation method provided by the above methods, which includes: retrieving a set of recall documents to be scored for the current query text; comparing the overall score of the recall document set to be scored and a score threshold, and when the comparison result is that the overall score of the document set is less than the score threshold, generating a retrieval strategy based on the overall score of the document set and a reinforcement learning network; based on the retrieval strategy, retrieving the next round of updated recall document set; using the updated recall document set as the recall document set to be scored, repeatedly comparing the overall score of the recall document set to be scored and the score threshold until the comparison result is that the overall score of the document set is not less than the score threshold, and generating a query reply text for the current query text based on the updated recall document set of the last round.

[0173] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the retrieval enhancement generation method provided by the above-mentioned methods, the method comprising: retrieving a set of recall documents to be scored for the current query text; comparing the overall score of the recall document set to be scored and a score threshold, and when the comparison result is that the overall score of the document set is less than the score threshold, generating a retrieval strategy based on the overall score of the document set and a reinforcement learning network; based on the retrieval strategy, retrieving the next round of updated recall document set; using the updated recall document set as the recall document set to be scored, repeatedly comparing the overall score of the recall document set to be scored and the score threshold, until the comparison result is that the overall score of the document set is not less than the score threshold, and generating a query reply text for the current query text based on the updated recall document set of the last round.

[0174] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0175] Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a necessary general-purpose hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer, server, or network device) to execute the methods described in each embodiment or certain portions of the embodiments.

[0176] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A search enhancement generation method, characterized in that: include: Retrieve the document set to be scored for the current query text; Comparing the overall score of the document set to be scored and the score threshold, and if the comparison result shows that the overall score of the document set is less than the score threshold, generating a retrieval strategy based on the overall score of the document set and the reinforcement learning network; Based on the retrieval strategy, the next round of updated recalled document set is retrieved; Using the updated recalled document set as the recalled document set to be scored, repeatedly comparing the overall score of the recalled document set to be scored with the scoring threshold until the comparison result shows that the overall score of the document set is not less than the scoring threshold, and generating a query reply text for the current query text based on the updated recalled document set of the last round; The reinforcement learning network includes a high-order network and a low-order network; the high-order network is used to implement strategy selection, and the low-order network is used to implement parameter adjustment; The generating of the retrieval strategy based on the overall score of the document set and the reinforcement learning network includes: generating a high-order search strategy based on the overall score of the document set and the high-order network; the high-order search strategy includes at least one of switching a target search engine, rewriting a query text, and adjusting the number of searches; Based on the overall score of the document set and the low-order network, adjusting the low-order retrieval parameters in the high-order retrieval strategy to obtain modified low-order retrieval parameters; the low-order retrieval parameters include at least one of a retrieval similarity threshold, an expansion word weight, and a vector temperature coefficient; the vector temperature coefficient refers to a hyperparameter used to adjust the steepness of the Softmax function distribution in semantic vector retrieval, and is used to control the retrieval granularity; The retrieved document set to be scored for the current query text includes: Encode the current query text to obtain a sentence vector; If there are historical query text-response pairs, extracting a conversation summary of the historical query text-response pairs, encoding the conversation summary to obtain a context vector; Concatenating the sentence vector and the context vector to obtain a query vector, and using the query vector and keywords in the current query text as candidate queries; Retrieving initial recall documents matching the candidate query from a target retrieval source corresponding to the candidate query; Based on the initial recalled documents, the set of recalled documents to be scored is obtained.

2. The search enhancement generation method according to claim 1, characterized in that: The step of obtaining the overall score of the document set of the recalled document set to be scored includes: Based on the current query text and each recalled document in the recalled document set to be scored, calculating a document quality index of each recalled document; the document quality index includes at least one of semantic similarity, keyword coverage, publication time, and retrieval source credibility; Calculating a document quality score for each of the recalled documents based on the document quality index of each of the recalled documents; Calculating the mutual information of each recalled document based on the semantic similarity of each recalled document and the maximum historical semantic similarity; Based on the document quality score and mutual information of each recalled document, an overall score of the document set to be scored is calculated.

3. The search enhancement generation method according to claim 2, characterized in that: The overall score of the document set to be scored is calculated based on the document quality score and mutual information of each recalled document, including: Based on the mutual information of the recalled documents, a preferred recalled document set is obtained from the recalled document set to be scored; Based on the document quality score and mutual information of each preferred recalled document in the preferred recalled document set, the overall score of the document set is calculated.

4. The search enhancement generation method according to claim 1, characterized in that: The candidate query also includes a user profile of the user to whom the current query text belongs; The step of acquiring the target search source includes: Target search sources that match the user profile are screened from the initial search sources.

5. The search enhancement generation method according to claim 4, characterized in that: The step of obtaining the set of recalled documents to be scored based on the initial recalled documents includes: Determining the search weight of each target search source based on the user profile; Calculating the recall priority of each of the initial recalled documents based on the retrieval weight and the semantic similarity of each of the initial recalled documents; The initial recalled documents are sorted according to the recall priority to obtain the set of recalled documents to be scored.

6. The search enhancement generation method according to any one of claims 1 to 5, characterized in that: The network parameters of the reinforcement learning network are optimized based on offline sample data; The offline sample data includes the overall score of the document set, the retrieval strategy and the strategy reward of multiple retrieval rounds; the strategy reward includes the overall score of the document set and the response quality score of the query response text.

7. A search enhancement generation device, characterized in that: include: Retrieval unit, retrieves the document set to be scored for the current query text; a strategy adjustment unit, which compares the overall score of the document set to be scored and the score threshold, and generates a retrieval strategy based on the overall score of the document set and the reinforcement learning network when the comparison result shows that the overall score of the document set is less than the score threshold; A cyclic retrieval unit, based on the retrieval strategy, retrieves and obtains the next round of updated recalled document set; a reply generating unit, taking the updated recalled document set as the recalled document set to be scored, repeatedly comparing the overall score of the recalled document set to be scored with the score threshold, until the comparison result shows that the overall score of the document set is not less than the score threshold, and generating a query reply text for the current query text based on the updated recalled document set of the last round; The reinforcement learning network includes a high-order network and a low-order network; the high-order network is used to implement strategy selection, and the low-order network is used to implement parameter adjustment; The generating of the retrieval strategy based on the overall score of the document set and the reinforcement learning network includes: generating a high-order search strategy based on the overall score of the document set and the high-order network; the high-order search strategy includes at least one of switching a target search engine, rewriting a query text, and adjusting the number of searches; Based on the overall score of the document set and the low-order network, adjusting the low-order retrieval parameters in the high-order retrieval strategy to obtain modified low-order retrieval parameters; the low-order retrieval parameters include at least one of a retrieval similarity threshold, an expansion word weight, and a vector temperature coefficient; the vector temperature coefficient refers to a hyperparameter used to adjust the steepness of the Softmax function distribution in semantic vector retrieval, and is used to control the retrieval granularity; The retrieved document set to be scored for the current query text includes: Encode the current query text to obtain a sentence vector; If there are historical query text-response pairs, extracting a conversation summary of the historical query text-response pairs, encoding the conversation summary to obtain a context vector; Concatenating the sentence vector and the context vector to obtain a query vector, and using the query vector and keywords in the current query text as candidate queries; Retrieving initial recall documents matching the candidate query from a target retrieval source corresponding to the candidate query; Based on the initial recalled documents, the set of recalled documents to be scored is obtained.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the search enhancement generation method according to any one of claims 1 to 6 is implemented.

Images