Model-based data retrieval method and electronic device

Abstract

The application discloses a model-based data retrieval method and an electronic device, relates to the technical field of large model data retrieval, and receives a retrieval request of a user, wherein the retrieval request carries request data; determines a target retrieval strategy and initial retrieval parameters corresponding to the target retrieval strategy according to reward estimation values of a plurality of strategies respectively pre-stored by a target language model, wherein the reward estimation values are used to represent the performance of the strategies; performs retrieval on the request data through the target language model according to the target retrieval strategy and the initial retrieval parameters, and obtains initial retrieval results; outputs the initial retrieval results, and in response to a feedback operation of the user on the initial retrieval results, determines feedback reward data of the initial retrieval results; determines target retrieval parameters according to the feedback reward data and the initial retrieval parameters; and performs retrieval on the request data through the target language model according to the target retrieval strategy and the target retrieval parameters, and obtains target retrieval results.

Description

Technical Field

[0001] This application relates to the field of large model data retrieval technology, and in particular to model-based data retrieval methods and electronic devices. Background Technology

[0002] Retrieval-Augmented Generation (RAG) is an artificial intelligence architecture that combines document retrieval and text generation. It is widely used in scenarios such as open-domain question answering, long document summarization, knowledge reasoning, and enterprise knowledge base search. Currently, it usually uses predefined and fixed retrieval strategies and parameters, which cannot adapt to the diverse needs of different users and different queries. Therefore, how to achieve dynamic updates of the model's retrieval strategy and model parameters has become an urgent problem to be solved. Summary of the Invention

[0003] This application provides a model-based data retrieval method and electronic device to at least address the issues of how to implement model retrieval strategies and dynamically update model parameters.

[0004] This application provides a model-based data retrieval method, including:

[0005] Receive a user's search request, which carries request data;

[0006] Based on the reward estimates corresponding to multiple retrieval strategies pre-stored in the target language model, the target retrieval strategy and the initial retrieval parameters corresponding to the target retrieval strategy are determined, and the reward estimates are used to characterize the effectiveness of the retrieval strategy.

[0007] According to the target retrieval strategy and the initial retrieval parameters, the requested data is retrieved through the target language model to obtain initial retrieval results;

[0008] Output the initial search results and, in response to the user's feedback on the initial search results, determine the feedback reward data for the initial search results;

[0009] Based on the feedback reward data and the initial search parameters, the target search parameters are determined;

[0010] According to the target retrieval strategy and the target retrieval parameters, the requested data is retrieved through the target language model to obtain the target retrieval results.

[0011] This application also provides a model-based data retrieval device, including:

[0012] The transceiver module is used to receive user search requests, which carry request data.

[0013] The processing module is used to determine the target retrieval strategy and the initial retrieval parameters corresponding to the target retrieval strategy based on the reward estimates corresponding to multiple retrieval strategies pre-stored in the target language model. The reward estimates are used to characterize the effectiveness of the retrieval strategy.

[0014] The processing module is also configured to retrieve the request data according to the target retrieval strategy and the initial retrieval parameters through the target language model to obtain initial retrieval results;

[0015] The transceiver module is also used to output the initial search results;

[0016] The processing module is also configured to respond to the user's feedback operation on the initial search results and determine the feedback reward data for the initial search results;

[0017] The processing module is also used to determine the target retrieval parameters based on the feedback reward data and the initial retrieval parameters;

[0018] The processing module is also used to retrieve the request data according to the target retrieval strategy and the target retrieval parameters through the target language model to obtain the target retrieval results.

[0019] This application also provides an electronic device, including: a memory for storing a computer program; and a processor for executing the computer program to implement the steps of any of the above-described model-based data retrieval methods.

[0020] This application also provides a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, it implements the steps of any of the above-described model-based data retrieval methods.

[0021] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of any of the above-described model-based data retrieval methods.

[0022] This application receives a user's search request, which carries requested data. Based on the reward estimates corresponding to multiple search strategies pre-stored in the target language model, a target search strategy and its corresponding initial search parameters are determined. The reward estimates characterize the effectiveness of the search strategy. Following the target search strategy and initial search parameters, the request data is searched using the target language model to obtain initial search results. The initial search results are output, and in response to user feedback on the initial search results, feedback reward data for the initial search results is determined. Based on the feedback reward data and initial search parameters, target search parameters are determined. Following the target search strategy and target search parameters, the request data is searched using the target language model to obtain target search results. In this scheme, after determining the optimal search strategy, the search parameters can be updated based on multi-dimensional user feedback on the initial search results, thereby achieving dynamic adjustment and optimization of the model and effectively improving the accuracy of the model's search results. Attached Figure Description

[0023] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 The flowchart of a model-based data retrieval method provided in this application embodiment Figure 1 ;

[0025] Figure 2 The flowchart of a model-based data retrieval method provided in this application embodiment Figure 2 ;

[0026] Figure 3 A structural diagram of a model-based data retrieval device provided in an embodiment of this application;

[0027] Figure 4 This is a structural diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0028] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.

[0029] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0030] It should be noted that in the embodiments of this application, the words "exemplary" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of the words "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0031] Retrieval Augmented Generation (RAG) is an artificial intelligence architecture that combines document retrieval and text generation. It is widely used in scenarios such as open-domain question answering, long document summarization, knowledge reasoning, and enterprise knowledge base search. The core process of RAG includes: user query processing: the user inputs a natural language question; retrieval module: retrieves relevant documents or fragments from a large-scale document database; generation module: concatenates the retrieval results with the query to form a large language model that prompts the input, and outputs the answer or summary.

[0032] In practical applications, the candidate documents returned by the retrieval system often vary in quality. For example, for the same query, some documents are highly relevant to the answer, while others are redundant or even irrelevant. If the retrieval strategy or parameters are inappropriate, the model's generated results may suffer from problems such as incorrect citations, irrelevant answers, and lengthy and unnecessary output. Most systems use predefined, fixed retrieval strategies (such as pure vector retrieval or keyword retrieval) and parameters (such as recall number k, similarity threshold), which cannot adapt to the diverse needs of different users and different queries. Some solutions use reinforcement learning (RL) for optimization, but its training process is complex, converges slowly, requires a large number of simulation environments, and is difficult to deploy efficiently online and adapt in real time. Existing systems typically utilize user feedback (such as explicit likes / dislikes, implicit dwell time, and click behavior) at a simple weighted summation stage, failing to intelligently handle multi-source, heterogeneous, and potentially noisy feedback signals, resulting in inaccurate optimization signals.

[0033] Static retrieval strategies are the approach adopted by most RAG systems. The technical solution involves pre-configuring a single retrieval strategy (such as pure vector retrieval or keyword retrieval) and a fixed combination of parameters (such as recall k=5, similarity threshold τ=0.7), and continuously using this fixed configuration across all user queries and scenarios. However, static retrieval cannot adapt to diverse user feedback and changing scenarios: different users ask questions about the same knowledge base from different angles and with varying levels of difficulty, and a fixed retrieval strategy cannot match each user's questions. Furthermore, manually adjusting the retrieval strategy based on the user's response raises the barrier to entry for users.

[0034] To overcome the shortcomings of static policies, existing technologies have proposed optimization schemes based on reinforcement learning. The technical solution involves modeling the retrieval strategy selection and parameter tuning process of the RAG system as a Markov Decision Process (MDP), learning the optimal policy through the interaction between the agent and the simulated environment. The agent selects actions (such as changing the policy or adjusting parameters) based on the current system state (e.g., query features, historical feedback) and updates its policy model based on the reward signals returned by the environment (e.g., answer accuracy). However, reinforcement learning training is extremely complex, requiring a specialized team to simulate and evaluate a large number of state-action pairs. This results in massive computational demands, complex training, slow convergence, and high computational and time costs, hindering the rapid application of optimized retrieval strategies to real-world business applications.

[0035] Existing methods combine multi-armed bandit (MAB) and simulated annealing (SA) at the same level, with MAB handling policy selection and SA serving as a perturbation mechanism to avoid getting trapped in local optima. However, this approach still mixes policy selection and parameter tuning together, lacking hierarchical decoupling. This results in parameter optimization being achieved only through discrete perturbations, lacking effective exploration of the continuous parameter space. Furthermore, policy and parameter optimization interfere with each other, making it difficult to accurately attribute feedback signals and affecting convergence efficiency.

[0036] In addition, existing systems typically only use user feedback (such as explicit likes / dislikes, and implicit dwell time and click behavior) at a simple weighted summation stage, which cannot handle multi-source, heterogeneous, and potentially noisy feedback signals, resulting in inaccurate optimization signals.

[0037] In summary, to address all or part of the aforementioned technical problems, this application provides a model-based data retrieval method and electronic device. The method receives a user's retrieval request, which carries requested data. Based on reward estimates corresponding to multiple retrieval strategies pre-stored in a target language model, a target retrieval strategy and its corresponding initial retrieval parameters are determined. The reward estimates characterize the effectiveness of the retrieval strategy. Following the target retrieval strategy and initial retrieval parameters, the requested data is retrieved using the target language model to obtain initial retrieval results. The initial retrieval results are output, and in response to user feedback on the initial retrieval results, feedback reward data for the initial retrieval results is determined. Based on the feedback reward data and initial retrieval parameters, target retrieval parameters are determined. Following the target retrieval strategy and target retrieval parameters, the requested data is retrieved using the target language model to obtain target retrieval results. In this solution, after determining the optimal retrieval strategy, the retrieval parameters can be updated based on multi-dimensional user feedback on the initial retrieval results, thereby achieving dynamic adjustment and optimization of the model and effectively improving the accuracy of the model's retrieval results.

[0038] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0039] like Figure 1 As shown, Figure 1 A flowchart of a model-based data retrieval method provided for embodiments of this application, the method may include the following steps:

[0040] 101. Receive user search requests.

[0041] In this embodiment of the application, the user's search request can be used to instruct the user to perform a search through the target language model. The search request can carry request data, which is the data that the user needs to input into the target language model.

[0042] It should be noted that the target language model can be a large language model.

[0043] 102. Based on the reward estimates corresponding to the multiple retrieval strategies pre-stored in the target language model, determine the target retrieval strategy and the initial retrieval parameters corresponding to the target retrieval strategy.

[0044] In this embodiment of the application, when the target language model retrieves the user's requested data, it needs to execute the corresponding retrieval strategy. That is, the target language model can store multiple retrieval strategies, and each time during the retrieval process, it can select one of the multiple retrieval strategies as the retrieval strategy for this retrieval. The basis for selecting a specific retrieval strategy is the reward estimate value corresponding to each of the multiple retrieval strategies. The reward estimate value can be used to characterize the effectiveness of the retrieval strategy. The reward estimate value can be determined based on the retrieval performance of each retrieval strategy in the historical retrieval process.

[0045] It should be noted that since the reward estimate is applicable to characterizing the effectiveness of the retrieval strategy, when selecting a retrieval strategy, we definitely want the retrieval strategy with the best performance. Therefore, based on the reward estimate corresponding to each retrieval strategy, we can determine the target retrieval strategy from multiple retrieval strategies. This target retrieval strategy can be the retrieval strategy with the best performance determined based on the reward estimate. Then, we can also determine the initial retrieval parameters corresponding to this target retrieval strategy.

[0046] It should be noted that the initial retrieval parameters can be configuration parameters for the target language model. In other words, when the target language model executes the target retrieval strategy to retrieve data, it needs to be configured according to the initial retrieval parameters before it can start the retrieval. The initial retrieval parameters can be parameters corresponding to the target retrieval strategy, and can be determined based on the retrieval performance of the target retrieval strategy in the historical retrieval process.

[0047] 103. Based on the target retrieval strategy and initial retrieval parameters, retrieve the requested data using the target language model to obtain initial retrieval results.

[0048] In this embodiment, after the target retrieval strategy and initial retrieval parameters are determined, the target language model can be configured according to the initial retrieval parameters, and then the target retrieval strategy is executed to retrieve the requested data to obtain the initial retrieval results. That is, the input of the target language model is the requested data, and the output is the initial retrieval results. This retrieval process is the process of large model data retrieval in related technologies, which will not be elaborated in this embodiment.

[0049] In some embodiments, the requested data is retrieved using a target language model according to the target retrieval strategy and initial retrieval parameters to obtain initial retrieval results. Specifically, this may include: configuring the target language model using the target retrieval strategy and initial retrieval parameters; and retrieving the requested data using the configured target language model to obtain initial retrieval results.

[0050] It should be noted that the target language model retrieves data by executing specific retrieval strategies. In addition, the retrieval parameters for the target language model also need to be configured. Different retrieval parameters can be set for different retrieval strategies and requested data. Therefore, after determining the target retrieval strategy and initial retrieval parameters, the target language model can be configured using the target retrieval strategy and initial retrieval parameters. Then, the configured target language model can be used to retrieve the requested data to obtain the initial retrieval results.

[0051] 104. Output the initial search results and, in response to the user's feedback on the initial search results, determine the feedback reward data for the initial search results.

[0052] In this embodiment, after obtaining the initial search results, since the initial search results are obtained by the target language model after being configured according to the initial search parameters, and these initial search parameters can be understood as the configuration when executing the target search strategy in the historical search process, and may also include custom parameter data, the initial search results may not be accurate, and users may not necessarily accept the initial search results. Therefore, the search parameters can be updated based on the user's feedback on the initial search results. Thus, the initial search results can be output to the user first, and the user's feedback operation on the initial search results can be monitored. In response to the user's feedback operation on the initial search results, the feedback reward data of the initial search results can be determined. This feedback reward data can be calculated based on the user's feedback operation and is used to indicate the user's degree of acceptance of the initial search results.

[0053] 105. Determine the target search parameters based on the feedback reward data and the initial search parameters.

[0054] In this embodiment of the application, after obtaining the feedback reward data, the initial search parameters can be updated based on the feedback reward data to obtain the target search parameters. Of course, the target search parameters can be the same as or different from the initial search parameters. That is to say, if the user's feedback indicates that the user is relatively satisfied with the initial search results, then the initial search parameters may be more suitable for the current scenario, and the target search parameters may be the initial search parameters; if the user is not satisfied with the initial search results, then the initial search parameters need to be updated to obtain the updated target search parameters.

[0055] 106. Based on the target retrieval strategy and target retrieval parameters, retrieve the requested data using the target language model to obtain the target retrieval results.

[0056] In this embodiment of the application, after obtaining the target retrieval parameters, it can be considered that the target retrieval parameters are the final accurate retrieval parameters of the target language model in the current retrieval task. Therefore, the target language model can be configured through the target retrieval parameters and the target retrieval strategy can be executed to retrieve the requested data and obtain the target retrieval results. The retrieval process here is the same as step 103 and the retrieval process of the big oracle model in related technologies, and will not be described in detail here.

[0057] In this embodiment, a user's search request, carrying request data, is received. Based on reward estimates corresponding to multiple pre-stored search strategies in the target language model, a target search strategy and its corresponding initial search parameters are determined. The reward estimates characterize the effectiveness of the search strategy. The request data is searched using the target language model according to the target search strategy and initial search parameters to obtain initial search results. The initial search results are output, and in response to user feedback on the initial search results, feedback reward data for the initial search results is determined. Based on the feedback reward data and initial search parameters, target search parameters are determined. The request data is then searched using the target language model according to the target search strategy and target search parameters to obtain target search results. In this scheme, after determining the optimal search strategy, the search parameters can be updated based on multi-dimensional user feedback on the initial search results, thereby achieving dynamic adjustment and optimization of the model and effectively improving the accuracy of the model's search results.

[0058] like Figure 2 As shown, Figure 2 Another flowchart of a model-based data retrieval method provided for embodiments of this application, the method may include the following steps:

[0059] 201. Receive user search requests.

[0060] In this embodiment, the description of step 201 is the same as the detailed description of step 101 in the above embodiments, and will not be repeated in this embodiment.

[0061] 202. Obtain the historical average reward value and the number of times the strategy was selected for each retrieval strategy.

[0062] In this embodiment, each retrieval strategy pre-stored in the target language model may have been selected during historical retrieval processes. Therefore, the historical average reward value and the number of times the strategy was selected can be obtained for each retrieval strategy. The historical average reward value is determined based on user feedback during historical model retrieval processes; that is, the historical average reward value is the feedback reward data for each retrieval strategy during historical retrieval processes. Since multiple retrieval strategies are pre-stored in the target language model, and the retrieval strategy selected each time may be different, the number of times the strategy is selected for each retrieval strategy can be determined.

[0063] 203. Based on the historical average reward value and the number of times the strategy was selected, determine the estimated reward value for each retrieval strategy.

[0064] In this embodiment of the application, the calculation of the reward estimate can be achieved according to the following formula:

[0065]

[0066] in, Let the reward estimate be for strategy a. Let be the historical average reward value of strategy a. Let t be the number of strategy selections for strategy a, t be the current interaction round, and c be the exploration coefficient, which is a constant.

[0067] It should be noted that this formula can also be called the upper confidence bound formula, which is used to balance exploration and exploitation. That is, the historical average reward value represents the exploitation of the current known reward, and t is the total number of visits to the parent node (or root node) (in the current scenario, t is the total number of attempts for all actions). This represents the number of times action 'a' is accessed. An exploration item is formed, which encourages trying actions that are visited less often (because the unknown has greater potential). The exploration coefficient is used to control the balance between exploration and exploitation. The larger the exploration coefficient, the more the algorithm tends to explore; the smaller the exploration coefficient, the more it tends to exploit known high-yield actions.

[0068] In some embodiments, since the number of strategy selections needs to be considered when calculating the reward estimate, in order to avoid cold start bias, it is necessary to ensure that each retrieval strategy has been selected once. That is, when the target language model starts running, each of the multiple retrieval strategies pre-stored in the target language model can be selected sequentially to retrieve data. After all retrieval strategies have been selected once, the reward estimate can be calculated according to the formula to determine the target retrieval strategy.

[0069] In some embodiments, a search strategy may correspond to a searcher type, and different search strategies may correspond to different searcher types. The searcher type may include: keyword search, semantic search, hybrid search (keyword and semantic combination), contextual search, etc.

[0070] 204. Select the retrieval strategy corresponding to the largest estimated reward as the target retrieval strategy.

[0071] In this embodiment of the application, since the reward estimate is calculated by the historical average reward value and the number of times the strategy is selected, a higher reward estimate indicates that the user is more satisfied with the search results output by the search strategy and that the strategy is selected more often. Therefore, the search strategy with the highest reward estimate can be selected as the target search strategy.

[0072] 205. Determine the initial search parameters corresponding to the target search strategy based on the number of historical search documents and the dynamic mixed search ratio.

[0073] In this embodiment of the application, after determining the target retrieval strategy, it is necessary to set retrieval parameters for the target retrieval strategy. The process of setting the retrieval parameters can also refer to the retrieval parameters of the target retrieval strategy in the historical retrieval process. The retrieval parameters may include the number of documents to be retrieved and the mixed retrieval ratio. The number of documents to be retrieved may refer to the upper limit of the number of documents returned in the retrieval results (i.e., only the top k documents are retained after each retrieval). The mixed retrieval ratio may refer to the mixed ratio of the results of dense retrieval (such as vector similarity) and sparse retrieval (such as keyword matching) in a mixed retrieval system. The mixed retrieval ratio is a value in [0, 1].

[0074] It should be noted that the number of documents to be retrieved in the current retrieval process can be determined based on the number of historical retrieval documents in the historical retrieval process, and the ratio of dense / sparse retrieval in the current retrieval process can be determined based on the ratio of dense / sparse retrieval in the historical retrieval process. Thus, the number of retrieval documents and the ratio of mixed retrieval are determined as the initial retrieval parameters corresponding to the target retrieval strategy.

[0075] In some embodiments, the initial retrieval parameters may also include a similarity threshold. Since the target language model needs to retrieve the user's requested data, that is, to determine the content that is the same as, similar to or related to the requested data and return it to the user, the initial retrieval parameters may also include a similarity threshold. This similarity threshold can help the target language model measure whether the retrieved content is similar to the requested data.

[0076] In some embodiments, the initial search parameters may be determined based on the search parameters in the historical search process. Of course, since the search parameters also need to be updated based on user feedback, the initial search parameters may also be user-defined values, which are not specifically limited in this application.

[0077] In this embodiment, based on historical reward information, the current optimal strategy is dynamically selected from multiple predefined retrieval strategies, and the strategy selection is separated from subsequent parameter optimization to avoid mutual interference and improve convergence speed and effect.

[0078] 206. Based on the target retrieval strategy and initial retrieval parameters, retrieve the requested data using the target language model to obtain initial retrieval results.

[0079] 207. Output the initial search results.

[0080] In this embodiment, the description of steps 206-207 is the same as the detailed description of steps 103-104 in the above embodiments, and will not be repeated in this embodiment.

[0081] 208. In response to user feedback on initial search results, collect multi-dimensional feedback data from users.

[0082] In this embodiment of the application, after the initial search results are output to the user, the user can perform some feedback operations on the initial search results. The user's feedback operations can be monitored to collect multi-dimensional feedback data of the user. The multi-dimensional feedback data includes at least one of the following: explicit feedback data, implicit feedback data, indicator feedback data, and context feature feedback data.

[0083] It's important to note that explicit feedback data can refer to user likes and dislikes regarding the initial search results. Likes indicate user approval of the initial search results, while dislikes indicate user disapproval. Implicit feedback data can refer to user dwell time, click-through rate (CTR), and bounce rate regarding the initial search results. Longer dwell time suggests users find the initial search results more useful, higher CTR suggests more usefulness, and more bounces suggest less usefulness. Both explicit and implicit feedback data can be directly obtained through user feedback actions. Contextual feature feedback data can refer to query type (factual / explanatory), strategy type (dense / sparse / hybrid), and parameter configuration (Top-k, threshold, etc.). This contextual feature feedback data can be directly obtained.

[0084] It should be noted that the feedback data can refer to the accuracy and relevance data of the initial search results. Answer accuracy indicates answer quality, which can be determined by the coverage of the retrieved documents and the semantic consistency of the generated content. Retrieval document coverage can be calculated by measuring the proportion of content in the generated answer that can be supported by the retrieved documents (e.g., based on keyword matching, sentence vector similarity, or alignment between the answer fragment and the document fragment). Semantic consistency of the generated content can be calculated by measuring the similarity between the generated answer and the retrieved document in the sentence vector space, such as cosine similarity or other semantic distances. Relevance is the degree of semantic matching between the output document and the user query. Relevance indicates search quality, which can be determined by vector similarity and the keyword coverage between the document fragment and the query. Vector similarity can be calculated by measuring the similarity between the query vector and the retrieved document vector, such as cosine similarity. Keyword coverage between the document fragment and the query can be calculated by measuring the density or coverage of matches between the retrieved documents and the query keywords.

[0085] 209. Based on multi-dimensional feedback data, calculate the reward score for at least one dimension.

[0086] In this embodiment of the application, after collecting multi-dimensional feedback data, a reward score can be calculated for each dimension of feedback.

[0087] In some embodiments, when the multi-dimensional feedback data includes explicit feedback data, one point can be added for a user's likes and one point can be subtracted for a user's dislikes, thus obtaining a reward score for the explicit feedback dimension, which can be expressed by the following formula:

[0088]

[0089] in, This refers to the reward score in the explicit feedback dimension.

[0090] In some embodiments, when multi-dimensional feedback data includes implicit feedback data, weights can be assigned to dwell time data, click-through rate data, and bounce rate data respectively, and the reward score for the implicit feedback dimension can be calculated using the following formula:

[0091]

[0092] in, This refers to the reward score in the implicit feedback dimension. For click-through rate data, For stay time data, This is a time threshold (e.g., 10 seconds, used for normalization). To exit the data, , and For weights.

[0093] In some embodiments, when the multi-dimensional feedback data includes indicator feedback data, the accuracy data and relevance data of the initial search results can be obtained; the accuracy data and relevance data are weighted and summed to obtain the reward score of the indicator feedback dimension.

[0094] It should be noted that weights can be assigned to accuracy data and relevance data separately, and the reward score for the indicator feedback dimension can be calculated using the following formula:

[0095]

[0096] in, This refers to the reward score in the indicator feedback dimension. For accuracy data, For correlation data, and For weights.

[0097] In some embodiments, when the multi-dimensional feedback data includes context feature feedback data, a feature vector can be directly generated as the reward score for the context feature feedback dimension based on the query type, strategy type, and parameter configuration, which can be represented as:

[0098]

[0099] in, This refers to the reward score in the context feature feedback dimension. For query type, For strategy type, Configure parameters, This is an embedded function.

[0100] In summary, reward scores from four feedback dimensions can be aggregated to generate a feature vector:

[0101]

[0102] in, For each feedback dimension, the feature vector of the reward score is... For explicit dimensions of reward scoring, For implicit dimension reward scoring, The reward score is a feedback dimension for the indicator. The reward score is given to the context feature feedback dimension.

[0103] 210. Determine the feedback reward data for the initial search results based on the reward weights and reward scores for each dimension.

[0104] In this embodiment of the application, after calculating the reward scores for each dimension, reward weights can be assigned to each dimension, and a weighted sum can be performed based on the reward weights and reward scores to obtain the feedback reward data for the initial search results, which can be expressed by the following formula:

[0105]

[0106] in, This refers to the feedback reward data for the initial search results. The reward weights for each dimension, The reward scores are assigned to each dimension, where i represents the dimension. There are a total of j dimensions, which can include the explicit feedback dimension, implicit feedback dimension, indicator feedback dimension, and context feature feedback dimension mentioned above.

[0107] In some embodiments, before determining the feedback reward data of the initial search results based on the reward weights and reward scores of each dimension, the reward weights of each dimension can be determined. This may include: calculating the weight score of each dimension based on the reward scores of each dimension and a preset parameter vector; and determining the reward weight of each dimension based on the ratio of the weight score of each dimension to the sum of the weight scores.

[0108] It should be noted that the preset parameter vector can be a parameter vector obtained through machine learning, or it can be determined based on data from the historical model retrieval process. The weight score can be expressed as:

[0109]

[0110] in, This is the weighted score. That is, the preset parameter vector. This refers to the reward score for each dimension, where i represents the dimension. The weight score for each dimension can be calculated using this formula.

[0111] Then, the weighted scores for each dimension can be summed, and the reward weight can be determined based on the ratio between the weighted scores for each dimension and the sum, which can be expressed as:

[0112]

[0113] in, This refers to the reward weights for each dimension. We assign weights to each dimension, where i represents the dimension, and there are j dimensions in total. It is a function.

[0114] In some embodiments, to enhance robustness, a smoothing mechanism can be introduced to determine the reward weight of each dimension based on the ratio of the weight score of each dimension to the sum of the weight scores. Specifically, this may include determining the reward weight of each dimension based on the ratio of the weight score of each dimension to the sum of the weight scores and a preset temperature coefficient.

[0115] It should be noted that, through the analysis of... Incorporating the effect of temperature to avoid excessive bias towards a single signal can be expressed as:

[0116]

[0117] in, This refers to the reward weights for each dimension. Score the weights of each dimension. The preset temperature coefficient has a value of [0.5, 2], where i represents the dimension, and there are a total of j dimensions. It is a function.

[0118] In this embodiment of the application, feedback data on the initial search results in various dimensions are collected to ensure that the reward signal is used reasonably. Even if the feedback is sparse, it can still converge efficiently to obtain the most comprehensive and accurate feedback reward data, thereby reflecting the effectiveness of the strategies and parameters adopted by the target language model.

[0119] 211. Perturb the initial search parameters to obtain candidate search parameters.

[0120] 212. Determine the target retrieval parameters based on the reward data and feedback reward data corresponding to the candidate retrieval parameters.

[0121] In this embodiment of the application, when updating the search parameters, there is no precise parameter value that needs to be updated. Instead, the initial search parameters are first perturbed to generate a new candidate search parameter. Then, based on the reward data, it is determined whether the candidate search parameter is better than the output result of the target language model. If it is better than the initial search parameter, the initial search parameter can be updated using the candidate search parameter. If the output result is not as good as the initial search parameter, then there is no need to update the initial search parameter.

[0122] It should be noted that candidate search parameters can be expressed as:

[0123]

[0124] in, These are the candidate search parameters. These are the initial search parameters. These are random perturbation values, responsible for exploring new parameters and local optima. The perturbation is related to the number of rounds with temperature. In the early stages, higher temperatures (not actual temperatures, but a concept in the simulated annealing algorithm) result in larger perturbations, while in the later stages, lower temperatures result in smaller perturbations. At the same time, the closer p is to the optimum, the smaller the perturbation of p' becomes.

[0125] In some embodiments, the target retrieval parameter is determined based on the reward data corresponding to the candidate retrieval parameter and the feedback reward data. Specifically, this may include: when the reward data corresponding to the candidate retrieval parameter is greater than or equal to the feedback reward data, determining the candidate retrieval parameter as the target retrieval parameter; when the reward data corresponding to the candidate retrieval parameter is less than the feedback reward data, calculating the parameter update probability based on the difference between the reward data corresponding to the candidate retrieval parameter and the feedback reward data and the temperature decay function, and determining the target retrieval parameter from the candidate retrieval parameter and the initial retrieval parameter based on the parameter update probability.

[0126] It should be noted that after obtaining the candidate retrieval parameters, it is also necessary to configure the target language model using the candidate retrieval parameters, retrieve the requested data, receive user feedback, and calculate the reward data corresponding to the candidate retrieval parameters. If the reward data corresponding to the candidate retrieval parameters is greater than or equal to the feedback reward data (the reward data corresponding to the initial retrieval parameters), it can be said that the target language model outputs more accurate results and better meets user needs under the configuration of the candidate retrieval parameters. Therefore, the candidate retrieval parameters can be determined as the target retrieval parameters.

[0127] It should be noted that if the reward data corresponding to the candidate retrieval parameters is less than the feedback reward data, it means that the output of the target language model under the configuration of the candidate retrieval parameters is not as accurate as the result of the initial retrieval parameters and does not meet the user's needs. Therefore, the parameter update probability can be used to determine whether to update the parameters, thereby determining the final target retrieval parameters.

[0128] In some embodiments, when the reward data corresponding to the candidate search parameter is greater than or equal to the feedback reward data, the parameter update probability can also be used to determine whether to update the search parameter. However, in this case, the parameter update probability is 1, which means that the candidate search parameter will definitely be used to update the target search parameter.

[0129] In some embodiments, the parameter update probability can be expressed by the following formula:

[0130]

[0131]

[0132]

[0133] in, Update the probability for the parameter. This represents the difference between the reward data corresponding to the candidate retrieval parameters and the feedback reward data. Candidate search parameters, The reward data corresponding to the candidate search parameters. These are the initial search parameters. To provide feedback on reward data, Let be the temperature decay function. This is the initial temperature value. The cooling rate is denoted by t, which ranges from 0 to 1, and t represents the current number of searches performed by the target language model.

[0134] It can be seen that when When the value is greater than or equal to 0, it means that the reward data corresponding to the candidate search parameter is greater than or equal to the feedback reward data. Therefore, the parameter update probability is 1, which means that it will definitely be updated through the candidate search parameter, i.e., the target search parameter is the candidate search parameter; when When the value is less than 0, it means that the reward data corresponding to the candidate retrieval parameter is less than the feedback reward data. In this case, the specific parameter update probability can be calculated using the above formula, thereby deciding whether to update the retrieval parameter.

[0135] In the embodiments of this application, during the dynamic adjustment and optimization of the retrieval parameters, the adjustment is made by comparing the reward data corresponding to the candidate retrieval parameters and the initial retrieval parameters respectively, so that the model always remains optimal and the efficiency and accuracy of parameter optimization are improved.

[0136] 213. Based on the target retrieval strategy and target retrieval parameters, retrieve the requested data using the target language model to obtain the target retrieval results.

[0137] In this embodiment, the description of step 213 is the same as the detailed description of step 106 in the above embodiments, and will not be repeated in this embodiment.

[0138] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method.

[0139] like Figure 3 As shown, embodiments of this application also provide a model-based data retrieval device, which may include:

[0140] The transceiver module 301 is used to receive user search requests, which carry request data.

[0141] The processing module 302 is used to determine the target retrieval strategy and the initial retrieval parameters corresponding to the target retrieval strategy based on the reward estimates corresponding to the multiple retrieval strategies pre-stored in the target language model. The reward estimates are used to characterize the effectiveness of the retrieval strategy.

[0142] The processing module 302 is also used to retrieve the request data according to the target retrieval strategy and initial retrieval parameters through the target language model to obtain the initial retrieval results;

[0143] The transceiver module 301 is also used to output the initial search results;

[0144] The processing module 302 is also used to respond to the user's feedback operation on the initial search results and determine the feedback reward data for the initial search results;

[0145] The processing module 302 is also used to determine the target retrieval parameters based on the feedback reward data and the initial retrieval parameters;

[0146] The processing module 302 is also used to retrieve the requested data according to the target retrieval strategy and target retrieval parameters through the target language model to obtain the target retrieval results.

[0147] In some embodiments, the acquisition module 303 is used to acquire the historical average reward value and the number of times the strategy is selected for each retrieval strategy. The historical average reward value is determined based on user feedback during the historical model retrieval process.

[0148] The processing module 302 is specifically used to determine the estimated reward value for each retrieval strategy based on the historical average reward value and the number of strategy selections.

[0149] Processing module 302 is specifically used to select the retrieval strategy corresponding to the largest reward estimate as the target retrieval strategy;

[0150] The processing module 302 is specifically used to determine the initial retrieval parameters corresponding to the target retrieval strategy based on the number of historical retrieval documents and the dynamic mixed retrieval ratio of the target retrieval strategy.

[0151] In some embodiments, the processing module 302 is specifically configured to configure the target language model through the target retrieval strategy and initial retrieval parameters;

[0152] The processing module 302 is specifically used to retrieve the requested data through the configured target language model and obtain the initial retrieval results.

[0153] In some embodiments, the acquisition module 303 is specifically used to collect multi-dimensional feedback data from the user in response to the user's feedback operation on the initial search results. The multi-dimensional feedback data includes at least one of explicit feedback data, implicit feedback data, indicator feedback data, and context feature feedback data.

[0154] The processing module 302 is specifically used to calculate a reward score for at least one dimension based on multi-dimensional feedback data.

[0155] The processing module 302 is specifically used to determine the feedback reward data of the initial search results based on the reward weight and reward score of each dimension.

[0156] In some embodiments, the acquisition module 303 is specifically used to acquire the accuracy data and relevance data of the initial search results when the multi-dimensional feedback data includes indicator feedback data;

[0157] The processing module 302 is specifically used to perform weighted summation on the accuracy data and the correlation data to obtain the reward score of the indicator feedback dimension.

[0158] In some embodiments, the processing module 302 is further configured to calculate the weight score of each dimension based on the reward score of each dimension and the preset parameter vector.

[0159] The processing module 302 is also used to determine the reward weight of each dimension based on the ratio of the weight score of each dimension to the sum of the weight scores.

[0160] In some embodiments, the processing module 302 is specifically used to determine the reward weight of each dimension based on the ratio of the weight score of each dimension to the sum of the weight scores and a preset temperature coefficient.

[0161] In some embodiments, the processing module 302 is specifically used to perturb the initial search parameters to obtain candidate search parameters;

[0162] The processing module 302 is specifically used to determine the target retrieval parameters based on the reward data and feedback reward data corresponding to the candidate retrieval parameters.

[0163] In some embodiments, the processing module 302 is specifically used to determine the candidate retrieval parameter as the target retrieval parameter when the reward data corresponding to the candidate retrieval parameter is greater than or equal to the feedback reward data.

[0164] The processing module 302 is specifically used to calculate the parameter update probability based on the difference between the reward data corresponding to the candidate retrieval parameter and the feedback reward data, as well as the temperature decay function, when the reward data corresponding to the candidate retrieval parameter is less than the feedback reward data, and to determine the target retrieval parameter from the candidate retrieval parameters and the initial retrieval parameters based on the parameter update probability.

[0165] In the embodiments of this application, the description of the features corresponding to the model-based data retrieval device can be found in the relevant description of the model-based data retrieval method, which will not be repeated here.

[0166] like Figure 4 As shown, embodiments of this application also provide an electronic device, including a memory 401 and a processor 402, wherein the memory 401 stores a computer program, and the processor 402 is configured to run the computer program to perform the steps in any of the above-described model-based data retrieval method embodiments.

[0167] Embodiments of this application also provide a computer-readable storage medium storing a computer program configured to execute the steps in any of the above-described model-based data retrieval method embodiments at runtime.

[0168] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0169] Embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above-described model-based data retrieval method embodiments.

[0170] Embodiments of this application also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps in any of the above-described model-based data retrieval method embodiments.

[0171] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0172] The foregoing has provided a detailed description of the process monitoring of a storage system provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only intended to help understand the method and core ideas of this application. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of the claims of this application.

Claims

1. A model-based data retrieval method, characterized in that, The method includes: Receive a user's search request, which carries request data; Based on the reward estimates corresponding to multiple retrieval strategies pre-stored in the target language model, the target retrieval strategy and the initial retrieval parameters corresponding to the target retrieval strategy are determined. The reward estimates are used to characterize the effectiveness of the retrieval strategy, and the initial retrieval parameters are determined based on the number of historically retrieved documents and the dynamic mixed retrieval ratio of the target retrieval strategy. According to the target retrieval strategy and the initial retrieval parameters, the requested data is retrieved through the target language model to obtain initial retrieval results; Output the initial search results; In response to the user's feedback on the initial search results, multi-dimensional feedback data of the user is collected. The multi-dimensional feedback data includes at least one of the following: explicit feedback data, implicit feedback data, indicator feedback data, and contextual feature feedback data. Based on the multi-dimensional feedback data, a reward score for at least one dimension is calculated; The feedback reward data for the initial search results is determined based on the reward weights and reward scores for each dimension. The initial search parameters are perturbed to obtain candidate search parameters; The target retrieval parameters are determined based on the reward data corresponding to the candidate retrieval parameters and the feedback reward data. According to the target retrieval strategy and the target retrieval parameters, the requested data is retrieved through the target language model to obtain the target retrieval results.

2. The method according to claim 1, characterized in that, The step of determining the target retrieval strategy and the initial retrieval parameters corresponding to the target retrieval strategy based on the reward estimates corresponding to multiple retrieval strategies pre-stored in the target language model includes: Obtain the historical average reward value and the number of times the strategy was selected for each retrieval strategy. The historical average reward value was determined based on the user's feedback during the historical model retrieval process. Based on the historical average reward value and the number of times the strategy was selected, the estimated reward value corresponding to each retrieval strategy is determined; Select the retrieval strategy corresponding to the largest estimated reward value as the target retrieval strategy; The initial retrieval parameters corresponding to the target retrieval strategy are determined based on the number of historically retrieved documents and the dynamic mixed retrieval ratio of the target retrieval strategy.

3. The method according to claim 1, characterized in that, The step of retrieving the request data using the target language model according to the target retrieval strategy and the initial retrieval parameters to obtain initial retrieval results includes: The target language model is configured using the target retrieval strategy and the initial retrieval parameters; The requested data is retrieved using the configured target language model to obtain the initial search results.

4. The method according to claim 1, characterized in that, The step of calculating a reward score in at least one dimension based on the multi-dimensional feedback data includes: When the multi-dimensional feedback data includes indicator feedback data, the accuracy data and relevance data of the initial search results are obtained; The accuracy data and the correlation data are weighted and summed to obtain the reward score of the indicator feedback dimension.

5. The method according to claim 1, characterized in that, Before determining the feedback reward data for the initial search results based on the reward weights and reward scores of each dimension, the method further includes: The weight scores for each dimension are calculated based on the reward scores for each dimension and the preset parameter vector. The reward weight for each dimension is determined based on the ratio of the weight score of each dimension to the sum of the weight scores.

6. The method according to claim 5, characterized in that, The step of determining the reward weight for each dimension based on the ratio of the weight score of each dimension to the sum of the weight scores includes: The reward weight for each dimension is determined based on the ratio of the weight score of each dimension to the sum of the weight scores and the preset temperature coefficient.

7. The method according to claim 1, characterized in that, The step of determining the target retrieval parameter based on the reward data corresponding to the candidate retrieval parameter and the feedback reward data includes: When the reward data corresponding to the candidate retrieval parameter is greater than or equal to the feedback reward data, the candidate retrieval parameter is determined to be the target retrieval parameter; When the reward data corresponding to the candidate retrieval parameter is less than the feedback reward data, the parameter update probability is calculated based on the difference between the reward data corresponding to the candidate retrieval parameter and the feedback reward data and the temperature decay function. Based on the parameter update probability, the target retrieval parameter is determined from the candidate retrieval parameter and the initial retrieval parameter.

8. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the model-based data retrieval method as described in any one of claims 1 to 7.

Images