Parameter configuration method and device

Abstract

The present disclosure provides a parameter configuration method and device, which are applied to the field of computer technology. The parameter configuration method includes: obtaining an evaluation data set, the evaluation data set includes at least one query text and a first answer text corresponding to each query text; in response to a change in at least one configuration parameter of multiple processing modules in a retrieval enhancement system, inputting the query text into M changed retrieval enhancement systems, obtaining M groups of second answer texts corresponding to the M changed retrieval enhancement systems, each group of second answer texts includes multiple second answer texts corresponding to multiple query texts respectively, and M is an integer greater than or equal to 1; based on the first answer text and the M groups of second answer texts, determining target configuration parameters for multiple processing modules; and deploying the target configuration parameters to multiple processing modules.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a parameter configuration method and device. Background Art

[0002] With the development of technology, various models based on artificial intelligence (for example, language models, text generation models, etc.) have been widely used in various fields. For models, the parameters of the model play a very critical role in the performance of the model. Different parameter configurations affect the quality of the model's problem processing results. Summary of the invention

[0003] In view of this, the present disclosure provides a parameter configuration method and device.

[0004] According to the first aspect of the present disclosure, a parameter configuration method is provided, comprising: obtaining an evaluation data set, the evaluation data set comprising at least one query text and a first answer text corresponding to each query text; in response to a change in at least one configuration parameter of a plurality of processing modules in a retrieval enhancement system, inputting the query text into M changed retrieval enhancement systems to obtain M groups of second answer texts corresponding to the M changed retrieval enhancement systems, each group of second answer texts comprising a plurality of second answer texts corresponding to the query text, M being an integer greater than or equal to 1; determining target configuration parameters for a plurality of processing modules based on the first answer text and the M groups of second answer texts; and deploying the target configuration parameters to the plurality of processing modules.

[0005] According to an embodiment of the present disclosure, based on a first answer text and M groups of second answer texts, target configuration parameters for multiple processing modules are determined, including: based on the first answer text and M groups of second answer texts, N evaluation information for each group of second answer texts is determined from N dimensions to obtain M groups of evaluation information, where N is an integer greater than or equal to 1; based on the M groups of evaluation information, a target retrieval enhancement system is determined from M changed retrieval enhancement systems; and the configuration parameters of multiple processing modules in the target retrieval enhancement system are used as target configuration parameters.

[0006] According to an embodiment of the present disclosure, a target retrieval enhancement system is determined from M changed retrieval enhancement systems based on M groups of evaluation information, including: based on the M groups of evaluation information, m candidate retrieval enhancement systems whose evaluation information in at least one dimension meets a preset condition are determined from the M changed retrieval enhancement systems, the candidate retrieval enhancement systems representing the changed retrieval enhancement systems, and m being an integer greater than or equal to 1 and less than or equal to M; based on the m groups of evaluation information, contribution information of each of the m candidate retrieval enhancement systems in N dimensions is calculated; based on the m contribution information, a target retrieval enhancement system is determined from the m candidate retrieval enhancement systems.

[0007] According to an embodiment of the present disclosure, a target retrieval enhancement system is determined from M modified retrieval enhancement systems based on M groups of evaluation information, including: based on the M groups of evaluation information, the M modified retrieval enhancement systems are non-dominatedly sorted to determine m candidate retrieval enhancement systems, the candidate retrieval enhancement systems characterizing that the number of dominated numbers of the modified retrieval enhancement systems satisfies a threshold; based on the m groups of evaluation information, the crowding distances between each of the m candidate retrieval enhancement systems for N dimensions are calculated; based on the m crowding distances, the candidate retrieval enhancement system with the largest crowding distance is determined as the target retrieval enhancement system.

[0008] According to an embodiment of the present disclosure, a target retrieval enhancement system is determined from M modified retrieval enhancement systems based on M groups of evaluation information, including: based on an asynchronous continuous halving algorithm, the M groups of evaluation information are analyzed to determine the target retrieval enhancement system from the M modified retrieval enhancement systems.

[0009] According to an embodiment of the present disclosure, the evaluation information of the changed retrieval enhancement system in at least one dimension satisfies preset conditions, including: the total evaluation value of the changed retrieval enhancement system in N dimensions is not lower than that of the retrieval enhancement system before the change, and the evaluation value of the changed retrieval enhancement system in at least one dimension is higher than that of the retrieval enhancement system before the change.

[0010] According to an embodiment of the present disclosure, the method also includes: based on different categories of query texts, determining multiple cascaded processing modules in the retrieval enhancement system corresponding to each category of query text and at least one configuration parameter of each processing module, wherein different categories of query texts represent different application scenarios.

[0011] According to an embodiment of the present disclosure, the plurality of processing modules include a pre-retrieval module, a retrieval module, a data source enhancement module, a post-retrieval processing module, a query combining module and a generation module which are connected in sequence.

[0012] According to an embodiment of the present disclosure, the first answer text includes a third answer text and associated information, wherein the third answer text represents answer information corresponding to the query text, and the associated information represents source information of the answer in the third answer text.

[0013] The second aspect of the present disclosure provides a parameter configuration device, including: an acquisition module, used to acquire an evaluation data set, wherein the evaluation data set includes multiple query texts and a first answer text corresponding to each query text; a processing module, used to input the multiple query texts into M changed retrieval enhancement systems in response to a change in at least one configuration parameter of multiple processing modules in the retrieval enhancement system, and obtain M groups of second answer texts corresponding to the M changed retrieval enhancement systems, each group of second answer texts including multiple second answer texts corresponding to the query texts, M being an integer greater than or equal to 1; a determination module, used to determine target configuration parameters for multiple processing modules based on the first answer texts and the M groups of second answer texts; and a deployment module, used to deploy the target configuration parameters to the multiple processing modules.

[0014] A third aspect of the present disclosure provides an electronic device, comprising: one or more processors; and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors execute the above-mentioned parameter configuration method.

[0015] The fourth aspect of the present disclosure further provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, causes the processor to execute the above-mentioned parameter configuration method.

[0016] The fifth aspect of the present disclosure also provides a computer program product, including a computer program, which implements the above-mentioned parameter configuration method when executed by a processor.

[0017] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The above and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0019] Figure 1 One of the flow charts of the parameter configuration method according to an embodiment of the present disclosure is schematically shown;

[0020] Figure 2 The second flowchart of the parameter configuration method according to the embodiment of the present disclosure is schematically shown;

[0021] Figure 3 The third flowchart of the parameter configuration method according to the embodiment of the present disclosure is schematically shown;

[0022] Figure 4A fourth flowchart of a parameter configuration method according to an embodiment of the present disclosure is schematically shown;

[0023] Figure 5 A schematic diagram of a retrieval enhancement system according to an embodiment of the present disclosure is schematically shown;

[0024] Figure 6 The following schematically shows a processing flow chart of each processing module in the retrieval enhancement system according to an embodiment of the present disclosure;

[0025] Figure 7 A structural block diagram of a parameter configuration device according to an embodiment of the present disclosure is schematically shown; and

[0026] Figure 8 A block diagram of an electronic device suitable for implementing a parameter configuration method according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0027] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0028] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0029] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0030] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0031] The embodiments of the present disclosure provide a parameter configuration method and device. Before introducing the technical solution provided by the embodiments of the present disclosure, the related technologies involved in the present disclosure are first described.

[0032] In related technologies, with the development of technology, various models based on artificial intelligence (for example, language models, text generation models, etc.) have been widely applied to various fields. For models, the parameters of the model play a very critical role in the performance of the model, and different parameter configurations affect the quality of the model's problem processing results.

[0033] For example, Large Language Model (LLM) is an AI technology based on deep learning for understanding and generating natural language. Large language models have good language understanding and generation capabilities in the field of natural language processing. However, when dealing with situations that are beyond the scope of their training data or require the latest data, large language models often have difficulty providing accurate answers.

[0034] To address this problem, researchers proposed the Retrieval Augmented Generation (RAG) framework and technology. RAG combines retrieval and generation capabilities to dynamically retrieve information from external knowledge sources and generate responses based on the retrieved information, thereby significantly improving the accuracy and relevance of answers generated by the model and reducing the generation of hallucinations.

[0035] In the implementation and application of the RAG system, although it has significant advantages in processing complex queries and improving the accuracy of information retrieval, the system design itself faces several key challenges. These challenges not only affect the performance of the system, but also determine its applicability and flexibility in different application scenarios. Specifically, there are two problems that need to be solved:

[0036] (1) Limitations of local optimization: The RAG system is characterized by its step-by-step, sequential processing flow, in which the output of each step directly affects the performance of the next step. For example, an effective "pre-retrieval operation" lays the foundation for the subsequent "retrieval operation". However, this dependence also leads to an important problem: optimizing a certain step alone (such as optimizing only the "retrieval operation") may only achieve the best local effect, but cannot guarantee the overall performance optimization of the entire RAG system. This is because the overall optimization of the RAG system is not just the sum of the optimization of each individual part, but requires balance and coordination in multiple dimensions in the final generated text.

[0037] For example, the multiple dimensions may include: context relevance, fidelity of generated content, relevance of generated content, text generation quality, computational efficiency, etc.

[0038] The key to solving this challenge is to optimize the overall process. It not only involves optimizing each independent goal, but also requires finding a balance between these goals to achieve the optimization of the overall process.

[0039] (2) Configuration requirements for different application scenarios: The performance requirements of RAG systems vary in different application scenarios. For example, in some scenarios, the relevance of the generated content may be the primary goal, while in other scenarios, response time or the quality of the generated content may be more important. In some cases, it may be necessary to consider several objectives at the same time, such as optimizing the fidelity of the answer, the quality of the generated text, and computational efficiency. Since there may be a trade-off between these objectives, manually adjusting the RAG configuration for each specific application scenario to achieve the best performance is not only complex and time-consuming, but also difficult to guarantee the optimal results.

[0040] At present, in the optimization process, people usually only focus on the local performance optimization of a certain independent step in the RAG system, such as the optimization of the retriever effect and the optimization of the input data, but fail to fully consider the overall balance between multiple objective values. In addition, for the optimization and configuration of multiple objectives in different application scenarios, engineers need to manually conduct a large number of experiments to find the right configuration, which is time-consuming and inefficient.

[0041] Before further describing the embodiments of the present disclosure in detail, the nouns and terms involved in the embodiments of the present disclosure are described. The nouns and terms involved in the embodiments of the present disclosure are subject to the following interpretations.

[0042] The RAG system is a technical architecture that combines information retrieval and generative models, and aims to enhance the performance of generative models in specific tasks by introducing external knowledge bases or documents. The core idea of ​​the RAG system is that in the process of generating answers, in addition to relying on the existing language model capabilities, it also retrieves information from external data sources (such as documents, knowledge bases, web pages, etc.) to provide more background information or details to help generate more accurate and relevant answers.

[0043] Non-dominated Sorting: In multi-objective optimization, a solution is considered "non-dominated" if it cannot be completely surpassed by other solutions in all objectives. Non-dominated sorting is to classify all solutions in this way to identify those that are relatively excellent in each objective.

[0044] Crowding Distance: It is used to measure the uniformity of the distribution of solutions in the target space, not only based on the performance of each solution, but also considering its relative position in the target space. This method helps the algorithm to make a comprehensive evaluation when selecting solutions, rather than based on indicators such as expected improvement (EI). By considering the spatial distribution of solutions, crowding distance ensures the diversity and representativeness of the optimal solution.

[0045] The asynchronous scheduling mechanism is a method for handling tasks and events in which the execution of tasks does not block the execution of the main program or other tasks. In the asynchronous scheduling mechanism, tasks can be submitted and executed in the background or in another thread without returning results immediately. Instead, the scheduler notifies the caller or triggers the corresponding callback function when the task is completed. The asynchronous scheduling mechanism is often used in conjunction with the event loop to handle concurrent tasks and events. In the event loop, the system polls the event queue and schedules the corresponding task or handler according to the event type. This mechanism can effectively improve the concurrent performance and responsiveness of the system, especially when dealing with I / O intensive tasks or waiting for external resources.

[0046] In traditional optimization algorithms, it is usually necessary to wait for each configuration (or each task) to be evaluated before moving on to the next step. However, the asynchronous scheduling mechanism does not require all configurations to be evaluated, but can make immediate adjustments based on the partial information that has been completed. The asynchronous scheduling algorithm can more flexibly adjust the state and resource allocation in a timely manner based on the performance of different solutions. This means that the algorithm does not need to wait until all results are completed before updating, but instead continuously adjusts the strategy based on the existing data at each step.

[0047] The embodiment of the present disclosure provides a parameter configuration method, including: obtaining an evaluation data set, the evaluation data set includes at least one query text and a first answer text corresponding to each query text; in response to a change in at least one configuration parameter of multiple processing modules in a retrieval enhancement system, inputting the query text into M changed retrieval enhancement systems, obtaining M groups of second answer texts corresponding to the M changed retrieval enhancement systems, each group of second answer texts includes multiple second answer texts corresponding to the query text, and M is an integer greater than or equal to 1; based on the first answer text and the M groups of second answer texts, determining target configuration parameters for multiple processing modules; deploying the target configuration parameters to multiple processing modules. By introducing an end-to-end automated parameter configuration method, refined parameter tuning is performed on each processing module in the retrieval enhancement system. According to the provided evaluation data set and the specified optimization target, a comprehensive analysis and adjustment process is automatically started to adapt to the needs of different application scenarios. Not only can the efficiency of parameter configuration optimization be improved, but also the accuracy and applicability of the optimization results are ensured. The overall development efficiency and flexibility are greatly improved.

[0048] The following will be passed Figure 1 to Figure 6 The parameter configuration method of the disclosed embodiment is described in detail.

[0049] Figure 1 One of the flow charts of the parameter configuration method according to an embodiment of the present disclosure is schematically shown.

[0050] like Figure 1 As shown, the parameter configuration method of this embodiment includes operations S210 to S240.

[0051] In operation S210 , an evaluation data set is acquired, where the evaluation data set includes at least one query text and a first answer text corresponding to each query text.

[0052] Exemplarily, the evaluation dataset can be understood as a dataset constructed for the retrieval enhancement system. The evaluation dataset can be used as a sample dataset of a model to which the retrieval enhancement system is applied. The retrieval enhancement system can be a technology that improves the accuracy of the model generation results. For example, the retrieval enhancement system can be a RAG system, or an attention mechanism, etc.

[0053] The model can be a natural language model, a text-graph model, a text generation model, etc.

[0054] The query text may be the input part of the sample data set for model training. The first answer text may be the standard answer result corresponding to the query text. The first answer text may be the result obtained by artificially processing the query text in advance. As sample data for model training, the query text usually appears in pairs with the first answer text. Here, the query text may be one or more. Multiple query texts may be the input of models for different application scenarios.

[0055] For example, when the model is a natural language model and the retrieval enhancement system is a RAG system, the query text may be "What is the average distance from the earth to the sun?", and the first answer text may be "149.6 million kilometers".

[0056] In operation S220, in response to a change in at least one configuration parameter of multiple processing modules in the retrieval enhancement system, the query text is input into M changed retrieval enhancement systems to obtain M groups of second answer texts corresponding to the M changed retrieval enhancement systems, each group of second answer texts including multiple second answer texts corresponding to the query text, and M is an integer greater than or equal to 1.

[0057] Exemplarily, the retrieval enhancement system may include multiple processing modules, each processing module is used to process the query text and / or the original query result of the model so that the second answer text has a higher degree of fit with the first answer text. Each processing module has at least one parameter, and each parameter corresponds to a parameter adjustment range. Within the parameter adjustment range, when a parameter in a processing module is changed, the performance of the model to which the retrieval enhancement system is applied can be changed, thereby predicting different second answer texts. That is, for the same query text, whenever a parameter in a processing module is changed, a second answer text can be predicted. In the case where different types of parameters in the processing modules have multiple adjustable values ​​and different processing modules have different adjustable parameters, multiple second answer texts (i.e., a group of second answer texts) can be predicted for the same query text.

[0058] For example, in the case where the model is a large language model, the multiple processing modules in the retrieval enhancement system may include a pre-retrieval module, a retrieval module, a data source enhancement module, a post-retrieval processing module, a query combination module, and a generation module connected in sequence. For example, the pre-retrieval module may query rewriting, query block size, and parameter configuration of the query enhancement strategy. The size of the query block usually refers to how to divide the query or text into multiple parts (blocks) for processing when processing long queries or large-scale documents. For example, it is divided into blocks by keywords, phrases, or sentences. When the size of the query block is set differently, the corresponding predicted results may be different. The parameters of the retrieval module may include index type and search depth. The index type may include inverted index and vector index. The search depth refers to the number of candidate documents or data items returned during the retrieval process. In short, it controls how many results are returned from the index during retrieval. The search depth can be 5, 10, 15, etc.

[0059] For example, the multiple processing modules in the retrieval enhancement system may include a pre-retrieval module, a retrieval module, a data source enhancement module, a post-retrieval processing module, a query combination module and a generation module connected in sequence. Each processing module has corresponding configuration parameters, and each configuration parameter has its specific selection range; the various configurations in all steps together constitute the search space optimized for the entire retrieval enhancement system. For example, in the retrieval operation step corresponding to the retrieval module, the retrieval algorithm is a configuration parameter, and its optional values ​​include multiple models, and each model has its corresponding multiple configuration parameters. Without loss of generality, define the steps , 1≤ ≤ The configuration item set is , for any given configuration item ∈ , 1≤ ≤ , whose optional parameter interval (continuous) or set (discrete) is expressed as ; then the entire search space is .

[0060] Pre-retrieval operation: During the pre-retrieval operation phase of the retrieval enhancement system, a series of refined strategies can be adopted to optimize the query process and ensure efficient and accurate processing of user queries. This includes implementing different query rewriting mechanisms to automatically modify the original user query to improve its clarity and searchability, such as synonym replacement and disambiguation. At the same time, query enhancement strategies can be adopted to enrich the context of the query by adding keywords or background information. In addition, the size of the query chunks can be adjusted for different types of queries to accommodate efficient information processing and retrieval in large databases.

[0061] In the retrieval operation step, you need to decide which retrieval method to use. The models that can be selected include traditional keyword-based sparse retrieval methods (such as BM25) and dense vector retrieval methods based on deep learning (such as the Transformer-based series model). Different retrieval models have their own specific configuration parameters that need to be selected, and these configuration parameters will affect the retrieval performance.

[0062] Data source enhancement: When integrating external data sources, it is necessary to consider the selection and weighting of data sources, which can include selecting specific news sources, academic databases, or other professional knowledge bases, and how to balance the weights of these sources.

[0063] Optimization configuration of post-retrieval processing: In the post-retrieval processing stage of the retrieval enhancement system, it is necessary to focus on optimizing and adjusting multiple key configurations, which can include filtering, sorting, and compressing parameters of the optimization results to ensure that the most relevant and high-quality information is provided for the next step of language model generation.

[0064] The query combination module can include information and query combination strategies. When combining the retrieved information with the original query, it is necessary to consider subtle adjustments to the information fusion strategy, including the design of combination parameters and the adjustment of information presentation methods, which will affect the accuracy of the final generation. In this step, users can define multiple parameter templates for selection.

[0065] Generate a module. When the model is LLM, the configuration items and parameters that need to be determined may include: 1) which LLM model to use, such as Llama2-13B, ChatGLM3-6b, Qwen-14b, etc.; 2) LLM sampling strategy: Greedy, BeamSearch, TopK, etc.; 3) other LLM calling parameters, such as temperature, etc.

[0066] The complexity of the retrieval enhancement system comes not only from its multi-step process design, but also from the unique requirements within each step. For example, in the retrieval operation, it is necessary to deal with the selection of various retrieval algorithms and the optimization of the corresponding retriever parameters; while in the data source enhancement and post-retrieval processing stages, it is necessary to balance the integration of external information sources and the effective presentation of processing results. Through a comprehensive understanding and analysis of these links, the key optimization points in the system can be more accurately located, thus providing an effective path to solving the overall complexity.

[0067] In operation S230, target configuration parameters for a plurality of processing modules are determined based on the first answer text and the M groups of second answer texts.

[0068] Exemplarily, the target configuration parameters can be understood as the setting values ​​of the parameters of each processing module of the detection enhancement system in the model when the results predicted by the model meet the user's expectations. For example, meeting user expectations can mean that multiple key dimensions such as the context relevance, content fidelity, content relevance, text generation quality, and computational efficiency of the second answer text meet the set standards. Meeting user expectations can also mean that when the fit of the second answer text and the fit of the first answer text reach a preset threshold, the corresponding settings of the parameters of each processing module of the detection enhancement system in the model. The preset threshold can be 98%, 98.5%, 99%, etc.

[0069] For example, when the model is a large language model, the query blocks in the pre-retrieval module are divided into blocks according to keywords and sentences, and the search depth in the retrieval module is set to 10, which can make the fit of the second answer text reach 98% with the fit of the first answer text.

[0070] Each processing module of the search enhancement system has one or more parameters to be configured. By changing each parameter in each module, the corresponding second answer text is obtained. For the same query text, the fit between the second answer text and the first answer text under different parameters is compared, so as to determine the optimal second answer text. The parameters in each processing module corresponding to the optimal second answer text are the target configuration parameters.

[0071] The target configuration parameters of different types of models may be different. The target configuration parameters of the same model may also be different when the user has different expectations. For example, when the user expects the model to generate results within 3 seconds and when the user expects the model to generate the highest quality text, the corresponding target configuration parameters of the model are different.

[0072] In operation S240 , the target configuration parameters are deployed to a plurality of processing modules.

[0073] Exemplarily, when the target configuration parameters of each processing module in the search enhancement system are determined, the parameters of each processing module are updated to the target configuration parameters, thereby obtaining a model that meets the user's expectations. The model with the target configuration parameters can be used to process the corresponding data.

[0074] The configuration optimization process of the retrieval enhancement system is carried out by introducing an end-to-end automated hyperparameter tuning framework. This automated tool eliminates the need for users to perform tedious manual configuration and verification repeatedly. A comprehensive optimization process can be started by simply entering the target dataset and optimization target. The performance of the entire retrieval enhancement system is automatically analyzed, and the configuration of each link (such as algorithm selection, parameter configuration, etc.) is adjusted to meet the specific needs of different application scenarios.

[0075] It is understandable that by introducing an end-to-end automated parameter configuration method, the parameters of each processing module in the retrieval enhancement system are finely tuned. According to the provided evaluation data set and the specified optimization goal, a comprehensive analysis and adjustment process is automatically started to adapt to the needs of different application scenarios. It not only improves the efficiency of parameter configuration optimization, but also ensures the accuracy and applicability of the optimization results. It greatly improves the overall development efficiency and flexibility.

[0076] Figure 2 The second flowchart of the parameter configuration method according to the embodiment of the present disclosure is schematically shown.

[0077] As described above, in operation S230, based on the first answer text and the M groups of second answer texts, target configuration parameters for multiple processing modules are determined. Figure 2 As shown, the operation may further include operation S231 to operation S233.

[0078] In operation S231, based on the first answer text and the M groups of second answer texts, N evaluation information for each group of second answer texts is determined from N dimensions to obtain M groups of evaluation information, where N is an integer greater than or equal to 1.

[0079] In operation S232, a target retrieval enhancement system is determined from the M modified retrieval enhancement systems according to the M sets of evaluation information.

[0080] In operation S233, configuration parameters of multiple processing modules in the target retrieval enhancement system are used as target configuration parameters.

[0081] Exemplarily, the N dimensions may include context relevance, fidelity of generated content, relevance of generated content, text generation quality, and computational efficiency.

[0082] For the three evaluation indicators of "relevance of context", "fidelity of generated content", and "relevance of generated content", RAGAS (Retrieval-Augmented Generation with Attention Scoring) or ARES (Analysis of Risk and Environmental Systems) can be used. RAGAS proposes to use a large language model with strong capabilities to decompose and construct a triple for evaluation ("query", "retrieved information", "generated content") by setting specific prompt words to calculate the final score for the three indicators of "relevance of context", "fidelity of generated content", and "relevance of generated content". ARES uses a large model to synthesize a data set for the corpus of a specific scenario, and then fine-tunes a model for scoring the above three evaluation indicators based on the relevant BERT variant model. ARES also uses prediction-driven inference (PPI) combined with a small amount of annotated data to provide confidence for the final prediction (i.e., score) of the above scoring model.

[0083] For the evaluation metric of “text generation quality”, the quality of generated text can be directly evaluated using existing fine-tuned LM models. These models can evaluate the fluency, naturalness, and grammatical correctness of the text.

[0084] Regarding the evaluation metric of "computational efficiency": The evaluation of computational efficiency is relatively straightforward, involving measuring indicators such as processing time and resource consumption.

[0085] Based on the indicators of the above five key dimensions, the performance of the retrieval enhancement system can be comprehensively evaluated, and necessary optimization can be performed based on the evaluation results. This is crucial to ensure the effectiveness and reliability of the retrieval enhancement system in practical applications. It should be noted that key indicators can be added as needed according to the needs of actual application scenarios, but the indicators must be of practical significance around the retrieval enhancement system itself.

[0086] For example, for the same query text, multiple second answer texts (a group of second answer texts) are obtained by different configuration parameters of each processing module in the retrieval system. Multiple query texts correspond to multiple groups of second answer texts. Evaluate the multiple second answer texts in the same query text from five dimensions respectively, and obtain five evaluation information corresponding to each second answer text. Consider and balance the five evaluation information at the same time to achieve the best overall performance of the model. Select the retrieval enhancement system corresponding to the configuration parameters of each processing module when the comprehensive performance of the five dimensions is the best as the target retrieval enhancement system, and use the configuration parameters of each processing module corresponding to the target retrieval enhancement system as the target configuration parameters. By setting multi-objective optimization, intelligently identify and select the configuration scheme that provides the best balance between various objectives.

[0087] It is understandable that by introducing a multi-objective optimization method in the optimization of the retrieval enhancement system, this comprehensive optimization strategy enables the retrieval enhancement system to no longer just make progress in a single dimension, but to achieve a significant improvement in overall performance in all important indicators. By analyzing and adjusting the mutual influence and dependencies between various objectives, the retrieval enhancement system can improve the quality of generated content and overall operating efficiency while maintaining a high degree of relevance and fidelity. It can automatically adjust the configuration of the retrieval enhancement system according to the specific needs of different application scenarios to achieve the best performance balance that comprehensively considers various objectives. The overall optimization of multiple dimensions provides the retrieval enhancement system with more stable and efficient performance in various complex application scenarios.

[0088] Figure 3 The third flowchart of the parameter configuration method according to the embodiment of the present disclosure is schematically shown.

[0089] As described above, in operation S232, based on the M sets of evaluation information, a target retrieval enhancement system is determined from the M modified retrieval enhancement systems. In one possible implementation, Figure 3 As shown, the operation may further include operation S2321a to operation S2323a.

[0090] In operation S2321a, based on M groups of evaluation information, m candidate retrieval enhancement systems whose evaluation information in at least one dimension meets a preset condition are determined from the M changed retrieval enhancement systems, where the candidate retrieval enhancement system represents the changed retrieval enhancement system, and m is an integer greater than or equal to 1 and less than or equal to M.

[0091] In operation S2322a, contribution information of each of the m candidate retrieval enhancement systems in N dimensions is calculated based on the m sets of evaluation information.

[0092] In operation S2323a, a target retrieval enhancement system is determined from the m candidate retrieval enhancement systems according to the m contribution information.

[0093] For example, the contribution information may be the distribution of the evaluation information of each second answer text in different dimensions (e.g., context relevance, fidelity of generated content, relevance of generated content, text generation quality, and computational efficiency). That is, the evaluation information of each second answer text in five dimensions is simultaneously expressed in five dimensions.

[0094] For example, the first second answer text performs well in the context relevance dimension, but performs poorly in the other four dimensions; the second second answer text performs well in all five dimensions, but is not as good as the first second answer text in the first dimension ("context relevance dimension"). The retrieval enhancement system corresponding to the second answer text that performs well in each dimension can be selected as the target retrieval enhancement system.

[0095] Evaluation information of multiple second answer texts in different dimensions for the same query text. Select the retrieval enhancement system corresponding to the second answer text whose evaluation information in one or more dimensions meets the preset conditions as the candidate retrieval enhancement system. Then calculate the contribution information of the second answer text corresponding to the candidate retrieval enhancement system in each dimension to determine the performance of the candidate retrieval enhancement system in each dimension. Thus, the candidate retrieval enhancement system with better performance in each dimension is selected as the target retrieval enhancement system.

[0096] Figure 4 A fourth flowchart of the parameter configuration method according to an embodiment of the present disclosure is schematically shown.

[0097] As described above, in operation S232, based on the M sets of evaluation information, a target retrieval enhancement system is determined from the M modified retrieval enhancement systems. In one possible implementation, Figure 4 As shown, the operation may further include operation S2321b to operation S2323b.

[0098] In operation S2321b, based on the M groups of evaluation information, the M modified retrieval enhancement systems are non-dominatedly sorted to determine m candidate retrieval enhancement systems, where the candidate retrieval enhancement systems represent that the dominated number of the modified retrieval enhancement systems meets a threshold.

[0099] In operation S2322b, based on the m sets of evaluation information, crowding distances between the m candidate retrieval enhancement systems for N dimensions are calculated.

[0100] In operation S2322b, based on the m crowding distances, a candidate retrieval enhancement system having the largest crowding distance is determined as a target retrieval enhancement system.

[0101] Exemplarily, the non-dominated sorting may be to compare the dominance relationships among the M modified retrieval enhancement systems and divide the M modified retrieval enhancement systems into a number of non-dominated levels, wherein the dominated number of each modified retrieval enhancement system is calculated, the M modified retrieval enhancement systems are sorted according to the size of the dominated number, and the modified retrieval enhancement systems whose dominated number exceeds a threshold are selected to obtain m candidate retrieval enhancement systems.

[0102] The crowding distance refers to the degree of uniformity of the evaluation information of m candidate retrieval enhancement systems in N dimensions.

[0103] For parameter tuning in each processing module of the retrieval enhancement system, the requirements of the parameter tuning algorithm may include: a) both continuous parameters in the interval range and categorical parameters (such as the selection of retrieval algorithms); b) relatively complex parameter deconstruction, for example, after a certain retrieval algorithm is determined, its corresponding configuration items will take effect; c) the dimension of the hyperparameter search space is medium to high dimensional (this depends on the process of the retrieval enhancement system); d) the algorithm iteration is stable without big ups and downs; the results are reliable, and different random middle and lower results are similar; e) support for multi-objective evaluation and optimization.

[0104] For the automated hyperparameter tuning algorithm, the following can also be used: 1) learning-based hyperparameter optimization based on reinforcement learning, which uses an intelligent agent in reinforcement learning to replace humans to adjust parameters and conduct trial and error, and models learning rules based on experience; 2) search-based hyperparameter optimization based on evolutionary algorithms, which focuses on how to perform random searches in the search space more efficiently; 3) probabilistic hyperparameter optimization based on Bayesian optimization, which focuses on guiding the selection direction of hyperparameter optimization through probabilistic proxy models; 4) random search-based hyperparameter optimization based on strategy and rule guidance. This type of algorithm guides the search for hyperparameters through specific strategies and rules (such as crowding distance).

[0105] For example, when the model is a large language model, the MO-ASHA (Multi-objective ASHA) algorithm that meets one of the above requirements is used as an example. The multiple processing modules in the retrieval enhancement system may include a pre-retrieval module, a retrieval module, a data source enhancement module, a post-retrieval processing module, a query combination module, and a generation module connected in sequence. The evaluation dimensions include: context relevance, content fidelity, content relevance, text generation quality, and computational efficiency.

[0106] MO-ASHA is an extension of the ASHA algorithm (Asynchronous Successive Halving Algorithm) to adapt it to multi-objective optimization environments. MO-ASHA retains the core features of ASHA, such as asynchronous scheduling and early stopping strategy, and adds key elements to adapt to multi-objective scenarios:

[0107] Global optimization considerations: The MO-ASHA algorithm not only considers the performance of each solution on a single objective, but also comprehensively considers the overall distribution of these solutions on all objectives. Through the mechanisms of non-dominated sorting and crowding distance, MO-ASHA does not rely on building complex probabilistic models (such as Gaussian process regression), but adopts a more direct approach to evaluate and explore hyperparameter configurations. Consider the overall geometry of the current Pareto frontier to more comprehensively explore the trade-offs between various objectives, thereby effectively exploring the solution space.

[0108] In the retrieval enhancement system, MO-ASHA can effectively balance these goals and help discover configurations that perform well in all key dimensions through non-dominated sorting and crowding distance strategies. In addition, its early stopping and asynchronous scheduling mechanisms enable the algorithm to efficiently handle large-scale and complex hyperparameter spaces, meeting the needs of complex retrieval enhancement systems.

[0109] The multi-objective hyperparameter optimization of the retrieval enhancement system using the MO-ASHA algorithm specifically includes: (1) Defining the objective function: When designing the multi-objective hyperparameter optimization of the retrieval enhancement system, the key is to carefully define and balance the various objective functions to reflect the special needs of the retrieval enhancement system. These objectives mainly include the relevance of the context mentioned above, the fidelity of the generated content, the relevance of the generated content, the quality of text generation, and the computational efficiency. Each objective function 𝑔𝑖(x) needs to be evaluated and optimized separately, rather than simply merged into a weighted objective. This is because the overall optimization of the retrieval enhancement system is not just the sum of the optimizations of each individual part, but requires finding a balance between these objectives. For example, improving the relevance of the generated content may affect the computational efficiency, so a trade-off needs to be made between these aspects. In this way, comprehensive optimization of multiple key dimensions can be achieved in the retrieval enhancement system.

[0110] (2) Non-dominated sorting: Non-dominated sorting plays a key role in multi-objective hyperparameter optimization of retrieval enhancement systems.

[0111] Evaluate the performance of each solution: For each configuration or solution in the retrieval enhancement system , evaluate it on multiple objective functions Performance on.

[0112] Identifying Nondominated Solutions: A Solution is considered non-dominated if there are no other solutions In all targets performs better. Mathematically, this can be expressed as: Solution is nondominated if and only if there is no other solution So that for all targets , ≤ Established, and there is at least one target , so that .

[0113] Through multi-objective optimization, solutions that perform well on various key dimensions are effectively identified; this approach considers the comprehensive performance of the solution on multiple objectives rather than relying solely on the performance metric of a single objective.

[0114] Crowding distance evaluation and search guidance: In the MO-ASHA algorithm, the evaluation of the crowding distance plays an important role in the search guidance process. This evaluation process can be described in detail by the following steps and formulas:

[0115] Calculation of crowding distance: For each set of solutions in the non-dominated layer, we calculate the crowding distance between the solutions To evaluate how evenly they are distributed in the target space. The calculation of crowding distance is based on comparing the performance differences of two solutions on each objective function. Specifically, the calculation formula of crowding distance is:

[0116]

[0117] in, Is the solution In the The value of the objective function.

[0118] Search guidance: By calculating the crowding distance between non-dominated solutions, the algorithm evaluates how evenly the solutions are distributed in the target space. A solution with a large crowding distance indicates that it is relatively independent in the target space and is exploring new areas of the space. Therefore, choosing a solution with a large crowding distance helps the algorithm move towards directions that have not been fully explored, thereby improving overall performance while maintaining solution diversity.

[0119] The MO-ASHA algorithm uses non-dominated sorting to identify solutions with excellent performance, while using crowding distance evaluation to maintain the diversity and comprehensiveness of solutions. The combination of crowding distance evaluation and non-dominated sorting can effectively guide the search process towards the direction of overall performance improvement.

[0120] In some embodiments, in operation S232, a target retrieval enhancement system is determined from the M modified retrieval enhancement systems according to the M groups of evaluation information. The operation may further include: analyzing the M groups of evaluation information based on an asynchronous continuous halving algorithm to determine the target retrieval enhancement system from the M modified retrieval enhancement systems.

[0121] Exemplarily, the asynchronous continuous halving algorithm may refer to allocating initial resources to the parameters in the M changed retrieval enhancement systems in the process of respectively generating M groups of evaluation information for the M changed retrieval enhancement systems, and starting training in a parallel computing environment, selecting half of the parameter combinations with worse performance based on the performance indicators of each group of parameters, and halving the resource allocation, and iterating until only one group of parameter combinations remains.

[0122] For example, the above MO-ASHA algorithm is used as an example for explanation.

[0123] Resource allocation strategy: For each solution, initially allocate limited resources As the iteration process progresses, resources grow at a certain factor Gradually increase, the calculation formula is as follows:

[0124]

[0125] in is the resource growth factor, and t is the number of iterations.

[0126] Early stopping mechanism: For those solutions that perform poorly in early iterations, an early stopping strategy is implemented to save resources. This means that only those solutions that perform well in the initial evaluation phase will continue to receive more resources for in-depth evaluation.

[0127] Asynchronous scheduling and iterative updates: By introducing asynchronous scheduling and iterative updates, the algorithm becomes more adaptable to complex search spaces and can more flexibly respond to the performance of different solutions. This helps to improve the efficiency and convergence speed of the algorithm. The specific implementation method is as follows:

[0128] Asynchronous scheduling mechanism: In each round of resource allocation, the algorithm does not need to wait for all configurations to complete their evaluation before proceeding to the next step, but can make adjustments instantly based on the partial information already available. This mechanism allows operations to be performed simultaneously on multiple rounds, thereby improving overall computational efficiency.

[0129] Iterative updates: Through this asynchronous scheduling, the algorithm can respond more flexibly to the performance of different solutions and update its status and resource allocation in a timely manner to converge to an approximate solution on the Pareto frontier more quickly.

[0130] In some embodiments, in operation S2321a, the modified evaluation information of the search enhancement system in at least one dimension satisfies a preset condition including:

[0131] The total evaluation value of the retrieval enhancement system after the change in N dimensions is not lower than that of the retrieval enhancement system before the change, and the evaluation value of the retrieval enhancement system after the change in at least one dimension is higher than that of the retrieval enhancement system before the change.

[0132] For example, first find a retrieval enhancement system (candidate retrieval enhancement system) that performs well in at least one dimension based on the evaluation information. On this basis, calculate the distribution of evaluation information in each dimension (contribution information). The one that performs well in each dimension is determined as the target retrieval enhancement system.

[0133] Among them, the distribution of evaluation information (contribution information) in each dimension can be obtained by referring to the calculation method of the above-mentioned crowding distance, which will not be repeated here.

[0134] In some embodiments, the parameter configuration method further includes:

[0135] Based on query texts of different categories, a plurality of cascaded processing modules in a retrieval enhancement system corresponding to each category of query text and at least one configuration parameter of each processing module are determined, wherein the query texts of different categories represent different application scenarios.

[0136] For example, the processing flow of the retrieval enhancement system can be defined according to different application scenarios. The process of the retrieval enhancement system can be defined by a specific generative AI application combined with actual usage scenarios (and restrictions). In order to ensure that the retrieval enhancement system can achieve optimal performance in different usage scenarios, it is important to perform refined hyperparameter tuning for each step in its process.

[0137] For example, in the case where the model is a large language model, the cascaded processing modules of the search enhancement system may include a pre-search module, a search module, a data source enhancement module, a post-search processing module, a query combination module, and a generation module connected in sequence. Other processing modules may be added on this basis, or existing processing modules may be deleted.

[0138] In other embodiments, the cascaded processing modules of the retrieval enhancement system may further include a pre-retrieval module, a retrieval module, a post-retrieval processing module and a generation module connected in sequence.

[0139] It should be noted that, for different application scenarios, different cascade processing modules in the search enhancement system can be defined according to actual application requirements.

[0140] Figure 5 A schematic diagram of a retrieval enhancement system according to an embodiment of the present disclosure is schematically shown; Figure 6 The processing flow chart of each processing module in the retrieval enhancement system according to the embodiment of the present disclosure is schematically shown.

[0141] In some embodiments, reference Figure 5The multiple processing modules in the retrieval enhancement system 300 include a pre-retrieval module 301, a retrieval module 302, a data source enhancement module 303, a post-retrieval processing module 304, a query combination module 305 and a generation module 306 which are connected in sequence.

[0142] Correspondingly, in the case where the multiple processing modules include a pre-retrieval module 301, a retrieval module 302, a data source enhancement module 303, a post-retrieval processing module 304, a query combination module 305 and a generation module 306 connected in sequence, the operation flow of the retrieval enhancement system is as follows: Figure 6 shown. Figure 6 The main process of the retrieval enhancement framework is schematically shown in Figure 1, where each step covers several tunable configuration items or hyperparameters.

[0143] For example, taking the model as a large language model, in the pre-retrieval stage, hyperparameter adjustment is not limited to query rewriting, but also includes the adjustment of query block size and query enhancement strategy. These adjustments directly affect the effect of subsequent retrieval. In the retrieval operation stage, in addition to selecting a suitable retrieval algorithm, it is also necessary to fine-tune the index type and search depth. In the data source enhancement stage, the integration and weight distribution of external data sources are also precisely controlled by hyperparameter adjustment. In the post-retrieval processing stage, by tuning the filtering parameters and rearrangement strategy, ensure that highly relevant data is provided to the model. In the process of combining information and queries, the adjustment of the information fusion strategy is crucial to the quality of the final generation. In the generation stage, the adjustment of hyperparameters such as decoding strategy and temperature parameters determines the quality and diversity of the generated text.

[0144] As described above, in operation S210, an evaluation data set is obtained, the evaluation data set including at least one query text and a first answer text corresponding to each query text. The first answer text includes a third answer text and associated information, wherein the third answer text represents the answer information corresponding to the query text, and the associated information represents the source information of the answer in the third answer text.

[0145] Exemplarily, an evaluation dataset is constructed for a retrieval enhancement system. The following strategies may be adopted:

[0146] (1) Depth and freshness of knowledge base data

[0147] Diversified and real-time knowledge sources: When building a knowledge base, in addition to traditional static data sources (such as Wikipedia), real-time updated data such as the latest news reports, industry reports, and academic papers should also be included. This ensures that the retrieval enhancement system can handle the latest information and trends.

[0148] Deep information integration: In order to provide in-depth analysis and understanding, the knowledge base should contain complex concepts and relationships. For example, the relationship and attribute information between entities can be provided by integrating knowledge graphs.

[0149] (2) Quality and depth of answer annotations

[0150] Answers and their basis: The answer to each query (first answer text) must not only be accurate, but also provide the basis or source of the answer, which can ensure the transparency and verifiability of the answer.

[0151] Handling of complex questions: For questions that are complex or require in-depth analysis, answer annotations should include detailed explanations and background information, rather than just simple statements of facts.

[0152] The specific steps for constructing the evaluation dataset can be as follows:

[0153] Query question (query text) generation: Use natural language processing tools (e.g., large language models) to generate a range of questions ranging from ground truth to complex questions that require deep analysis. Make sure these questions represent different domains and difficulty levels.

[0154] Data source annotation: For each question, record the source of the answer, for example, a specific news article, Wikipedia page paragraph, or other knowledge base content.

[0155] Manual review and annotation: A professional team will annotate answers to some questions to ensure the accuracy, completeness and relevance of the answers. If necessary, the answers will be corrected or supplemented.

[0156] Data enhancement and context analysis: Use natural language processing tools (e.g., large language models) to enhance data, including semantic equivalence transformation and strengthening of contextual relationships, which helps increase the diversity and coverage of data.

[0157] Data partitioning and normalization: The dataset is partitioned into training, validation, and test sets and stored in a normalized format to facilitate efficient processing by the retrieval enhancement system.

[0158] It should be noted that the above describes the key process and elements for building a broad and representative evaluation dataset for retrieval enhancement tasks. For specific application scenarios, users can use similar processes to build relevant application or field-level evaluation datasets.

[0159] Based on the above parameter configuration method, the present disclosure also provides a parameter configuration device. Figure 7 The device is described in detail.

[0160] Figure 7The structural block diagram of the parameter configuration device according to an embodiment of the present disclosure is schematically shown.

[0161] like Figure 7 As shown, the parameter configuration device 400 of this embodiment includes an acquisition module 410 , a processing module 420 , a determination module 430 and a deployment module 440 .

[0162] The acquisition module 410 is used to acquire an evaluation data set, which includes at least one query text and a first answer text corresponding to each query text. In one embodiment, the acquisition module 410 can be used to perform the operation S210 described above, which will not be described in detail here.

[0163] The processing module 420 is used to respond to a change in at least one configuration parameter of multiple processing modules in the retrieval enhancement system, input the query text into M changed retrieval enhancement systems, and obtain M groups of second answer texts corresponding to the M changed retrieval enhancement systems, each group of second answer texts includes multiple second answer texts corresponding to the query text, and M is an integer greater than or equal to 1. In one embodiment, the processing module 420 can be used to perform the operation S220 described above, which will not be repeated here.

[0164] The determination module 430 is used to determine target configuration parameters for multiple processing modules based on the first answer text and the M groups of second answer texts. In one embodiment, the determination module 430 can be used to perform the operation S230 described above, which will not be described in detail here.

[0165] The deployment module 440 is used to deploy the target configuration parameters to multiple processing modules. In one embodiment, the deployment module 440 can be used to perform the operation S240 described above, which will not be described in detail here.

[0166] According to an embodiment of the present disclosure, any multiple modules among the acquisition module 410, the processing module 420, the determination module 430 and the deployment module 440 can be combined into one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present disclosure, at least one of the acquisition module 410, the processing module 420, the determination module 430 and the deployment module 440 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation modes of software, hardware and firmware or in a proper combination of any of them. Alternatively, at least one of the acquisition module 410, the processing module 420, the determination module 430 and the deployment module 440 can be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding function can be performed.

[0167] Figure 8 A block diagram of an electronic device suitable for implementing a parameter configuration method according to an embodiment of the present disclosure is schematically shown.

[0168] like Figure 8 As shown, the electronic device 500 according to an embodiment of the present disclosure includes a processor 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage part 508 to a random access memory (RAM) 503. The processor 501 may include, for example, a general-purpose microprocessor (such as a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (for example, an application-specific integrated circuit (ASIC)), etc. The processor 501 may also include an onboard memory for caching purposes. The processor 501 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0169] In RAM503, various programs and data required for the operation of electronic device 500 are stored. Processor 501, ROM502 and RAM503 are connected to each other via bus 504. Processor 501 performs various operations of the method flow according to the embodiment of the present disclosure by executing the program in ROM502 and / or RAM503. It should be noted that the program can also be stored in one or more memories other than ROM502 and RAM503. Processor 501 can also perform various operations of the method flow according to the embodiment of the present disclosure by executing the program stored in one or more memories.

[0170] According to an embodiment of the present disclosure, the electronic device 500 may further include an input / output (I / O) interface 505, which is also connected to the bus 504. The electronic device 500 may further include one or more of the following components connected to the I / O interface 505: an input portion 506 including a keyboard, a mouse, etc.; an output portion 507 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 508 including a hard disk, etc.; and a communication portion 509 including a network interface card such as a LAN card, a modem, etc. The communication portion 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the I / O interface 505 as needed. A removable medium 511, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 510 as needed, so that a computer program read therefrom is installed into the storage portion 508 as needed.

[0171] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present disclosure is implemented.

[0172] According to an embodiment of the present disclosure, a computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device. For example, according to an embodiment of the present disclosure, a computer-readable storage medium may include the ROM502 and / or RAM503 described above and / or one or more memories other than ROM502 and RAM503.

[0173] The embodiment of the present disclosure also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the parameter configuration method provided by the embodiment of the present disclosure.

[0174] The above functions defined in the system / device of the embodiment of the present disclosure are performed when the computer program is executed by the processor 501. According to the embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0175] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 509, and / or installed from the removable medium 511. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0176] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 509, and / or installed from the removable medium 511. When the computer program is executed by the processor 501, the above functions defined in the system of the embodiment of the present disclosure are performed. According to the embodiment of the present disclosure, the system, device, means, module, unit, etc. described above can be implemented by a computer program module.

[0177] According to an embodiment of the present disclosure, the program code for executing the computer program provided by the embodiment of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect through the Internet).

[0178] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0179] It will be appreciated by those skilled in the art that the features described in the various embodiments and / or claims of the present disclosure may be combined and / or combined in a variety of ways, even if such combinations and / or combinations are not explicitly described in the present disclosure. In particular, the features described in the various embodiments and / or claims of the present disclosure may be combined and / or combined in a variety of ways without departing from the spirit and teachings of the present disclosure. All of these combinations and / or combinations fall within the scope of the present disclosure.

[0180] The embodiments of the present disclosure are described above. However, these embodiments are only for illustrative purposes and are not intended to limit the scope of the present disclosure. Although the embodiments are described above separately, this does not mean that the measures in the various embodiments cannot be used in combination to advantage. The scope of the present disclosure is defined by the attached claims and their equivalents. Without departing from the scope of the present disclosure, those skilled in the art may make a variety of substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. A parameter configuration method, comprising: Acquire an evaluation data set, the evaluation data set comprising at least one query text and a first answer text corresponding to each query text; In response to a change in at least one configuration parameter of a plurality of processing modules in the retrieval enhancement system, the query text is input into M modified retrieval enhancement systems to obtain M groups of second answer texts corresponding to the M modified retrieval enhancement systems, each group of the second answer texts including a plurality of second answer texts corresponding to the query text, where M is an integer greater than or equal to 1; Determining target configuration parameters for the plurality of processing modules based on the first answer text and the M groups of second answer texts; The target configuration parameters are deployed to the plurality of processing modules.

2. The method according to claim 1, wherein determining target configuration parameters for the plurality of processing modules based on the first answer text and the M groups of second answer texts comprises: Based on the first answer text and the M groups of second answer texts, determine N evaluation information for each group of the second answer texts from N dimensions to obtain M groups of evaluation information, where N is an integer greater than or equal to 1; Determining a target retrieval enhancement system from the M modified retrieval enhancement systems according to the M groups of evaluation information; The configuration parameters of the multiple processing modules in the target retrieval enhancement system are used as target configuration parameters.

3. The method according to claim 2, wherein determining a target retrieval enhancement system from the M modified retrieval enhancement systems according to the M groups of evaluation information comprises: According to the M groups of evaluation information, determine m candidate retrieval enhancement systems whose evaluation information in at least one dimension meets a preset condition from the M modified retrieval enhancement systems, wherein the candidate retrieval enhancement systems represent the modified retrieval enhancement system, and m is an integer greater than or equal to 1 and less than or equal to M; Calculate contribution information of each of the m candidate retrieval enhancement systems in the N dimensions according to the m groups of evaluation information; According to the m pieces of contribution information, the target retrieval enhancement system is determined from the m candidate retrieval enhancement systems.

4. The method according to claim 2 or 3, wherein determining a target retrieval enhancement system from the M modified retrieval enhancement systems according to the M groups of evaluation information comprises: According to the M groups of evaluation information, performing non-dominated sorting on the M modified retrieval enhancement systems to determine m candidate retrieval enhancement systems, wherein the candidate retrieval enhancement systems represent that the number of dominated members of the modified retrieval enhancement systems meets a threshold; Calculating, according to the m groups of evaluation information, crowding distances between the m candidate retrieval enhancement systems for the N dimensions; According to the m crowding distances, the candidate retrieval enhancement system with the largest crowding distance is determined as the target retrieval enhancement system.

5. The method according to claim 1 or 4, wherein determining a target retrieval enhancement system from the M modified retrieval enhancement systems according to the M groups of evaluation information comprises: Based on the asynchronous continuous halving algorithm, the M groups of evaluation information are analyzed, and a target retrieval enhancement system is determined from the M modified retrieval enhancement systems.

6. According to the method of claim 4, the evaluation information of the modified search enhancement system in at least one dimension meets the preset condition, including: The total evaluation value of the modified retrieval enhancement system in N dimensions is not lower than that of the retrieval enhancement system before the modification, and the evaluation value of the modified retrieval enhancement system in at least one dimension is higher than that of the retrieval enhancement system before the modification.

7. The method according to claim 1, further comprising: Based on the query texts of different categories, a plurality of cascaded processing modules in the retrieval enhancement system corresponding to the query texts of each category and at least one configuration parameter of each processing module are determined, wherein the query texts of different categories represent different application scenarios.

8. The method according to claim 7, wherein the plurality of processing modules include a pre-retrieval module, a retrieval module, a data source enhancement module, a post-retrieval processing module, a query combination module and a generation module which are connected in sequence.

9. The method according to claim 1, wherein the first answer text includes a third answer text and associated information, wherein: The third answer text represents answer information corresponding to the query text, and the associated information represents source information of the answer in the third answer text.

10. A parameter configuration device, comprising: An acquisition module, configured to acquire an evaluation data set, wherein the evaluation data set includes at least one query text and a first answer text corresponding to each query text; a processing module, configured to input the query text into M modified retrieval enhancement systems in response to a change in at least one configuration parameter of the plurality of processing modules in the retrieval enhancement system, and obtain M groups of second answer texts corresponding to the M modified retrieval enhancement systems, each group of the second answer texts including a plurality of second answer texts corresponding to the query text, where M is an integer greater than or equal to 1; A determination module, configured to determine target configuration parameters for the plurality of processing modules based on the first answer text and the M groups of second answer texts; A deployment module deploys the target configuration parameters to the multiple processing modules.

Images