A rag system configuration method based on multi-objective bayesian optimization

Abstract

This invention provides a configuration method for RAG systems based on multi-objective Bayesian optimization, belonging to the technical field of RAG systems. This invention obtains an initial observation dataset by randomly sampling initial configurations from a structured search space using a multi-objective evaluation function. For three performance indicators, independent Gaussian process models are established to fit the mean and covariance functions of the objective function. Based on the current Pareto front and the posterior prediction distribution of the Gaussian process model, the expected hypervolume improvement acquisition function is calculated, and the configuration with the largest improvement value is selected for iterative evaluation and updating. Finally, non-dominated solutions are identified to form the Pareto optimal configuration set. This invention solves the technical problem of complex configuration parameter space and conflicts between multiple performance indicators in retrieval-enhanced generation systems, which makes it difficult to automatically obtain the Pareto optimal configuration set.

Description

Technical Field

[0001] This invention belongs to the field of RAG system technology, and more specifically, relates to a RAG system configuration method based on multi-objective Bayesian optimization. Background Technology

[0002] Retrieval-enhanced generation systems achieve real-time knowledge updates by combining external knowledge base retrieval with large language model generation, and are widely used in scenarios such as intelligent question answering, document analysis, and knowledge management. Traditional configuration methods employ grid search or random search to traverse the parameter combinations of components such as text segmentation strategies, vector embedding models, retrieval parameters, and generation models, and evaluate and select based on a single performance metric. In existing technologies, because retrieval-enhanced generation systems involve a multi-dimensional hyperparameter space of text segmentation modules, vector embedding modules, vector retrieval modules, result re-ranking modules, and text generation modules, and because there are inherent conflicts between performance metrics such as fidelity, answer relevance, and contextual recall, traditional methods require a large number of evaluations to explore the parameter space, and can only obtain suboptimal solutions for a single metric, unable to simultaneously optimize multiple conflicting objectives. In other words, existing technologies suffer from the technical problem of complex configuration parameter spaces and conflicts between multiple performance metrics, making it difficult to automatically obtain Pareto optimal configuration sets. Summary of the Invention

[0003] In view of this, the present invention provides a RAG system configuration method based on multi-objective Bayesian optimization, which can solve the technical problems in the prior art where the configuration parameter space of the retrieval enhancement generation system is complex and conflicts exist between multiple performance indicators, making it difficult to automatically obtain the Pareto optimal configuration set.

[0004] This invention is implemented as follows: This invention provides a RAG system configuration method based on multi-objective Bayesian optimization, comprising the following steps: Constructing a structured search space, defining the component selection and parameter settings of the text segmentation module, vector embedding module, vector retrieval module, result reordering module, and text generation module of the retrieval enhancement generation system pipeline as the structured search space, wherein the structured search space includes categorical hyperparameters, integer hyperparameters, and conditional hyperparameters; Establishing a multi-objective evaluation function, wherein the multi-objective evaluation function receives configuration vectors sampled from the structured search space as input and outputs performance index vectors including loyalty, answer relevance, and contextual recall; Initializing the observation dataset, randomly sampling a preset number of initial configurations from the structured search space and executing the multi-objective evaluation function on each initial configuration to obtain the corresponding performance index vector, and constructing the initial observation dataset by pairing the configuration vectors with the performance index vectors; Constructing a probabilistic proxy model, targeting loyalty, answer relevance, and contextual recall... Recall is modeled using independent Gaussian processes, each model fitting the mean and covariance of the objective function to the initial observation dataset. The expected hypervolume improvement acquisition function is calculated, and the expected hypervolume improvement value for each candidate configuration in the structured search space is calculated based on the current Pareto front and the posterior prediction distribution of the Gaussian process model. The configuration with the largest expected hypervolume improvement value is selected as the next evaluation configuration. Iterative optimization and updating are performed, executing a multi-objective evaluation function on the next evaluation configuration to obtain a new performance index vector. The data pairs formed by the configuration vector and the new performance index vector are added to the initial observation dataset to form an updated observation dataset, and the posterior distribution of all Gaussian process models is updated. The expected hypervolume improvement acquisition function and iterative optimization and updating steps are repeated until a preset number of iterations is reached. A Pareto optimal configuration set is output, by traversing all evaluated configurations in the updated observation dataset, identifying non-dominated solutions to form the Pareto optimal configuration set, and selecting deployment configurations from the Pareto optimal configuration set according to business requirements.

[0005] Specifically, the construction of the structured search space involves formally defining all variable components and parameters of the retrieval enhancement generation system pipeline as a configuration vector, which contains multiple hyperparameter components.

[0006] The categorical hyperparameters include text segmentation strategy, vector embedding model selection, and large language model selection.

[0007] The integer hyperparameters include text block size, number of documents returned by retrieval, and temperature parameter.

[0008] The conditional hyperparameters include text block overlap size, reordering model selection, and reordering return quantity. These conditional hyperparameters only take effect when the activation condition is met.

[0009] Specifically, the establishment of the multi-objective evaluation function involves dynamically instantiating the retrieval enhancement generation system pipeline based on the configuration vector, preprocessing and indexing the knowledge base documents, traversing the evaluation question set, executing the retrieval enhancement generation query for each evaluation question, and recording the generated answer and retrieval context.

[0010] The fidelity calculation uses an answer support function to determine whether the generated answer is entirely based on the retrieval context.

[0011] The calculation of answer relevance uses a relevance scoring function to measure the degree of relevance between the generated answer and the user's question.

[0012] The context recall rate is calculated by dividing the number of intersection elements between the retrieval context and the real context by the number of real context elements.

[0013] Specifically, the Gaussian process model is constructed by independently establishing Gaussian processes for fidelity, answer relevance, and contextual recall, with the objective function following a Gaussian distribution of mean and covariance functions.

[0014] The covariance function uses the Matern 5 / 2 kernel function to measure the similarity between input points and infer the correlation of the objective function value.

[0015] Specifically, the calculation of the expected hypervolume improvement acquisition function is based on the current Pareto front, which calculates the expected increment of the Pareto front hypervolume after adding the prediction results of the candidate configuration.

[0016] The hypervolume index measures the volume of the multidimensional space enclosed by the solution set and the preset reference point.

[0017] The next evaluation configuration is obtained by solving for the maximum value of the desired hypervolume improvement acquisition function in the structured search space.

[0018] In the iterative optimization and update step, the posterior distribution of the Gaussian process model is updated using the data consisting of the configuration vector and the new performance index vector.

[0019] The non-dominated solution refers to a configuration in the Pareto optimal configuration set that has no other configuration that is not inferior to the non-dominated solution in all performance metrics and is superior to the non-dominated solution in at least one performance metric.

[0020] This invention employs multi-objective Bayesian optimization to construct a structured search space and establishes objective functions for fitting fidelity, answer relevance, and contextual recall of independent Gaussian process models. By using the expected hypervolume improvement acquisition function, it calculates the expected contribution of candidate configurations to the Pareto front on the posterior prediction distribution and selects the configuration with the largest improvement value for iterative evaluation. This solves the problems of low evaluation efficiency and the inability to obtain only single-objective suboptimal solutions in traditional methods. Furthermore, this invention unifies categorical, integer, and conditional hyperparameters into configuration vectors. For each performance index, it independently establishes a Gaussian process model to learn the mean and covariance functions of the objective function. It utilizes the expected hypervolume improvement acquisition function to balance the exploration of unknown regions with the utilization of known high-performance regions, automatically discovering the Pareto front composed of non-dominated solutions within a limited number of evaluations. In summary, this invention solves the technical problems mentioned in the background art, such as the complex configuration parameter space of retrieval enhancement generation systems and the conflict between multiple performance indices, which makes it difficult to automatically obtain the Pareto optimal configuration set. Attached Figure Description

[0021] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below.

[0023] like Figure 1 The diagram shown is a flowchart of a RAG system configuration method based on multi-objective Bayesian optimization provided by the present invention. This method includes the following steps:

[0024] S01. Construct a structured search space. Define the component selection and parameter settings of the text segmentation module, vector embedding module, vector retrieval module, result reordering module, and text generation module of the retrieval enhancement generation system pipeline as a structured search space. The structured search space includes categorical hyperparameters, integer hyperparameters, and conditional hyperparameters.

[0025] S02. Establish a multi-objective evaluation function, which receives a configuration vector sampled from the structured search space as input and outputs a performance index vector containing fidelity, answer relevance and context recall.

[0026] S03. Initialize the observation dataset. Randomly sample a preset number of initial configurations from the structured search space and execute a multi-objective evaluation function on each initial configuration to obtain the corresponding performance index vector. Combine the data pairs of configuration vectors and performance index vectors to form the initial observation dataset.

[0027] S04. Construct a probabilistic proxy model, and establish independent Gaussian process models for fidelity, answer relevance and contextual recall respectively. The Gaussian process model is based on the mean function and covariance function of the objective function fitted to the initial observation dataset.

[0028] S05. Calculate the expected hypervolume improvement acquisition function. Based on the posterior prediction distribution of the current Pareto front and Gaussian process model, calculate the expected hypervolume improvement value of each candidate configuration in the structured search space, and select the configuration with the largest expected hypervolume improvement value as the next evaluation configuration.

[0029] S06. Iterative optimization and update: Execute the multi-objective evaluation function on the next evaluation configuration to obtain a new performance index vector. Add the data pair consisting of the configuration vector and the new performance index vector to the initial observation dataset to form an updated observation dataset and update the posterior distribution of all Gaussian process models. Repeat steps S05 to S06 until the preset number of iterations is reached.

[0030] S07. Output the Pareto optimal configuration set. Traverse and update all evaluated configurations in the observation dataset to identify non-dominated solutions to form the Pareto optimal configuration set. Select deployment configurations from the Pareto optimal configuration set according to business requirements.

[0031] The structured search space is constructed by formally defining all variable components and parameters of the retrieval enhancement generation system pipeline as configuration vectors, wherein the configuration vectors contain... There are 10 hyperparameter components, of which represents the total dimension of hyperparameters. Categorical hyperparameters include text segmentation strategy, vector embedding model selection, and large language model selection. Possible values ​​for the text segmentation strategy include fixed-length segmentation and recursive character segmentation. Possible values ​​for the vector embedding model selection include the bge-base-en-v1.5 model, the m3e-base model, and the text-embedding-3-small model. Possible values ​​for the large language model selection include the DeepSeek-V3.1 model, the Kimi-k2 model, and the Qwen3-235B-A22B-2507 model.

[0032] Integer hyperparameters include text block size, number of retrieved documents, and temperature parameter. The text block size ranges from 256 to 1024 with a step size of 64. The number of retrieved documents ranges from 1 to 10. The temperature parameter ranges from 0.0 to 1.0. Conditional hyperparameters include text block overlap size, re-ranking model selection, and re-ranking return count. The text block overlap size ranges from 0 to 200 with a step size of 10 and is activated only when the text segmentation strategy is recursive character segmentation. The re-ranking model selection options include the bge-reranker-base model and the cohere-rerank-english-v2.0 model and are activated only when re-ranking is enabled. The re-ranking return count ranges from 1 to 5 and is activated only when re-ranking is enabled, and the value of the re-ranking return count is not greater than the value of the number of retrieved documents.

[0033] The multi-objective evaluation function is constructed by dynamically instantiating the retrieval enhancement generation system pipeline based on the configuration vector, preprocessing and indexing the knowledge base documents, traversing the evaluation question set, executing the retrieval enhancement generation query for each evaluation question, and recording the generated answer and retrieval context. The average fidelity, average answer relevance, and average context recall of all evaluation questions are calculated as the three components of the performance metric vector. The fidelity calculation formula is as follows: fidelity equals the total number of evaluation questions. The inverse of the product is summed by the answer support function of all evaluated questions. This answer support function determines whether the generated answer is entirely based on the retrieval context and returns 1 or 0. The fidelity calculation formula is... .

[0034] The formula for calculating answer relevance is as follows: Answer relevance equals the total number of questions evaluated. The reciprocal of the product is multiplied by the sum of all the relevance scoring functions for the evaluation questions. This relevance scoring function measures the relevance of the generated answer to the user's question and returns a score from 1 to 5. The formula for calculating answer relevance is: The formula for calculating contextual recall is as follows: Contextual recall equals the total number of evaluated questions. The reciprocal of the product of the number of elements in the intersection of the retrieval context and the true context of all evaluation questions, divided by the sum of the number of true context elements, gives the context recall formula: .

[0035] The Gaussian process model is constructed by independently establishing Gaussian processes for fidelity, answer relevance, and contextual recall. The next iteration is given an updated observation dataset. objective function The Gaussian process model follows the mean function Sum of covariance functions The Gaussian distribution and the Gaussian process model are expressed as follows: Mean function Represents the objective function Prior estimates of the covariance function. The Matern 5 / 2 kernel function is used to measure the similarity between input points and infer the correlation of the objective function values. After each iteration, the posterior distribution of the Gaussian process model is updated using the data consisting of the configuration vector and the new performance index vector, thereby obtaining a more accurate estimate of the objective function and quantifying the uncertainty.

[0036] The expected hypervolume improvement acquisition function is calculated using the current Pareto front. Based on the prediction results after incorporating candidate configurations, the expected increment of the Pareto front hypervolume is calculated. The hypervolume metric measures the volume of the multidimensional space enclosed by the solution set and a preset reference point; a larger multidimensional space volume indicates better solution set performance. The formula for calculating the expected hypervolume improvement is as follows: The expected hypervolume improvement value is equal to the improvement in the target space. The above is the current Pareto frontier The supervolume of the union of the predicted point set minus the current Pareto front The hypervolume and the joint target vector at the candidate configuration posterior prediction distribution Integrating the product of the products, the expected improvement in supervolume is calculated using the following formula: The next evaluation configuration is obtained by solving for the maximum value of the desired hypervolume improvement acquisition function in the structured search space. The next evaluation configuration is expressed as follows: .

[0037] Here, a non-dominated solution is defined as a configuration in the Pareto optimal configuration set that has no other configuration inferior to the non-dominated solution in all performance metrics and superior to the non-dominated solution in at least one performance metric. The retrieval-enhanced generation system pipeline is a technical framework that achieves real-time knowledge updates by combining the retrieval capabilities of an external knowledge base with the generation capabilities of a large language model. The text segmentation module segments knowledge base documents into text blocks suitable for vectorization. The vector embedding module converts text blocks into high-dimensional vector representations. The vector retrieval module retrieves similar text blocks from a vector database based on the query vector. The result re-ranking module performs a secondary ranking of the retrieval results to improve relevance. The text generation module generates the final answer based on the retrieval context.

[0038] The current Pareto front is the set of performance index vectors for all non-dominated solutions in the evaluated configuration set. The posterior prediction distribution is the probability distribution prediction of the objective function value for the unevaluated configuration based on observed data from the Gaussian process model. The preset number of iterations is the maximum number of executions of the Bayesian optimization loop, set according to computational resources and time constraints. Business requirements are the user's performance trade-off preferences between loyalty, answer relevance, and contextual recall. The deployment configuration is the retrieval enhancement generation system pipeline configuration selected from the Pareto optimal configuration set for practical application.

[0039] Optionally, the present invention also provides a computer-based method for forming a retrieval enhancement generation system configuration system based on multi-objective Bayesian optimization, wherein the computer is provided with a readable storage medium storing program instructions, and the program instructions execute the above-described method when running in the computer.

[0040] The specific implementation methods of the above steps are described in detail below.

[0041] The specific implementation of step S01 is as follows: First, determine all components and parameters that need to be optimized in the retrieval enhancement generation system pipeline. Define the text segmentation strategy as a categorical hyperparameter and set two optional values: fixed-length segmentation and recursive character segmentation. Define the text block size as an integer hyperparameter and set its value range to 256 to 1024 with a step size of 64. Define the text block overlap size as a conditional hyperparameter and set its value range to 0 to 200 with a step size of 10, and activate it only when the text segmentation strategy is recursive character segmentation. Define the vector embedding model selection as a categorical hyperparameter and set three optional values: bge-base-en-v1.5 model, m3e-base model, and text-embedding-3-small model. Define the number of retrieved documents as an integer hyperparameter and set its value range to 1 to 10. Define whether to enable reordering as a categorical hyperparameter and set two optional values: true and false. Define the reordering model selection as a conditional hyperparameter. The system sets two optional values ​​for the bge-reranker-base model and the cohere-rerank-english-v2.0 model, activating them only when re-ranking is enabled. The number of re-ranked documents returned is defined as a conditional hyperparameter, with a value range of 1 to 5, activating only when re-ranking is enabled and constrained to not exceed the number of retrieved documents. The large language model selection is defined as a categorical hyperparameter, with three optional values: DeepSeek-V3.1, Kimi-k2, and Qwen3-235B-A22B-2507. The temperature parameter is defined as a continuous hyperparameter, with a value range of 0.0 to 1.0. The structured search space is constructed to unify the discrete component selection and continuous parameter adjustment of the retrieval enhancement generation system pipeline into a computable vector form, thereby supporting subsequent mathematical optimization algorithms. The design principle of the conditional hyperparameter is to reflect the logical dependencies between components and avoid generating invalid configuration combinations.

[0042] The specific implementation of step S02 is as follows: First, the retrieval enhancement generation system pipeline is instantiated based on the input configuration vector. The knowledge base document set is then segmented into blocks according to the text segmentation strategy and text block size in the configuration vector. When the text segmentation strategy is recursive character segmentation, the overlapping area of ​​adjacent text blocks is set according to the text block overlap size. All text blocks are vectorized and encoded according to the vector embedding model selected in the configuration vector and stored in the vector database. Each evaluation question is sequentially extracted from the evaluation question set and vectorized using the vector embedding model to generate a query vector. The query vector is then used to retrieve the most similar document in the vector database. The top-ranked retrieving document returns several text blocks. When re-ranking is enabled in the configuration vector, the re-ranking model is used to re-evaluate the relevance of the retrieved text blocks and selects the highest-scoring top-ranked retrieving text blocks as the retrieval context. The retrieval context and evaluation questions are combined into prompt text and input into the large language model selection in the configuration vector. The generation randomness of the large language model is set according to the temperature parameter, and answers are generated. For each evaluation question, the answer support function is used to determine whether each statement in the generated answer is completely derived from the retrieval context. The sum of the return values ​​of the answer support functions for all evaluation questions is divided by the total number of evaluation questions. To obtain fidelity, for each evaluation question, the generated answer is used to calculate the semantic relevance between the generated answer and the evaluation question using a relevance scoring function, returning a score from 1 to 5. The sum of the relevance scoring function returns for all evaluation questions is then divided by the total number of evaluation questions. To obtain the relevance of the answers, for each evaluation question, calculate the number of intersection elements between the retrieval context and the true context, divide by the number of true context elements, sum the ratios of all evaluation questions, and divide by the total number of evaluation questions. The context recall rate is obtained, and the fidelity, answer relevance and context recall rate are combined into a performance index vector as the output of the multi-objective evaluation function. The role of the multi-objective evaluation function is to map the abstract configuration vector into a quantifiable performance index, thereby supporting the comparison of configuration quality. The answer support function is usually implemented using a natural language inference model, and the relevance scoring function is usually implemented using a cross-encoder model or a large language evaluation model.

[0043] The specific implementation of step S03 is as follows: Ten initial configurations are set. Ten initial configurations are generated from the structured search space using the Latin hypercube sampling method to ensure the uniform distribution of the initial configurations in the search space. A multi-objective evaluation function is sequentially executed for each initial configuration to obtain the corresponding performance index vector. The configuration vector of each initial configuration is paired with its corresponding performance index vector to form a data pair, and all data pairs are collected to form an initial observation dataset. The purpose of this initial observation dataset is to provide training data for the subsequent Gaussian process model, thereby establishing a mapping relationship between the configuration vector and the performance index. The principle of the Latin hypercube sampling method is to divide the range of each hyperparameter into equal parts and then randomly sample once within each sub-interval to maximize spatial coverage.

[0044] The specific implementation of step S04 involves extracting all configuration vectors and their corresponding fidelity values ​​from the initial observation dataset to construct a fidelity training set. This fidelity training set is then used to train the first Gaussian process model and fit the mean function of the fidelity values. Sum of covariance functions A second Gaussian process model is trained using the answer relevance training set. This model is then used to extract all configuration vectors and their corresponding answer relevance values ​​from the initial observation dataset and to fit the mean function of the answer relevance. Sum of covariance functions A context recall training set is constructed by extracting all configuration vectors and their corresponding context recall values ​​from the initial observation dataset. This training set is then used to train a third Gaussian process model and fit the mean function of the context recall. Sum of covariance functions The covariance function uniformly adopts the Matern 5 / 2 kernel function and optimizes the length scale parameter and signal variance parameter of the kernel function by maximizing the marginal likelihood function. The Gaussian process model is used to make probabilistic predictions of the performance indicators of the unevaluated configuration and quantify the uncertainty of the prediction. The advantage of the Matern 5 / 2 kernel function is that it makes moderate assumptions about the smoothness of the function and has high computational efficiency.

[0045] The specific implementation of step S05 is to extract the performance index vectors of all non-dominated solutions from the initial observation dataset or the updated observation dataset to form the current Pareto front. The reference point is set as the minimum value of all performance metrics minus a fixed offset of 0.1. For each candidate configuration in the structured search space, three Gaussian process models are used to predict the posterior distributions of loyalty, answer relevance, and contextual recall, respectively. Based on the posterior distributions, multiple sets of performance metric vector samples are generated using a Monte Carlo sampling method. For each set of samples, the sample is added to the current Pareto front. The supervolume after that and the current Pareto front The difference in hypervolume is used to calculate the expected hypervolume improvement value of the candidate configuration by averaging the hypervolume increments of all samples. All candidate configurations in the structured search space are traversed and their expected hypervolume improvement values ​​are compared. The candidate configuration with the largest expected hypervolume improvement value is selected as the next evaluation configuration. The expected hypervolume improvement sampling function is used to balance the utilization in the high-performance region and the exploration in the high-uncertainty region, thereby guiding the optimization algorithm to search efficiently. The principle of the Monte Carlo sampling method is to approximate the calculation of complex integrals through random sampling. The physical meaning of maximizing the expected hypervolume improvement value is to select the configuration most likely to improve the current Pareto front.

[0046] The specific implementation of step S06 involves executing a multi-objective evaluation function on the next evaluation configuration and recording the computation time and resources consumed during the execution process. A new performance index vector corresponding to the next evaluation configuration is obtained. The configuration vector of the next evaluation configuration and the new performance index vector are combined to form a data pair and added to the initial observation dataset to form an updated observation dataset. The updated observation dataset is used to retrain the three Gaussian process models and update the parameters of the mean function and covariance function. It is then determined whether the current iteration count has reached the preset iteration count. If the current iteration count has not reached the preset iteration count, the process returns to step S05 to continue execution. If the current iteration count has reached the preset iteration count, the iteration is terminated and the process proceeds to step S07. The preset iteration count is typically set to 100 times. The purpose of the iterative optimization update is to improve the prediction accuracy of the Gaussian process model and gradually approach the global optimal configuration by continuously accumulating observation data. The core advantage of the Bayesian optimization is that a high-quality solution can be found with only a small number of evaluations.

[0047] The specific implementation of step S07 involves extracting all configuration vectors and their corresponding performance metric vectors from the updated observation dataset. For each configuration vector in the updated observation dataset, it is determined whether there are other configuration vectors that are not inferior to the configuration vector in all three performance metrics (fidelity, answer relevance, and context recall) and are strictly superior to the configuration vector in at least one performance metric. Configuration vectors that do not meet the conditions are identified as non-dominated solutions, and all non-dominated solutions are collected to form a Pareto optimal configuration set. The normalized scores of each configuration in the Pareto optimal configuration set are calculated in terms of fidelity, answer relevance, and context recall. The weight coefficients of the three performance metrics are set according to business requirements, and the weighted comprehensive score of each configuration is calculated. The configuration with the highest score is selected as the recommended deployment configuration. The purpose of the Pareto optimal configuration set is to provide users with multiple optimal choices representing different performance trade-offs to adapt to the needs of different application scenarios.

[0048] It should be noted that the key technical approach of this invention includes unifying the modeling of all components and parameters of the retrieval enhancement generation system pipeline into a structured search space, employing a multi-objective Bayesian optimization algorithm to achieve automated configuration search, and using probabilistic prediction based on a Gaussian process model to guide sampling decisions. Compared to traditional grid search and random search methods, this invention constructs a probabilistic surrogate model using a Bayesian optimization algorithm to predict the performance of unevaluated configurations. It utilizes the expected hypervolume to improve the acquisition function, intelligently balancing regions with high prediction performance and regions with high uncertainty. This allows for exploring a larger search space and discovering better-performing configuration combinations within the same evaluation budget, avoiding the excessively high computational cost caused by combinatorial explosion in grid search and the inefficiency caused by lack of guidance in random search. This invention employs a multi-objective optimization framework to simultaneously optimize three conflicting performance metrics: fidelity, answer relevance, and contextual recall. It outputs a Pareto-optimal configuration set, providing users with multiple performance trade-offs. Compared to traditional methods that focus only on a single performance metric or transform multiple objectives into a single objective through weighted averaging, this invention can more comprehensively evaluate configuration quality and meet the differentiated performance requirements of different application scenarios.

[0049] The synergistic effect of the aforementioned key technological approaches is reflected in the following aspects: the modeling of the structured search space unifies the heterogeneous parameters of the retrieval enhancement generation system pipeline into an optimizable mathematical object; the multi-objective evaluation function quantifies configuration quality into a comparable performance vector; the Gaussian process model makes probabilistic predictions of unknown configurations based on existing evaluation data; the expected hypervolume improvement acquisition function intelligently selects the next evaluation point using the prediction results; the iterative optimization process continuously accumulates data and improves model accuracy; and the final output Pareto optimal configuration set integrates the advantages of global search and data adaptation. Compared to traditional manual parameter tuning methods that rely on domain expert experience and are difficult to systematically explore the combinatorial space, this invention achieves adaptive optimization of dataset characteristics through an algorithm-driven automated process, significantly reducing the time and labor costs of configuration optimization and improving the performance of the retrieval enhancement generation system on datasets from different domains.

[0050] It should be noted that this invention also solves the following technical problem: traditional retrieval enhancement generation system configuration methods struggle to handle incomplete search space definitions caused by conditional hyperparameter dependencies. This invention introduces conditional hyperparameters into the structured search space and defines activation conditions. It sets the text block overlap size to activate only when the text segmentation strategy is recursive character segmentation, and sets the reordering model selection and reordering return quantity to activate only when reordering is enabled. Furthermore, it constrains the reordering return quantity to not exceed the number of retrieved documents, ensuring that the configuration vector can fully express the dependency constraints between components. When fitting the objective function related to the conditional hyperparameters, the Gaussian process model only uses observation data that satisfies the activation conditions for posterior updates, avoiding interference from invalid configurations on the surrogate model's prediction accuracy and ensuring that the desired hypervolume improvement acquisition function is optimized within the effective subspace of the structured search space.

[0051] Specifically, the principle of this invention is as follows: This invention solves the technical problem of the complex configuration parameter space and conflicts among multiple performance indicators in retrieval enhancement generation systems, making it difficult to automatically obtain the Pareto optimal configuration set. The principle lies in multi-objective Bayesian optimization, which models the uncertainty of the objective function through a probabilistic surrogate model and uses a collection function to guide the search process. The Gaussian process model makes probabilistic predictions of the performance of unevaluated configurations based on observed data. Its mean function characterizes the expected value of the objective function, while the covariance function quantifies the uncertainty of the prediction, enabling the optimization process to adaptively balance between exploring high-uncertainty regions and utilizing known high-performance regions. The expected hypervolume improves the collection function by calculating the expected increment of the Pareto front hypervolume after adding candidate configurations, transforming the multi-objective optimization problem into a single-objective collection function maximization problem, automatically balancing the optimization directions of multiple conflicting objectives. During the iterative update process, new evaluation data continuously corrects the posterior distribution of the Gaussian process model, causing the surrogate model to gradually approach the true objective function and guide the search to focus on the Pareto optimal region, consistent with the theoretical logic of Bayesian sequential decision-making.

[0052] The following provides a specific embodiment 1 of the present invention, and the specific implementation of each step in this embodiment 1 is described in detail below.

[0053] The specific implementation of step S01 is as follows: First, determine all components and parameters that need to be optimized in the retrieval enhancement generation system pipeline. Define the text segmentation strategy as a categorical hyperparameter and set two optional values: fixed-length segmentation and recursive character segmentation. Define the text block size as an integer hyperparameter and set its value range to 256 to 1024 with a step size of 64. Define the text block overlap size as a conditional hyperparameter and set its value range to 0 to 200 with a step size of 10, activating only when the recursive character segmentation strategy is selected. Define the vector embedding model selection as a categorical hyperparameter and set three optional values: bge-base-en-v1.5 model, m3e-base model, and text-embedding-3-small model. Define the number of retrieved documents as an integer hyperparameter and set its value range to 1 to 10. Then, define whether... Enable reranking as a categorical hyperparameter with two possible values: true and false. Define the reranking model selection as a conditional hyperparameter with two possible values: bge-reranker-base and cohere-rerank-english-v2.0, activated only when reranking is enabled. Define the number of reranked documents returned as a conditional hyperparameter with a value range of 1 to 5, activated only when reranking is enabled, and constrained to not exceed the number of retrieved documents. Define the large language model selection as a categorical hyperparameter with three possible values: DeepSeek-V3.1, Kimi-k2, and Qwen3-235B-A22B-2507. Define the temperature parameter as a continuous hyperparameter with a value range of 0.0 to 1.0. Combine all hyperparameters to form a configuration vector. ,in The configuration vector is defined as the total dimension of the hyperparameters, with an empirical value of 9. Constructing a structured search space .

[0054] The specific implementation of step S02 is to first, based on the input configuration vector... The instantiated retrieval enhancement generation system pipeline divides the knowledge base document set into blocks according to the text segmentation strategy and text block size in the configuration vector. When the text segmentation strategy is recursive character segmentation, the overlap area of ​​adjacent text blocks is set according to the overlap size of the text blocks. All text blocks are vectorized and encoded according to the vector embedding model selection in the configuration vector and stored in the vector database. Each evaluation question is extracted sequentially from the evaluation question set and vectorized using the vector embedding model selection to generate a query vector. The query vector is used to retrieve the top-returned document text blocks with the highest similarity in the vector database. When re-ranking is enabled in the configuration vector, the re-ranking model selection is used to re-score the relevance of the retrieved text blocks and select the top-ranked re-returned document text blocks with the highest scores as the retrieval context. The retrieval context and evaluation question are combined into prompt text and input into the large language model selection in the configuration vector. The generation randomness of the large language model is set according to the temperature parameter to generate the answer and calculate the fidelity. Relevance of answers and contextual recall The three performance metrics are combined into a performance metric vector. The formula for calculating loyalty is as follows:

[0055] ;

[0056] In the formula, The value represents fidelity and ranges from 0 to 1. To assess the total number of issues, with an empirical value of 50; For the evaluation question number; For the first Generating answers to each assessment question; For the first The retrieval context for each evaluation question; The answer support function is used when generating the answer. All statements can be found in the retrieval context. The return value is 1 if supporting evidence is found, otherwise it is 0. The formula for calculating the relevance of the answer is as follows:

[0057] ;

[0058] In the formula, The answer is relevant and its value ranges from 1 to 5; For the first One assessment question; The relevance scoring function is used to calculate and generate answers using a cross-encoder model or a large language evaluation model. With assessment issues The context recall rate is calculated based on the semantic relevance of the data and returned as a score from 1 to 5. The formula for calculating context recall is as follows:

[0059] ;

[0060] In the formula, Context recall, with a value ranging from 0 to 1; For the first A set of retrieval contexts for each evaluation question; For the first The true context set of an evaluation question; The symbol for counting the number of elements in a set; This is the intersection operator for sets.

[0061] The specific implementation of step S03 is to set the initial configuration quantity to 10, and search the structured search space. Ten initial configurations were generated using the Latin hypercube sampling method. For each initial configuration Execute the multi-objective evaluation function to obtain the corresponding performance index vector. The data pair consisting of configuration vector and performance metric vector. Collect and form the initial observation dataset ,in For the initial configuration sequence number and .

[0062] The specific implementation of step S04 is from the initial observation dataset. Or update the observation dataset Extract all configuration vectors and their corresponding fidelity values ​​to construct a fidelity training set, and use this fidelity training set to train the first Gaussian process model. And fit the mean function of fidelity Sum of covariance functions From the initial observation dataset Or update the observation dataset Extract all configuration vectors and their corresponding answer relevance values ​​to construct an answer relevance training set, and use this training set to train a second Gaussian process model. And fit the mean function of the correlation between the answers. Sum of covariance functions From the initial observation dataset Or update the observation dataset Extract all configuration vectors and their corresponding context recall values ​​to construct a context recall training set, and use this context recall training set to train the third Gaussian process model. And fit the mean function of context recall. Sum of covariance functions ,in This is another configuration vector in the structured search space. The Gaussian process model is expressed as follows:

[0063] ;

[0064] In the formula, For the first 0 objective functions and The values ​​1, 2, and 3 correspond to fidelity, answer relevance, and contextual recall, respectively. It is a Gaussian process distribution; The mean function represents the objective function. In configuration vector Prior estimates at the location; The configuration vector is represented by the covariance function. With configuration vector The correlation between them. Covariance function. The Matern 5 / 2 kernel function is used, and the calculation formula is expressed as follows:

[0065] ;

[0066] In the formula, For the first The signal variance parameters of a Gaussian process model are obtained by optimization through maximizing the marginal likelihood function; For the first The length scale parameters of a Gaussian process model are obtained by optimization through maximizing the marginal likelihood function; For configuration vectors With configuration vector The normalized Euclidean distance between them; It is a natural exponential function. The Euclidean distance calculation formula is as follows:

[0067] ;

[0068] In the formula, For configuration vectors The The normalized values ​​of each component; For configuration vectors The The normalized values ​​of each component; This represents the total dimension of the hyperparameters. Before calculating the Euclidean distance, the configuration vectors need to be configured. Each component is normalized, categorical hyperparameters are encoded as values ​​between 0 and 1, and integer and continuous hyperparameters are linearly mapped to values ​​between 0 and 1 according to their range.

[0069] The specific implementation of step S05 is from the initial observation dataset. Or update the observation dataset Extract the performance index vectors of all nondominated solutions to form the current Pareto front. Set reference points Each component is the minimum value of all performance indicators minus a fixed offset of 0.1, where As a reference value for fidelity, This serves as a reference value for the relevance of the answer. This serves as a reference value for contextual recall. For the structured search space... Each candidate configuration in Using three Gaussian process models , and Predict the posterior distributions of fidelity, answer relevance, and contextual recall respectively, and generate a Monte Carlo sampling method based on the posterior distributions. Group performance index vector samples and The empirical value is 1000, for each sample group. Calculate the hypervolume increment The candidate configuration is obtained by averaging the hypervolume increments of all samples. Expected supervolume improvement value The formula for calculating the expected volume improvement is as follows:

[0070] ;

[0071] In the formula, For candidate configuration The expected improvement in supervolume; Number of Monte Carlo samples; The Monte Carlo sampling sequence number; For the first The performance metric vector samples generated from the second sampling and from the candidate configuration Posterior predicted distribution at Obtained by mid-sample; The solution set and reference point are represented by the hypervolume calculation function. The enclosed multidimensional space volume; This is the current Pareto frontier; This is the set union operator. Next, evaluate the configuration. The formula for calculating the desired hypervolume improvement value is obtained by solving the problem of maximizing the desired hypervolume improvement value, and is expressed as follows:

[0072] ;

[0073] In the formula, For the first The next evaluation configuration for the next iteration; To return the sign of the independent variable that maximizes the objective function; This is a structured search space.

[0074] The specific implementation of step S06 is the configuration for the next evaluation. Execute the multi-objective evaluation function to obtain a new performance index vector. , put the data Add to the initial observation dataset Or update the current observation dataset Forming a new, updated observation dataset Use updated observation dataset Retrain the three Gaussian process models , and And update the mean function Sum of covariance functions The parameters determine the current iteration number. Has the preset number of iterations been reached? and The experience value is 100, of which This is the current iteration round number, and its initial value is 0. When season And return to step S05 to continue execution, when The iteration terminates and proceeds to step S07.

[0075] The specific implementation method of step S07 is the same as described above, and will not be repeated in detail here.

[0076] It should be noted that the variables involved in this invention are explained in detail in Table 1.

[0077] Table 1. Variable Explanation Table

[0078]

[0079] The advancements of this invention compared to traditional methods are primarily reflected in the scientific and systematic nature of its pricing approach. Traditional computing power pricing typically relies on simple cost-plus models or direct reference to market prices, lacking comprehensive consideration of equipment depreciation patterns, operating time fluctuations, and multi-dimensional performance indicators. This invention, through a comparison of the declining balance method and the annual summation method, can select the most suitable depreciation method based on the actual depreciation characteristics of the computing power equipment, accurately reflecting the time-depreciation pattern of equipment value and avoiding the problem of unreasonable cost allocation caused by fixed depreciation rates. Furthermore, this invention employs cubic spline interpolation to smooth and fit historical operating data for future prediction, fully utilizing the trend characteristics and periodic patterns of time series data. Compared to simple mean prediction or linear extrapolation methods, it can more accurately estimate annual operating time, reducing unit cost calculation errors caused by operating time estimation biases. This invention introduces the Analytic Hierarchy Process (AHP) to construct a multi-level pricing model. It structurally decomposes and quantifies the weights of multiple influencing factors, such as computing power, storage performance, network performance, and reliability assurance. Expert judgment matrices and consistency checks ensure the rationality of weight assignment. Compared to traditional methods that rely on single performance indicators or subjective experience to set weights, this invention achieves objectivity and traceability in pricing. Furthermore, this invention establishes a comparison and verification mechanism between estimated and market prices, along with an iterative optimization process. Through multiple rounds of weight adjustments and price corrections, the final pricing aligns with both cost-benefit principles and closely reflects actual market levels, avoiding the problems of excessively high pricing leading to insufficient market competitiveness or excessively low pricing resulting in low resource utilization.

[0080] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A configuration method for a RAG system based on multi-objective Bayesian optimization, characterized in that, The process includes the following steps: Constructing a structured search space, defining the component selection and parameter settings of the text segmentation module, vector embedding module, vector retrieval module, result reordering module, and text generation module of the retrieval enhancement generation system pipeline as the structured search space, which includes categorical hyperparameters, integer hyperparameters, and conditional hyperparameters; establishing a multi-objective evaluation function, which receives configuration vectors sampled from the structured search space as input and outputs performance index vectors containing fidelity, answer relevance, and contextual recall; initializing the observation dataset, randomly sampling a preset number of initial configurations from the structured search space and executing the multi-objective evaluation function on each initial configuration to obtain the corresponding performance index vector, and combining the configuration vectors and performance index vectors to form the initial observation dataset; A probabilistic proxy model is constructed, and independent Gaussian process models are established for fidelity, answer relevance, and contextual recall. The Gaussian process models are fitted with the mean and covariance functions of the objective function based on the initial observation dataset. The expected hypervolume improvement acquisition function is calculated. Based on the current Pareto front and the posterior prediction distribution of the Gaussian process model, the expected hypervolume improvement value of each candidate configuration in the structured search space is calculated, and the configuration with the largest expected hypervolume improvement value is selected as the next evaluation configuration. Iterative optimization and update are performed. The multi-objective evaluation function is executed on the next evaluation configuration to obtain a new performance index vector. The data pair consisting of the configuration vector and the new performance index vector is added to the initial observation dataset to form an updated observation dataset and update the posterior distribution of all Gaussian process models. The expected hypervolume improvement acquisition function and iterative optimization and update steps are repeated until the preset number of iterations is reached. Output the Pareto optimal configuration set, traverse and update all evaluated configurations in the observation dataset to identify non-dominated solutions to form the Pareto optimal configuration set, and select deployment configurations from the Pareto optimal configuration set according to business requirements.

2. The RAG system configuration method based on multi-objective Bayesian optimization according to claim 1, characterized in that, The construction of the structured search space specifically involves formally defining all variable components and parameters of the retrieval enhancement generation system pipeline as a configuration vector, which contains multiple hyperparameter components.

3. The RAG system configuration method based on multi-objective Bayesian optimization according to claim 2, characterized in that, The categorical hyperparameters include text segmentation strategy, vector embedding model selection, and large language model selection.

4. The RAG system configuration method based on multi-objective Bayesian optimization according to claim 3, characterized in that, The integer hyperparameters include text block size, number of documents returned by retrieval, and temperature parameter.

5. The RAG system configuration method based on multi-objective Bayesian optimization according to claim 4, characterized in that, The conditional hyperparameters include text block overlap size, reordering model selection, and reordering return quantity. These conditional hyperparameters only take effect when the activation condition is met.

6. The RAG system configuration method based on multi-objective Bayesian optimization according to claim 5, characterized in that, The establishment of the multi-objective evaluation function specifically involves dynamically instantiating the retrieval enhancement generation system pipeline based on the configuration vector, preprocessing and indexing the knowledge base documents, traversing the evaluation question set, executing the retrieval enhancement generation query for each evaluation question, and recording the generated answer and retrieval context.

7. The RAG system configuration method based on multi-objective Bayesian optimization according to claim 6, characterized in that, The fidelity calculation uses an answer support function to determine whether the generated answer is entirely based on the retrieval context.

8. The RAG system configuration method based on multi-objective Bayesian optimization according to claim 7, characterized in that, The calculation of answer relevance uses a relevance scoring function to measure the degree of relevance between the generated answer and the user's question.

9. The RAG system configuration method based on multi-objective Bayesian optimization according to claim 8, characterized in that, The context recall rate is calculated by dividing the number of elements in the intersection of the search context and the real context by the number of elements in the real context.

10. The RAG system configuration method based on multi-objective Bayesian optimization according to claim 9, characterized in that, The construction of the Gaussian process model involves independently establishing Gaussian processes for fidelity, answer relevance, and contextual recall, with the objective function following a Gaussian distribution of mean and covariance functions.

Images