Hierarchical system tuning method and system based on large language model

Abstract

The invention discloses a hierarchical database system tuning method and system based on a large language model, and relates to the technical field of database system tuning. The method comprises the steps that expert knowledge in the database field is processed based on a large language model, a cue word integration algorithm is used for converting the knowledge into structured knowledge capable of being used by a computer so as to optimize a search space, and a coarse granularity-fine granularity Bayesian optimization algorithm is used for determining optimal configuration, so that configuration optimization of a system level is completed; a parameter screening algorithm, a space pruning algorithm and a context Bayesian optimization algorithm are utilized, and an optimal execution plan is generated by selecting an appropriate optimization prompt for each query in the complete workload, so that configuration optimization of query levels is completed. According to the method, the search space is optimized by utilizing expert knowledge, the optimized space is explored by utilizing a Bayesian optimization algorithm, multi-level optimization is performed on the database system from a system configuration level and a query level, and the optimization efficiency and the system performance are improved.

Description

Technical Field

[0001] The invention relates to the technical field of database system tuning, and more particularly to a hierarchical system tuning method and system based on a large language model. Background Art

[0002] A database management system is a software system used to create and manage databases, and its performance is critical to enterprise applications and scientific research. Database management systems have many configuration parameters, often figuratively referred to as knobs. Determining the appropriate values ​​of these knobs to improve the performance of modern database management systems is a long-standing problem in the database field. The field of database tuning technology has gone through three distinct stages: manual tuning, heuristic-based tuning, and machine learning-based tuning.

[0003] In the manual tuning stage, database administrators and experts manually adjust configuration parameters by deeply analyzing factors such as database workload, query performance, and hardware resources in order to achieve optimal performance. However, the limitations of this method are also obvious: it is costly, difficult to expand, and difficult to apply to complex and high-dimensional database environments.

[0004] As experience and knowledge accumulate, people begin to explore heuristic-based tuning methods. The tuning technology at this stage combines rules and heuristic algorithms to adjust database parameters more intelligently. Rule-based methods rely on a set of predefined rules, which are usually based on the experience and knowledge of experts and database administrators. Search-based methods, on the other hand, automatically explore different combinations of configurations and query plans to find the optimal performance solution. Although heuristic-based tuning methods have improved the efficiency and accuracy of tuning to a certain extent, their limitations are gradually emerging in the face of ever-changing database environments and complex performance issues. For example, static rules are difficult to adapt to the characteristics of different workloads and database operating environments, and search algorithms may also perform poorly when dealing with high-dimensional and nonlinear problems.

[0005] In recent years, with the rapid development of machine learning technology, tuning methods based on machine learning have gradually become a research hotspot. This method realizes the automation and intelligence of database tuning through technologies such as Bayesian optimization and reinforcement learning. Bayesian optimization can find near-optimal configuration parameters in a small number of iterations by modeling the relationship between configuration and performance. Reinforcement learning explores the configuration space through trial and error strategies, and can continuously learn and optimize tuning strategies. These technologies have brought revolutionary changes to database tuning, not only greatly improving the efficiency and accuracy of tuning, but also dynamically adapting to real-time changing environments. However, tuning methods based on machine learning also face many challenges. First, the performance of machine learning methods is highly dependent on data quality and the accuracy of model training. If the initial data set is biased or noisy, or the model training is insufficient, the tuning results may be greatly reduced. Second, methods such as Bayesian optimization and reinforcement learning have challenges in terms of computational complexity, initial data set quality, and time cost. For large-scale databases and complex performance problems, these methods may take longer and require more computing resources to achieve satisfactory results. In addition, most machine learning methods ignore external domain knowledge and cannot make full use of previous tuning experience to guide automated tuning. Summary of the invention

[0006] The purpose of the present invention is to provide a hierarchical database system tuning method and system based on a large language model, which can perform hierarchical processing on the tuning of the database system to improve the tuning efficiency and effect of the database system.

[0007] To achieve the above object, the present invention provides the following solutions for the database system:

[0008] A hierarchical system tuning method based on a large language model, comprising:

[0009] Determine credible database domain knowledge; the credible database domain knowledge is obtained by using a large language model to convert multi-source heterogeneous noisy data obtained through a data pipeline based on a large language model into credible knowledge;

[0010] The trusted database domain knowledge is converted into structured knowledge by using a prompt integration algorithm to optimize the search space, and the coarse-grained-fine-grained Bayesian optimization algorithm is used to determine the optimal system configuration and complete the system level configuration optimization; the structured knowledge is the knowledge that has been converted into a structured data format that can be used by a computer;

[0011] The parameter screening algorithm, spatial pruning algorithm and contextual Bayesian optimization algorithm are used to determine the optimal execution plan for each query statement and complete the query hierarchical configuration optimization.

[0012] Optionally, determine trusted database domain knowledge, including:

[0013] Acquire multi-source database domain knowledge from multiple knowledge sources; knowledge sources include but are not limited to system manuals, expert experience and large language models;

[0014] Use a large language model to denoise the domain knowledge of multi-source databases, filter out unreasonable knowledge, and obtain denoised database domain knowledge;

[0015] The multi-source database domain knowledge is summarized by using a large language model, and a knowledge summary is generated for each parameter; the summarizing of the multi-source database domain knowledge includes: abandoning low-credibility knowledge, retaining high-credibility knowledge, and generating a knowledge summary of knowledge from different sources but compatible with each other;

[0016] A large language model is used to check the factual consistency of knowledge summaries, and knowledge summaries that pass the consistency check are determined to be credible database domain knowledge.

[0017] Optionally, the trusted database domain knowledge is converted into structured knowledge using a prompt integration algorithm to optimize the search space, and a coarse-grained-fine-grained Bayesian optimization algorithm is used to determine the optimal system configuration to complete the system level configuration optimization, including:

[0018] According to the trusted database domain knowledge, a structured view is constructed by using a prompt integration algorithm based on a large language model; the prompt integration algorithm provides prompt words for solving the same task but with different contents to the large language model, and then integrates different results generated by different prompt words into a trusted result; the structured view is constructed by extracting values ​​of four types of attributes, namely, recommended value, recommended maximum value, recommended minimum value and special value, from the unstructured trusted database domain knowledge, and then storing the recommended value, recommended maximum value, recommended minimum value and special value in a structured data format;

[0019] Based on the structured view, the search space is optimized from two sub-levels: dimension and value range, to obtain an optimized search space;

[0020] The coarse-grained-fine-grained Bayesian optimization algorithm is used to search the optimization search space, determine the optimal configuration, and complete the system-level configuration optimization.

[0021] Optionally, based on the structured view, the search space is optimized from two sub-levels of dimension and value range to obtain an optimized search space, including:

[0022] Based on the structured view, a large language model is used to identify and adjust a target knob in the complete search space, thereby obtaining a reduced-dimensional search space; the target knob has an impact on database performance that reaches a preset level and has the greatest tuning value;

[0023] Determine any knob in the dimension reduction search space as the current knob;

[0024] Determine the optimal value range of the current knob;

[0025] Update the current knob and return to step "determine any knob in the dimensionality reduction search space as the current knob" until the dimensionality reduction search space is traversed to obtain the optimized search space.

[0026] Optionally, determining an optimal value range of the current knob includes:

[0027] For a numeric knob, a preset maximum value is obtained from the structured view to replace the current knob default maximum value officially provided by the database; the preset maximum value is less than or equal to the current knob default maximum value officially provided by the database;

[0028] For a numeric knob, a preset minimum value is obtained from the structured view to replace the current knob default minimum value officially provided by the database; the preset minimum value is greater than or equal to the current knob default minimum value officially provided by the database;

[0029] Determine that the interval enclosed by the preset minimum value and the preset maximum value is the current knob pending value range;

[0030] Determine whether the current knob has a special value and obtain a determination result; when the knob has a special value, the influence of the knob on the operation of the database system is essentially different from the influence in other value ranges;

[0031] If the judgment result is no, determining the undetermined value range as the optimized value range of the current knob;

[0032] If the judgment result is yes, a separate Boolean virtual knob is added to the current search space; when the Boolean virtual knob takes the value of "yes", a special value is directly taken; when the Boolean virtual knob takes the value of "no", a value is taken in the pending value range.

[0033] Optionally, a coarse-grained-fine-grained Bayesian optimization algorithm is used to search the optimization search space, determine the optimal configuration, and complete the system-level configuration optimization, including:

[0034] Determining two levels of sub-feasible spaces according to the structured view; the sub-feasible spaces include a coarse-grained sub-feasible space and a fine-grained sub-feasible space;

[0035] A Latin hypercube sampling algorithm is used to perform stratified sampling on the coarse-grained sub-feasible space to generate a user-specified number of knob configuration samples;

[0036] Evaluate the knob configuration sample in the database management system to obtain the performance index corresponding to the knob configuration sample;

[0037] Initializing a proxy model of a Bayesian optimization framework using a plurality of the knob configuration samples and their corresponding performance indicators;

[0038] Let the number of iterations n = 1;

[0039] Let the current sub-feasible space be the coarse-grained sub-feasible space;

[0040] Perform local sampling and random sampling in the current sub-feasible space, substitute multiple sampling points into the proxy model respectively, and determine the estimated objective function value and corresponding uncertainty of each sampling point;

[0041] According to the estimated objective function value and uncertainty, the point corresponding to the maximum function value is determined as the candidate point;

[0042] Evaluate the candidate points in the database management system to obtain performance indicators corresponding to the candidate points;

[0043] Using the candidate points and their corresponding performance indicators, the proxy model at the nth iteration is updated to obtain the proxy model at the n+1th iteration;

[0044] Increase the value of the number of iterations n by 1, and return to the step "perform local sampling and random sampling in the current sub-feasible space, substitute multiple sampling points into the proxy model, and determine the estimated objective function value and corresponding uncertainty of each sampling point" until the value of the number of iterations n reaches the preset number of iterations N1;

[0045] The current sub-feasible space will be made a fine-grained sub-feasible space;

[0046] Repeat steps "perform local sampling and random sampling in the current sub-feasible space, substitute multiple sampling points into the proxy model, and determine the estimated objective function value and corresponding uncertainty of each sampling point" to step "increase the value of the number of iterations n by 1" until the number of iterations n reaches the resource limit N2;

[0047] According to the optimization records, the configuration that optimizes the performance indicators is determined as the optimal configuration, and the system-level configuration optimization is completed.

[0048] Optionally, the method for determining the coarse-grained sub-feasible space includes:

[0049] Determine any knob as the current knob;

[0050] The discrete value set is obtained by multiplying the recommended value of the current knob in the structured view by a set of scaling multipliers α; the scaling multipliers are:

[0051]

[0052] Where β is the scaling factor, ranging from 0 to 1, r1 is the candidate value of the first coefficient predefined by the user, r2 is the candidate value of the second coefficient predefined by the user, and r n is the candidate value of the nth coefficient predefined by the user, r i is the candidate value of the i-th coefficient predefined by the user, U is the preset maximum or minimum value obtained from the structured view, and V is the recommended value in the structured view;

[0053] Determine that the space obtained by the set of discrete value groups corresponding to all knobs is the coarse-grained sub-feasible space;

[0054] The fine-grained sub-feasible space is a search space formed based on the value range of each knob.

[0055] Optionally, the optimal execution plan for each query statement is determined using a parameter screening algorithm, a spatial pruning algorithm, and a contextual Bayesian optimization algorithm to complete query hierarchical configuration optimization, including:

[0056] Using a large language model-based screening algorithm, according to the database manual, determine the target Boolean knob in the optimization search space; the target Boolean knob can change the execution plan of the query statement;

[0057] By using a large language model, the knobs that have a significant impact on the execution performance of the query statement are selected from the above Boolean knobs based on domain knowledge and expert experience, and a reduced-dimensional search space is obtained;

[0058] Construct multiple knob combinations based on the selected knobs;

[0059] Based on the space pruning algorithm, the knob rules generated by the large language model are used to delete the infeasible knob combinations in the dimensionality reduction search space and the knob combinations whose impact on performance does not reach the preset level, and the pruning space is obtained;

[0060] The contextual Bayesian optimization algorithm is used to search the pruning space, determine the optimal execution plan for each query statement, and complete the query hierarchy configuration optimization; the context refers to the distribution of multi-table data in the database, which changes over time.

[0061] A hierarchical system tuning system based on a large language model, comprising:

[0062] A trusted knowledge acquisition module is used to determine trusted database domain knowledge; the trusted database domain knowledge is obtained by using a large language model to convert multi-source heterogeneous noisy data obtained by the data pipeline of the large language model into trusted knowledge;

[0063] The system-level configuration optimization module is used to use the prompt integration algorithm to transform the trusted database domain knowledge into structured knowledge to optimize the search space, and use the coarse-grained-fine-grained Bayesian optimization algorithm to determine the optimal system configuration and complete the system-level configuration optimization; the structured knowledge is the knowledge that has been transformed into a structured data format that can be used by a computer; the query-level configuration optimization module is used to use the parameter screening algorithm, the space pruning algorithm and the contextual Bayesian optimization algorithm to determine the optimal execution plan for each query statement and complete the query-level configuration optimization.

[0064] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0065] The present invention provides a hierarchical database tuning method and system. First, the database system is hierarchically tuned. Starting from the two levels of system and query optimization, the resource configuration of the system and the execution plan of the query statement are tuned in a targeted manner. Compared with the single-level tuning, the hierarchical tuning aims to consider the optimization requirements of the overall system performance and the specific query at the same time, so as to achieve a more comprehensive, more accurate and more applicable optimization effect. Secondly, the present invention makes full use of extremely valuable domain knowledge to optimize the search space, collects and refines multi-source heterogeneous domain knowledge, and converts the domain knowledge into a structured view that can be used by the machine, thereby reducing the dimension of the search space and optimizing the value range on each dimension, so that the convergence speed is greatly improved and better tuning performance is achieved, which significantly reduces the high tuning cost of traditional machine learning-based technologies. In addition, the present invention also correctly handles the defects of the large language model and makes full use of the powerful functions of the large language model. By using a prompt integration algorithm based on the large language model, the influence of the vulnerability and hallucination problems of the large language model in the text conversion process is successfully solved, and its reliability in the information processing process is greatly improved. At the same time, taking into full account the powerful functions of the large language model, the present invention integrates the contextual learning ability, text understanding, and semantic analysis ability of the large language model into the optimization process, making the database optimization work more accurate and efficient. Finally, the present invention improves the traditional Bayesian optimization framework and embeds structured domain knowledge into it: in the first level, by searching the tiny feasible space and the optimized heterogeneous space from coarse to fine in turn, the structured domain knowledge is fully utilized, and the efficiency and thoroughness of the search are well balanced, thereby greatly reducing the huge iteration cost and finding a configuration with good effect; in the second level, through a Gaussian process that considers the context, the contextual factor of data distribution characteristics is additionally considered, and then the optimization process is adjusted according to the data distribution characteristics, which improves the convergence speed and accuracy, avoids unnecessary computational overhead, and makes the optimization process more efficient, improving the efficiency and performance of the optimization. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0067] Figure 1 A schematic diagram of a hierarchical database tuning method flow provided in Example 1 of the present invention. DETAILED DESCRIPTION

[0068] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0069] The purpose of the present invention is to provide a hierarchical tuning method and system based on a large language model, which can perform hierarchical tuning on computer systems such as databases to improve the tuning efficiency and effect of the computer systems.

[0070] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is directed to a database system and further described in detail in conjunction with the following drawings and specific implementation methods.

[0071] Example 1

[0072] like Figure 1 As shown, a hierarchical system tuning method based on a large language model in this embodiment includes:

[0073] Step 101: Determine the trusted database domain knowledge. The trusted database domain knowledge is obtained by using a large language model to convert multi-source heterogeneous noisy data obtained through a data pipeline based on a large language model into trusted knowledge.

[0074] Step 102: Use the prompt integration algorithm to transform the trusted database domain knowledge into structured knowledge to optimize the search space, and use the coarse-grained-fine-grained Bayesian optimization algorithm to determine the optimal system configuration and complete the system level configuration optimization. Structured knowledge is knowledge that has been transformed into a structured data format that can be used by computers.

[0075] Step 103: Utilize the parameter screening algorithm, the spatial pruning algorithm and the contextual Bayesian optimization algorithm to determine the optimal execution plan for each query statement and complete the query hierarchical configuration optimization.

[0076] Step 101 includes:

[0077] Step 101-1: Acquire multi-source database domain knowledge from multiple knowledge sources. Knowledge sources include but are not limited to system manuals, expert experience, and large language models.

[0078] Step 101-2: Use a large language model to denoise the domain knowledge of the multi-source database, filter out unreasonable knowledge, and obtain denoised database domain knowledge.

[0079] Step 101-3: Use the large language model to summarize the domain knowledge of the multi-source database and generate a knowledge summary for each parameter. Summarizing the domain knowledge of the multi-source database includes: abandoning low-credibility knowledge, retaining high-credibility knowledge, and generating a knowledge summary of knowledge from different sources but compatible with each other.

[0080] Step 101 - 4 : Use the large language model to perform factual consistency check on the knowledge summary, and determine that the knowledge summary that passes the consistency check is credible database domain knowledge.

[0081] Step 102 includes:

[0082] Step 102-1: Based on the trusted database domain knowledge, a structured view is constructed using a prompt integration algorithm based on a large language model. The prompt integration algorithm provides prompt words for solving the same task but with different contents to the large language model, and then integrates the different results generated by different prompt words into a trusted result. The construction of the structured view is to extract the values ​​of four types of attributes, namely, recommended value, recommended maximum value, recommended minimum value, and special value, from the unstructured trusted database domain knowledge, and then store the recommended value, recommended maximum value, recommended minimum value, and special value in a structured data format.

[0083] Step 102 - 2: Based on the structured view, the search space is optimized from two sub-levels: dimension and value range, to obtain an optimized search space.

[0084] Step 102-3: Use the coarse-grained-fine-grained Bayesian optimization algorithm to search the optimization search space, determine the optimal configuration, and complete the system-level configuration optimization.

[0085] Step 102-2 includes:

[0086] Step 102-2-1: Based on the structured view, the large language model is used to identify and adjust the target knob in the complete search space, thereby obtaining a reduced-dimensional search space. The target knob has a preset impact on database performance and has the greatest tuning value.

[0087] Step 102-2-2: Determine any knob in the dimensionality reduction search space as the current knob.

[0088] Step 102-2-3: Determine the optimal value range of the current knob.

[0089] Step 102-2-4: Update the current knob and return to step 102-2-2 until the dimensionality reduction search space is traversed to obtain the optimized search space.

[0090] Step 102-2-3 includes:

[0091] Step 102-2-3-1: For a numeric knob, obtain a preset maximum value from the structured view to replace the current knob default maximum value provided by the database. The preset maximum value is less than or equal to the current knob default maximum value provided by the database.

[0092] Step 102-2-3-2: For a numeric knob, obtain a preset minimum value from the structured view to replace the current knob default minimum value provided by the database. The preset minimum value is greater than or equal to the current knob default minimum value provided by the database.

[0093] Step 102-2-3-3: Determine that the interval enclosed by the preset minimum value and the preset maximum value is the pending value range of the current knob.

[0094] Step 102-2-3-4: Determine whether the current knob has a special value and obtain a determination result. When the knob has a special value, the impact of the knob on the operation of the database system is substantially different from the impact of the knob in other value ranges.

[0095] Step 102-2-3-5: If the judgment result is no, then determine that the undetermined value range is the optimized value range of the current knob.

[0096] Step 102-2-3-6: If the result of the judgment is yes, then add a separate Boolean virtual knob to the current search space. When the Boolean virtual knob takes the value "yes", it directly takes the special value. When the Boolean virtual knob takes the value "no", it takes the value in the pending value range.

[0097] Step 102-3 includes:

[0098] Step 102-3-1: Determine two levels of sub-feasible spaces according to the structured view. The sub-feasible spaces include a coarse-grained sub-feasible space and a fine-grained sub-feasible space.

[0099] Step 102-3-2: Use the Latin hypercube sampling algorithm to perform stratified sampling on the coarse-grained sub-feasible space to generate a user-specified number of knob configuration samples.

[0100] Step 102-3-3: Evaluate the knob configuration sample in the database management system to obtain the performance index corresponding to the knob configuration sample.

[0101] Step 102-3-4: Initialize the proxy model of the Bayesian optimization framework using multiple knob configuration samples and their corresponding performance indicators.

[0102] Step 102-3-5: Set the number of iterations n=1.

[0103] Step 102-3-6: Set the current sub-feasible space to be a coarse-grained sub-feasible space.

[0104] Step 102-3-7: Perform local sampling and random sampling in the current sub-feasible space, substitute multiple sampling points into the proxy model respectively, and determine the estimated objective function value and corresponding uncertainty of each sampling point.

[0105] Step 102-3-8: According to the estimated objective function value and uncertainty, determine the point corresponding to the maximum function value as the candidate point.

[0106] Step 102-3-9: Evaluate the candidate points in the database management system to obtain performance indicators corresponding to the candidate points.

[0107] Step 102-3-10: Using the candidate points and their corresponding performance indicators, update the proxy model at the nth iteration to obtain the proxy model at the (n+1)th iteration.

[0108] Step 102-3-11: Increase the value of the number of iterations n by 1, and return to the step "perform local sampling and random sampling in the current sub-feasible space, substitute multiple sampling points into the proxy model respectively, and determine the estimated objective function value and corresponding uncertainty of each sampling point" until the value of the number of iterations n reaches the preset number of iterations N1.

[0109] Step 102-3-12: Set the current sub-feasible space to a fine-grained sub-feasible space.

[0110] Step 102-3-13: Repeat steps "perform local sampling and random sampling in the current sub-feasible space, substitute multiple sampling points into the proxy model, and determine the estimated objective function value and corresponding uncertainty of each sampling point" to step "increase the value of the number of iterations n by 1" until the number of iterations n reaches the resource limit N2.

[0111] Step 102-3-14: According to the optimization record, determine the configuration that optimizes the performance index as the optimal configuration, and complete the system-level configuration optimization.

[0112] In step 102-3-1, the method for determining the coarse-grained sub-feasible space includes:

[0113] Step 102-3-1-1: Determine any knob as the current knob.

[0114] Step 102-3-1-2: A discrete value group is obtained by multiplying the recommended value of the current knob in the structured view by a set of scaling multipliers α. The scaling multipliers are:

[0115]

[0116] Where β is the scaling factor, ranging from 0 to 1, r1 is the candidate value of the first coefficient predefined by the user, r2 is the candidate value of the second coefficient predefined by the user, and r n is the candidate value of the nth coefficient predefined by the user, r i is the candidate value of the i-th coefficient predefined by the user, U is the preset maximum or minimum value obtained from the structured view, and V is the recommended value in the structured view.

[0117] Step 102-3-1-3: Determine that the space obtained by the set of discrete value groups corresponding to all knobs is the coarse-grained feasible sub-space, and the fine-grained feasible sub-space is the search space formed based on the value range of each knob.

[0118] Step 103 includes:

[0119] Step 103-1: Using a screening algorithm based on a large language model, according to the database manual, determine the target Boolean knob in the optimization search space. The target Boolean knob can change the execution plan of the query statement.

[0120] Step 103 - 2: Using a large language model, based on domain knowledge and expert experience, select the knobs in the above Boolean knobs that have a significant impact on the execution performance of the query statement, and obtain a reduced-dimensional search space.

[0121] Step 103 - 3 : Construct multiple knob combinations based on the selected knobs.

[0122] Step 103 - 4: Based on the space pruning algorithm, the knob rules generated by the large language model are used to delete infeasible knob combinations in the dimensionality reduction search space and knob combinations whose impact on performance does not reach a preset level to obtain a pruned space.

[0123] Step 103-5: Use the contextual Bayesian optimization algorithm to search the pruning space, determine the optimal execution plan for each query statement, and complete the query level configuration optimization. The context refers to the distribution of multi-table data in the database, which changes over time.

[0124] A hierarchical database tuning method provided by this embodiment is described in detail below. This embodiment includes the following steps.

[0125] Step 1: Generate credible database domain knowledge. Use the text understanding and logical inference capabilities of the large language model to build a data pipeline to process multi-source, heterogeneous, and noisy database domain knowledge, filter out the noise content, summarize the remaining multi-source reliable knowledge into a knowledge summary, and finally perform a factual consistency check on the knowledge summary. Specifically, the following sub-steps are involved:

[0126] Step 1.1: Obtain multi-source, heterogeneous, and noisy database domain knowledge based on knowledge sources such as system manuals, expert experience, and large language models.

[0127] Multi-source, heterogeneous domain knowledge, for example, domain knowledge from sources such as the Wenxin Yiyan model, database system manuals, online forums, etc., which exists in the form of natural language but has different computer storage formats (txt, html, xml, etc.).

[0128] Noisy domain knowledge: Domain knowledge from various sources may contain problems such as human summary errors, outdated knowledge, and incomplete knowledge.

[0129] Step 1.2: Use the large language model to denoise the database domain knowledge, filter out unreasonable knowledge, and obtain denoised database domain knowledge.

[0130] Filtering noisy knowledge can be achieved in the following way: let the large language model compare whether the collected knowledge conflicts with the system view of the DBMS (Database Management System) (for example, pg_settings from the PostgreSQL database), filter out the conflicting knowledge, and retain the non-conflicting knowledge.

[0131] Step 1.3: Use the large language model to summarize the domain knowledge of multi-source databases. The large language model discards low-credibility knowledge and retains high-credibility knowledge, and finally generates knowledge summaries for knowledge from different sources but compatible with each other.

[0132] Summarizing multi-source knowledge into knowledge summaries can be achieved in the following ways: there are multiple sources of knowledge for the same database configuration parameters, different priorities are set for different knowledge sources (for example, manual summary > network experience > recommendations of the large language model), and when there is a conflict between knowledge from different sources, the large language model abandons the knowledge with low priority, retains the knowledge with high priority, and summarizes all retained knowledge.

[0133] Step 1.4: Use the large language model to perform factual consistency check on the knowledge before and after the summary to ensure that the generated knowledge summary is consistent with the knowledge before the summary. The correct knowledge summary is the trusted database domain knowledge.

[0134] Fact consistency check means that the summary generated by the large language model may be inconsistent with the content before the summary in fact, so this embodiment needs to check and modify the knowledge summary to ensure its fact consistency.

[0135] Step 2: System-level resource configuration tuning. Use the big language model to convert the trusted database domain knowledge into a unified structured view. Based on this view, optimize the search space from the two levels of dimension and value range. Use the Bayesian optimization framework from coarse-grained to fine-grained to search the optimized space to find the optimal resource configuration. Specifically, it involves the following four sub-steps:

[0136] Step 2.1: Use a large language model to transform the trusted database domain knowledge obtained in step 1 into a structured view for machine use, and use a prompt integration algorithm based on a large language model to address the impact of the large language model's fragility and hallucination problems during the conversion process.

[0137] The construction of structured views extracts the values ​​of four types of attributes, namely recommended value, recommended maximum value, recommended minimum value, and special value, from the unstructured trusted database domain knowledge, and then stores these four types of attribute values ​​in a structured data format.

[0138] The fragility of large language models, where slight modifications to the prompts can lead to large changes in the output of the large language model.

[0139] The prompt integration algorithm based on the large language model, that is, by integrating multiple imperfect but effective results, a more reliable result is obtained. Specifically, this embodiment needs to prepare multiple different prompt words, each of which uses a template of multi-sample learning (few-shots learning) to utilize the contextual learning ability of the large language model. Then this embodiment randomly samples cases from the case library prepared in advance, and provides different samples for each prompt word template to generate different prompt words. Different prompt words will produce different results. Finally, the results generated by all prompt words are aggregated through a majority voting algorithm to generate a final more credible result.

[0140] Step 2.2: Based on the structured view constructed in step 2.1, optimize the search space from two levels: the search space dimension and the value range of each dimension. At the dimension level, use the large language model to identify the knobs that have a significant impact on DBMS performance and are most valuable for tuning, so as to reduce the dimensionality of the search space. At the value range level, use the structured knowledge in step 2.1 to optimize the value range of each knob.

[0141] The search space dimension, that is, the number of knobs to be adjusted, is that in this embodiment only the knobs that have a significant impact on the DBMS performance are adjusted, rather than all knobs, so as to achieve space dimensionality reduction.

[0142] Among them, when using the large language model to identify knobs that have a significant impact on DBMS performance, the following four factors are considered: the characteristics of the DBMS product, the characteristics of the workload to be optimized, the performance bottleneck of the query statement, and the dependencies between the knobs.

[0143] The value range of each knob is optimized through the four values ​​of maximum value, minimum value, special value and recommended value in the structured knowledge in step 2.1.

[0144] Among them: the maximum value and the minimum value are for numeric knobs, and the value range of the knob is limited between the maximum value and the minimum value. This value range is much smaller than the default value range officially provided by the database, so as to discard meaningless value ranges that are unlikely to produce good results, thereby reducing resource waste and the number of system crashes.

[0145] Among them, special values ​​refer to the fact that when some knobs take certain values, the impact of the knobs on the operation of the database system is essentially different from the impact under other value ranges. These values ​​are special values. However, since the possibility of sampling this value is very low, a separate Boolean virtual knob is provided to control each knob with a special value. When the Boolean virtual knob takes the value of "yes", the special value is directly taken; when the Boolean virtual knob takes the value of "no", the value is taken in the undetermined value range, so as to avoid missing the best performance when taking special values.

[0146] See step 2.3 for use of suggested values.

[0147] Step 2.3: Guided by the domain knowledge obtained in step 2.1, a coarse-grained to fine-grained Bayesian optimization framework is used to search the search space optimized in step 2.2 to recommend the optimal resource configuration.

[0148] Coarse granularity means compressing the search space in advance and searching only a subspace of the complete space, aiming to make full use of the high efficiency and low cost of coarse search to find a solution set that is not optimal but effective. Specifically, only a "tiny feasible space" is searched using the Bayesian optimization framework. This space is generated by multiplying the recommended values ​​given in the manual. Although this discrete space is tiny, it is generated based on domain knowledge and has important reference value, so it is worth prioritizing the optimization algorithm to search.

[0149] Among them, the Bayesian optimization proxy model M uses random forest, and the acquisition function uses EI (Expected Improvement), the formula is as follows:

[0150] EI(x)=E[max(0,f(x)-f(x + ))].

[0151] In the formula, EI(x): the expected improvement at point x, f(x): the predicted value of the objective function at point x, and f(x+): the maximum value of the objective function observed so far.

[0152] Tiny feasible space, "tiny" means the number of possible values ​​is significantly reduced, "feasible" means the selected value has a good chance of optimizing performance. Specifically, the set of discrete spaces obtained by multiplying the manual recommended value of each numerical knob by a series of multipliers α is the tiny feasible space. The formula for the value of α is as follows:

[0153]

[0154] Among them, β is a scaling factor ranging from 0 to 1, controlling the degree of change, and n candidate values ​​r i is a user-defined coefficient, U is the maximum value (makes the value larger) or the minimum value (makes the value smaller), and V is the recommended value. Specifically, when β is set to 0 and 1, V remains unchanged or expands to the maximum value (minimum value). When β is set between 0 and 1, V moves proportionally to its maximum value or minimum value.

[0155] Fine-grained, that is, using Bayesian optimization to perform an exhaustive search in the space optimized in step 2.2 until the resource limit is reached or the expected performance improvement is achieved, so as to avoid the loss of some important configurations due to space compression in coarse-grained search.

[0156] Among them, the Bayesian optimized proxy model used in fine-grained search follows the proxy model after coarse-grained search iteration.

[0157] In terms of process, the specific steps are as follows:

[0158] Step 2.3.1: First, generate a tiny feasible space based on the structured knowledge in step 2.2, and then perform stratified sampling on the space to generate n knob configuration samples to be evaluated.

[0159] Here, n is the number of samples specified by the user to initialize the surrogate model.

[0160] The stratified sampling of the tiny feasible space is implemented using the Latin Hypercube Sampling (LHS) algorithm.

[0161] Step 2.3.2: Evaluate the n samples obtained in step 2.3.1 in the database management system to obtain the performance indicators corresponding to the knob configurations, and finally apply these n knob configurations and performance indicators to initialize the proxy model of the Bayesian optimization framework.

[0162] Performance indicator data includes: database throughput and database latency indicators.

[0163] Step 2.3.3: Perform local and random sampling in the tiny feasible space, substitute the sample points into the proxy model, calculate the corresponding acquisition function value, and find the candidate point that maximizes the acquisition function value.

[0164] Step 2.3.4: Evaluate the candidate points in the database management system, obtain the performance indicators corresponding to the candidate points, and update the proxy model M with the candidate points and the evaluation results.

[0165] Step 2.3.5: Repeat steps 2.3.3 and 2.3.4 N1 times.

[0166] Wherein, N1 is a rough search threshold predefined by the user.

[0167] Step 2.3.6: Use the coarse-grained search agent model M to search the heterogeneous space.

[0168] Among them, the heterogeneous space is the optimized space obtained in step 2.2.

[0169] Step 2.3.7: Repeat steps 2.3.3 and 2.3.4 until resource limitations are reached or the expected performance improvement is achieved.

[0170] Step 3: Query-level execution plan tuning. Use a large language model-based screening algorithm to select knobs that control the query statement execution plan, then use a large language model-based space pruning algorithm to eliminate infeasible knob value combinations, and finally use a data distribution-aware contextual Bayesian optimization algorithm to search the pruned space to find the most appropriate configuration for each query statement. Specifically, it involves the following three sub-steps:

[0171] Step 3.1: Using the knob screening algorithm based on the large language model, first select the Boolean knobs that can change the execution plan of the SQL query statement according to the database manual, and then let the large language model screen out the knobs that are more likely to improve the execution performance of the SQL query statement based on domain knowledge and expert experience.

[0172] An execution plan is a series of execution plans generated by the query optimizer of the database system for SQL query statements. These plans describe the operation steps and sequence taken by the database system when executing queries. The execution results of different execution plans are the same, but the time required to complete different execution plans is different, so this embodiment hopes to find the most efficient execution plan.

[0173] The process of selecting Boolean knobs that change the execution plan of SQL query statements is to use a screening algorithm based on a large language model to select Boolean knobs that can change the query plan from hundreds of knobs according to the database manual. Take PostgreSQL as an example: the knobs that control the physical implementation of the JOIN operator: {"enable_nestloop", "enable_hashjoin", "enable_mergejoin"}. The knobs that control the SCAN operator: {"enable_seqscan", "enable_indexscan", "enable_indexonlyscan"}.

[0174] The knobs selected above can generate different knob value combinations. The query statement optimizer of the database system will generate different execution plans under different value combinations, and different execution plans will bring different execution performances.

[0175] Filter the knobs that are more likely to improve the execution performance of SQL query statements, that is, only adjust the knobs that have a significant impact on the execution performance of query statements instead of all knobs, thereby reducing the dimension of the search space, accelerating the convergence speed of the optimizer, and improving the convergence effect.

[0176] Step 3.2: Use the space pruning algorithm based on the large language model to generate rules based on domain knowledge, logical reasoning, knob characteristics, and dependencies between knobs. Then prune the space based on the rules to eliminate infeasible knob value combinations that have no significant impact on performance.

[0177] The spatial pruning algorithm uses the text understanding and semantic analysis capabilities of the large language model to allow the language model to generate rules based on the characteristics of the knobs and the dependencies between the knobs, and prune the search space based on the rules. Specifically:

[0178] Input: The knobs selected in step 3.1

[0179] Output: The knob value combinations that should be discarded

[0180] Among them, the generation rule is that there are some knobs. According to the characteristics of the knobs and the dependencies between the knobs, it can be known that these knobs should not be activated at the same time or not activated at the same time. The generation rule is generated by finding such knob value pairs.

[0181] Prune the space according to the rules, that is, prune the search space corresponding to the rules according to the generated combinations, exclude infeasible knob value combinations, and thus reduce the search space to facilitate the next step. Formal expression, for k Boolean parameters, the size of the complete search space is 2 k , after the large language model gives m value combinations, the optimized space size is 2 k -m.

[0182] Step 3.3: Use a data distribution-aware contextual Bayesian optimization algorithm to search the pruned space in step 3.2 to find the most appropriate configuration for each query statement.

[0183] The contextual Bayesian optimization algorithm that can perceive data distribution, that is, in the process of constructing the Bayesian optimization proxy model, this embodiment additionally considers the contextual factor of data distribution characteristics, and then adjusts the optimization process according to the data distribution characteristics, thereby improving the efficiency and performance of the optimization.

[0184] The data distribution refers to the data distribution state of multiple tables in a single database, which state will change over time. This changing data distribution is the context that needs to be considered when optimizing a single query statement in the present invention.

[0185] The proxy model for Bayesian optimization is a Gaussian process that takes context into account, and the remaining steps are the same as step 2.3. In the Gaussian process, the kernel function is a function that describes the similarity between points in the input space, and it is hoped that the kernel function can be used to capture the contextual characteristics, that is, the changing data distribution. Therefore, this embodiment defines a product kernel function for each query statement.

[0186] The product kernel function is the product of two kernel functions k1 and k2, and is defined as:

[0187]

[0188] in, It is the Boolean knob configuration corresponding to the SQL statement. is the data distribution corresponding to the SQL statement, The SQL statement is in another data distribution Boolean knob configuration below.

[0189] Among them, k1 is the kernel function for modeling the space of knob value combinations. Since the space formed by the knob value combinations is discrete, the Hamming kernel function suitable for processing discrete space data is selected.

[0190] Among them, k2 is the kernel function for modeling the space of data distribution. Since the space is heterogeneous, the Matérn kernel function is selected because it has flexibility, smoothness, adjustability and generalization ability for processing data distribution.

[0191] This embodiment discloses a hierarchical database tuning method and system based on a large language model and Bayesian optimization. The present invention aims to solve the problem that a large amount of resources are wasted to explore the space in a high-dimensional heterogeneous search space in the field of database tuning, and only configurations that produce suboptimal performance can be found. The present invention includes:

[0192] Knowledge preprocessing stage: (1) This embodiment proposes a data pipeline based on a large language model to collect multi-source heterogeneous noisy domain knowledge. (2) This embodiment uses a large language model to denoise the database domain knowledge to obtain denoised database domain knowledge. (3) This embodiment generates a knowledge summary for conflicting multi-source knowledge according to the credibility of different sources. (4) This embodiment uses a large language model to perform a factual consistency check on the knowledge before and after the summary to ensure that the generated credible knowledge summary is consistent with the knowledge before and after the summary.

[0193] First level, system-level resource configuration optimization: (1) This embodiment proposes a prompt integration algorithm based on a large language model to convert refined knowledge into a structured view so that it can be used by the machine. (2) Based on the constructed view, this embodiment optimizes the search space from two levels: dimension and value range. (3) This embodiment proposes a Bayesian optimization framework from coarse-grained to fine-grained to search the optimized space and find the optimal resource configuration.

[0194] The second level is query-level execution plan tuning: (1) This embodiment proposes a screening algorithm based on a large language model to select Boolean knobs that control the query statement execution plan and have greater tuning value. (2) This embodiment proposes a space pruning algorithm based on a large language model to generate rules to exclude infeasible knob value combinations. (3) This embodiment proposes a contextual Bayesian optimization algorithm that can perceive data distribution to search the pruned space and find the most appropriate configuration for each query statement. It belongs to the field of database tuning.

[0195] Example 2

[0196] This embodiment provides a hierarchical database tuning system, including:

[0197] A trusted knowledge acquisition module, used to determine trusted database domain knowledge; the trusted database domain knowledge is obtained by using a large language model to convert multi-source heterogeneous noisy data obtained through a data pipeline based on a large language model into trusted knowledge;

[0198] A system-level configuration optimization module, which is used to transform the trusted database domain knowledge into structured knowledge using a prompt integration algorithm to optimize the search space, and to determine the optimal system configuration using a coarse-grained-fine-grained Bayesian optimization algorithm to complete system-level configuration optimization; the structured knowledge is knowledge that has been transformed into a structured data format that can be used by a computer;

[0199] The query level configuration optimization module is used to determine the optimal execution plan for each query statement using parameter screening algorithm, spatial pruning algorithm and contextual Bayesian optimization algorithm to complete query level configuration optimization

[0200] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0201] The principles and implementation methods of the present invention are described herein using specific examples. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those of ordinary skill in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A hierarchical system tuning method based on a large language model, characterized in that: include: Identify trusted database domain knowledge; The trusted database domain knowledge is obtained by using a large language model to convert multi-source heterogeneous noisy data obtained through a data pipeline based on a large language model into trusted knowledge; The trusted database domain knowledge is converted into structured knowledge by using a prompt integration algorithm to optimize the search space, and the coarse-grained-fine-grained Bayesian optimization algorithm is used to determine the optimal system configuration and complete the system level configuration optimization; the structured knowledge is the knowledge that has been converted into a structured data format that can be used by a computer; The parameter screening algorithm, spatial pruning algorithm and contextual Bayesian optimization algorithm are used to determine the optimal execution plan for each query statement and complete the query hierarchical configuration optimization.

2. The hierarchical system tuning method based on a large language model according to claim 1, characterized in that: Identify trusted database domain knowledge, including: Acquire multi-source database domain knowledge from multiple knowledge sources; knowledge sources include but are not limited to system manuals, expert experience and large language models; Use a large language model to denoise the domain knowledge of multi-source databases, filter out unreasonable knowledge, and obtain denoised database domain knowledge; The multi-source database domain knowledge is summarized by using a large language model, and a knowledge summary is generated for each parameter; the summarizing of the multi-source database domain knowledge includes: abandoning low-credibility knowledge, retaining high-credibility knowledge, and generating a knowledge summary of knowledge from different sources but compatible with each other; A large language model is used to check the factual consistency of knowledge summaries, and knowledge summaries that pass the consistency check are determined to be credible database domain knowledge.

3. The hierarchical system tuning method based on a large language model according to claim 1, characterized in that: The trusted database domain knowledge is converted into structured knowledge using a prompt integration algorithm to optimize the search space, and the coarse-grained-fine-grained Bayesian optimization algorithm is used to determine the optimal system configuration and complete the system-level configuration optimization, including: According to the trusted database domain knowledge, a structured view is constructed by using a prompt integration algorithm based on a large language model; the prompt integration algorithm provides prompt words for solving the same task but with different contents to the large language model, and then integrates different results generated by different prompt words into a trusted result; the structured view is constructed by extracting values ​​of four types of attributes, namely, recommended value, recommended maximum value, recommended minimum value and special value, from the unstructured trusted database domain knowledge, and then storing the recommended value, recommended maximum value, recommended minimum value and special value in a structured data format; Based on the structured view, the search space is optimized from two sub-levels: dimension and value range, to obtain an optimized search space; The coarse-grained-fine-grained Bayesian optimization algorithm is used to search the optimization search space, determine the optimal configuration, and complete the system-level configuration optimization.

4. The hierarchical system tuning method based on a large language model according to claim 3 is characterized in that: Based on the structured view, the search space is optimized from two sub-levels: dimension and value range, to obtain an optimized search space, including: Based on the structured view, a large language model is used to identify and adjust a target knob in the complete search space, thereby obtaining a reduced-dimensional search space; the target knob has an impact on database performance that reaches a preset level and has the greatest tuning value; Determine any knob in the dimension reduction search space as the current knob; Determine the optimal value range of the current knob; Update the current knob and return to step "determine any knob in the dimensionality reduction search space as the current knob" until the dimensionality reduction search space is traversed to obtain the optimized search space.

5. The hierarchical system tuning method based on a large language model according to claim 4 is characterized in that: Determine the optimal value range of the current knob, including: For a numeric knob, a preset maximum value is obtained from the structured view to replace the current knob default maximum value officially provided by the database; the preset maximum value is less than or equal to the current knob default maximum value officially provided by the database; For a numeric knob, a preset minimum value is obtained from the structured view to replace the current knob default minimum value officially provided by the database; the preset minimum value is greater than or equal to the current knob default minimum value officially provided by the database; Determine that the interval enclosed by the preset minimum value and the preset maximum value is the current knob pending value range; Determine whether the current knob has a special value and obtain a determination result; when the knob has a special value, the influence of the knob on the operation of the database system is essentially different from the influence in other value ranges; If the judgment result is no, determining the undetermined value range as the optimized value range of the current knob; If the judgment result is yes, a separate Boolean virtual knob is added to the current search space; when the Boolean virtual knob value is "yes", a special value is directly taken; when the Boolean virtual knob value is "no", a value is taken in the pending value range.

6. The hierarchical system tuning method based on a large language model according to claim 4 is characterized in that: The optimization search space is searched using a coarse-grained-fine-grained Bayesian optimization algorithm to determine the optimal configuration and complete the system-level configuration optimization, including: Determining two levels of sub-feasible spaces according to the structured view; the sub-feasible spaces include a coarse-grained sub-feasible space and a fine-grained sub-feasible space; A Latin hypercube sampling algorithm is used to perform stratified sampling on the coarse-grained sub-feasible space to generate a user-specified number of knob configuration samples; Evaluate the knob configuration sample in the database management system to obtain the performance index corresponding to the knob configuration sample; Initializing a proxy model of a Bayesian optimization framework using a plurality of the knob configuration samples and their corresponding performance indicators; Let the number of iterations n = 1; Let the current sub-feasible space be the coarse-grained sub-feasible space; Perform local sampling and random sampling in the current sub-feasible space, substitute multiple sampling points into the proxy model respectively, and determine the estimated objective function value and corresponding uncertainty of each sampling point; According to the estimated objective function value and uncertainty, the point corresponding to the maximum function value is determined as the candidate point; Evaluate the candidate points in the database management system to obtain performance indicators corresponding to the candidate points; Using the candidate points and their corresponding performance indicators, the proxy model at the nth iteration is updated to obtain the proxy model at the n+1th iteration; Increase the value of the number of iterations n by 1, and return to step "perform local sampling and random sampling in the current sub-feasible space, substitute multiple sampling points into the proxy model, and determine the estimated objective function value and corresponding uncertainty of each sampling point" until the value of the number of iterations n reaches the preset number of iterations N1; The current sub-feasible space will be made a fine-grained sub-feasible space; Repeat steps "perform local sampling and random sampling in the current sub-feasible space, substitute multiple sampling points into the proxy model, and determine the estimated objective function value and corresponding uncertainty of each sampling point" to step "increase the value of the number of iterations n by 1" until the number of iterations n reaches the resource limit N2; According to the optimization records, the configuration that optimizes the performance indicators is determined as the optimal configuration, and the system-level configuration optimization is completed.

7. The hierarchical system tuning method based on a large language model according to claim 6 is characterized in that: The method for determining the coarse-grained sub-feasible space comprises: Determine any knob as the current knob; The discrete value set is obtained by multiplying the recommended value of the current knob in the structured view by a set of scaling multipliers α; the scaling multipliers are: Where β is the scaling factor, ranging from 0 to 1, r1 is the candidate value of the first coefficient predefined by the user, r2 is the candidate value of the second coefficient predefined by the user, and r n is the candidate value of the nth coefficient predefined by the user, r i is the candidate value of the i-th coefficient predefined by the user, U is the preset maximum or minimum value obtained from the structured view, and V is the recommended value in the structured view; Determine that the space obtained by the set of discrete value groups corresponding to all knobs is the coarse-grained sub-feasible space; The fine-grained sub-feasible space is a search space formed based on the value range of each knob.

8. The hierarchical system tuning method based on a large language model according to claim 1, characterized in that: Utilize parameter screening algorithm, spatial pruning algorithm and contextual Bayesian optimization algorithm to determine the optimal execution plan for each query statement and complete query level configuration optimization, including: Using a large language model-based screening algorithm, according to the database manual, determine the target Boolean knob in the optimization search space; the target Boolean knob can change the execution plan of the query statement; By using a large language model, the knobs that have a significant impact on the execution performance of the query statement are selected from the above Boolean knobs based on domain knowledge and expert experience, and a reduced-dimensional search space is obtained; Construct multiple knob combinations based on the selected knobs; Based on the space pruning algorithm, the knob rules generated by the large language model are used to delete the infeasible knob combinations in the dimensionality reduction search space and the knob combinations whose impact on performance does not reach the preset level, and the pruning space is obtained; The contextual Bayesian optimization algorithm is used to search the pruning space, determine the optimal execution plan for each query statement, and complete the query hierarchy configuration optimization; the context refers to the distribution of multi-table data in the database, which changes over time.

9. A hierarchical system tuning system based on a large language model, characterized in that: include: Trusted knowledge acquisition module, used to determine trusted database domain knowledge; The trusted database domain knowledge is obtained by using a large language model to convert multi-source heterogeneous noisy data obtained through a data pipeline based on a large language model into trusted knowledge; A system-level configuration optimization module, which is used to transform the trusted database domain knowledge into structured knowledge using a prompt integration algorithm to optimize the search space, and to determine the optimal system configuration using a coarse-grained-fine-grained Bayesian optimization algorithm to complete system-level configuration optimization; the structured knowledge is knowledge that has been transformed into a structured data format that can be used by a computer; The query level configuration optimization module is used to determine the optimal execution plan for each query statement by using parameter screening algorithm, spatial pruning algorithm and contextual Bayesian optimization algorithm to complete query level configuration optimization.

Images