Database configuration parameter tuning methods based on load characterization and large model exploration
By characterizing the database workload and utilizing historical knowledge bases, combined with large model inference, we have achieved efficient database configuration parameter tuning, solving the problem of low efficiency in existing methods and reducing tuning costs.
Patent Information
- Application Number
- CN202510799589.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-16
- Publication Date
- 2026-11-10
- Estimated Expiration
- 2045-06-16
Smart Images

Figure CN120821715B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of database system configuration parameter tuning, specifically, to a database configuration parameter tuning method based on load characterization and large model exploration. Background Technology
[0002] Modern database systems typically offer hundreds of configurable parameters to adapt to different business needs, hardware environments, and workload characteristics. Different combinations of configuration parameters can lead to variations in database system performance. Achieving high-performance database services is crucial for improving business efficiency and reducing resource consumption. Therefore, research on the optimization of database system configuration parameters has significant practical value.
[0003] The goal of database system configuration parameter tuning is to find the optimal or near-optimal combination of configuration parameters from a high-dimensional and complex configuration space to meet the database performance requirements of specific business scenarios. Due to the continuous expansion of database system scale and the increasing complexity of application scenarios, traditional database tuning methods based on expert experience and rules are increasingly difficult to meet real-world needs. Researchers are dedicated to designing more efficient tuning methods. Based on the core ideas and technical means of tuning methods, existing research can be mainly divided into three categories: search-based methods, learning-based methods, and natural language processing-based methods. The existing technologies have the following problems:
[0004] First, search-based database system configuration parameter tuning methods do not require building performance prediction models and are directly effective, but the huge configuration space will lead to a high consumption of time and resources. In addition, most methods do not consider the constraints between configuration parameters during the search process, resulting in a large number of configuration samples that violate domain knowledge rules. Moreover, most methods cannot utilize historical tuning knowledge and need to start searching from scratch for new tuning tasks, which is inefficient.
[0005] Second, traditional machine learning-based database system configuration tuning methods predict database system performance by building performance models, thus replacing the actual execution of workloads and significantly reducing the resource overhead caused by repeatedly executing workloads during the exploration process. However, existing methods also have problems such as strong empirical reliance on parameter selection, the need for a large number of high-quality samples to build performance models, and dynamic changes in workloads. Deep reinforcement learning-based database system configuration tuning methods do not require collecting a large number of samples to train performance models, but they suffer from slow convergence speed, with each round of tuning typically requiring hundreds of iterations. Therefore, they are only suitable for scenarios with long tuning times and have weaker versatility than traditional machine learning-based methods.
[0006] Third, the database system configuration parameter optimization method based on natural language processing uses large language models to extract effective information from domain knowledge, accelerates the convergence of configuration parameter tuning algorithms, and improves tuning performance. However, this direction is still in the early exploration stage and still has some shortcomings: (1) Some work requires fine-tuning of the model, which will lead to additional resource consumption, whether in the initialization stage or the tuning stage; (2) The domain knowledge is not fully utilized. Most work only extracts the default value, suggested value, and range information of parameters, while paying less attention to the dependency relationship and constraint conditions between parameters. This will sample and explore a large number of configurations that violate the dependency relationship between parameters, resulting in resource waste; (3) The dynamic changes of workload are not considered. The tuning process is restarted for each new workload, and the experience in the historical tuning tasks cannot be effectively utilized, resulting in low efficiency and resource waste; (4) The natural language understanding and reasoning ability of large models are not fully utilized. Most work is limited to using large language models to complete knowledge extraction work, but does not use large language models for knowledge reasoning. However, the latest research progress of large language models shows its great potential in reasoning. Summary of the Invention
[0007] To overcome at least one deficiency in the prior art, this application provides a database configuration parameter tuning method based on load characterization and large model exploration.
[0008] Firstly, a method for tuning database configuration parameters based on load characterization and large model exploration is provided, including:
[0009] The database workload is encoded using a hierarchical Transformer encoder to obtain a workload representation. The database workload includes N SQL statements. The hierarchical Transformer encoder includes a statement-level encoder and a load-level aggregator. Each SQL statement is encoded based on the statement-level encoder to generate statement-level semantic features. All statement-level semantic features are merged based on the load-level aggregator to generate workload features.
[0010] Obtain the historical tuning knowledge base, which stores multiple historical workload features and their high-performance parameter configurations; calculate the cosine similarity between the workload features and each historical workload feature in the historical tuning knowledge base; select the high-performance parameter configurations corresponding to the top K historical workload features with the highest cosine similarity to form the initial parameter configuration space.
[0011] The Latin hypercube sampling method is used to randomly and uniformly sample in the initial parameter configuration space to obtain the sampled parameter configuration space; a high-performance parameter configuration is generated based on the large model knowledge reasoning method for the sampled parameter configuration space.
[0012] In one embodiment, each SQL statement is encoded based on a statement-level encoder to generate statement-level semantic features, including:
[0013] Break the SQL statement into multiple tokens;
[0014] Each token is embedded and positionally encoded to obtain the input vector of the SQL statement;
[0015] A statement-level encoder is used to encode the input vector of the SQL statement to obtain the statement-level semantic features of the SQL statement; the statement-level encoder includes L Transformer layers.
[0016] In one embodiment, workload features are generated by fusing all statement-level semantic features based on a load-level aggregator, including:
[0017] A load-level aggregator is used to capture the dependencies between all statement-level semantic features, resulting in statement-level semantic features with dependencies. The load-level aggregator consists of L Transformer layers.
[0018] The workload features are obtained by average pooling the semantic features of statements with dependencies.
[0019] In one embodiment, the hierarchical Transformer encoder is a trained encoder, and the training process includes:
[0020] A training model is constructed, which includes a hierarchical Transformer encoder and a load reconstruction decoder; the load reconstruction encoder is used to generate each term of each SQL statement based on the workload features output by the hierarchical Transformer encoder.
[0021] Obtain the training dataset; the samples in the training dataset constitute the database workload.
[0022] The training model is trained based on the training dataset to obtain the trained hierarchical Transformer encoder.
[0023] In one embodiment, the load reconstruction decoder is used for:
[0024] The workload features output by the hierarchical Transformer encoder are extended to a temporal dimension and used as a global memory vector.
[0025] By introducing positional encoding into the global memory, we obtain the position-encoded global memory vector;
[0026] The position-encoded global memory vector is input into the Transformer decoder, which includes a masked self-attention mechanism and a cross-attention mechanism. The position-encoded global memory vector is input into the masked self-attention mechanism and the cross-attention mechanism respectively. The output of the masked self-attention mechanism is used as the query of the cross-attention mechanism, and the position-encoded global memory vector is used as the key and value of the cross-attention mechanism. The output of the cross-attention mechanism includes each token of each SQL statement.
[0027] Secondly, a database configuration parameter tuning device based on load characterization and large model exploration is provided, including:
[0028] The workload characterization module is used to encode the database workload using a hierarchical Transformer encoder to obtain a workload characterization. The database workload includes N SQL statements. The hierarchical Transformer encoder includes a statement-level encoder and a load-level aggregator. The statement-level encoder encodes each SQL statement to generate statement-level semantic features. The load-level aggregator merges all statement-level semantic features to generate workload features.
[0029] The cold start module is used to acquire the historical tuning knowledge base, which stores multiple historical workload features and their high-performance parameter configurations. It calculates the cosine similarity between the workload features and each historical workload feature in the historical tuning knowledge base, and selects the high-performance parameter configurations corresponding to the top K historical workload features with the highest cosine similarity to form the initial parameter configuration space.
[0030] The module explores the use of a Latin hypercube sampling method to randomly and uniformly sample parameters in the initial parameter configuration space to obtain a sampled parameter configuration space. Based on a large model knowledge reasoning method, a high-performance parameter configuration is generated from the sampled parameter configuration space.
[0031] Thirdly, a computer-readable storage medium is provided, which stores a computer program that, when executed by a processor, implements the aforementioned database configuration parameter tuning method based on load characterization and large model exploration.
[0032] Fourthly, a computer program product is provided, including a computer program / instruction, which, when executed by a processor, implements the aforementioned database configuration parameter tuning method based on load characterization and large model exploration.
[0033] Compared with the prior art, this application has the following beneficial effects: The database configuration parameter tuning method based on load characterization and large model exploration in this application can not only use historical workload tuning experience to cold start the tuning process without tuning from scratch for each new workload, but also has high exploration efficiency. After cold start, it can keenly explore potential high-performance configurations and avoid executing too much workload due to inefficient exploration, thereby reducing tuning costs. Attached Figure Description
[0034] This application can be better understood by referring to the description given below in conjunction with the accompanying drawings, which, together with the detailed description below, are incorporated in and form part of this specification. In the drawings:
[0035] Figure 1 A flowchart illustrating a database configuration parameter tuning method based on load characterization and large model exploration is shown.
[0036] Figure 2 The optimization results of each algorithm for TPC-H and TPC-C under different configuration metric costs are shown;
[0037] Figure 3 A block diagram of a database configuration parameter tuning device based on load characterization and large model exploration is shown. Detailed Implementation
[0038] Exemplary embodiments of the present application will be described below with reference to the accompanying drawings. For clarity and brevity, not all features of the actual embodiments are described in the specification. However, it should be understood that many embodiment-specific decisions can be made in the development of any such actual embodiment to achieve the developer’s specific objectives, and these decisions may vary as the embodiments differ.
[0039] It should also be noted that, in order to avoid obscuring this application with unnecessary details, only the device structure closely related to the solution according to this application is shown in the accompanying drawings, while other details that are not closely related to this application are omitted.
[0040] It should be understood that this application is not limited to the described embodiments by virtue of the following description with reference to the accompanying drawings. In this document, embodiments may be combined with each other, features may be substituted or borrowed between different embodiments, and one or more features may be omitted in one embodiment, where feasible.
[0041] This application provides a method for optimizing database configuration parameters based on load characterization and large model exploration. Figure 1 A flowchart illustrating a database configuration parameter tuning method based on load characterization and large model exploration is shown. (See [link]). Figure 1 The method mainly includes the following steps:
[0042] Step S1: Encode the database workload using a hierarchical Transformer encoder to obtain a workload representation; the database workload W includes N SQL statements, i.e., W = {q1, q2, ..., q...} i ,…q N}, q i This represents the i-th SQL statement. The hierarchical Transformer encoder includes a statement-level encoder and a workload-level aggregator. The statement-level encoder encodes each SQL statement to generate statement-level semantic features, and the workload-level aggregator merges all statement-level semantic features to generate workload features.
[0043] Database workloads typically consist of a set of SQL statements describing database operations such as queries, updates, and inserts. In practice, workloads are dynamic, and different workloads often have different optimal database configurations. Tuning each new workload from scratch would inevitably waste time and resources. Therefore, this embodiment leverages the experience gained from similar historical workload tuning tasks to improve the tuning efficiency of target workloads. To quickly and accurately find the most similar workload in historical data, the workload needs to be characterized.
[0044] The hierarchical Transformer encoder first encodes each SQL statement in the workload individually based on the statement-level encoder to generate statement-level semantic features, and then uses a workload-level aggregator to merge all statement features to generate workload features.
[0045] Step S2: Obtain the historical tuning knowledge base, which stores multiple historical workload features and their high-performance parameter configurations. Calculate the cosine similarity between the workload features and each historical workload feature in the historical tuning knowledge base. Select the top K (e.g., K=3) historical workload features with the highest cosine similarity to form the initial parameter configuration space.
[0046] Most learning-based database configuration tuning algorithms assume a fixed workload and do not consider the changes that may occur during actual applications. Therefore, most methods typically lack a cold-start mechanism during tuning, starting the process from scratch for each new workload. For example, they might use the database's default configuration as the starting point, or employ Latin hypercube sampling to randomly and uniformly sample a set of points in the configuration space as the initial configuration point set, then combine this with a specific search strategy to find the configuration that achieves optimal database performance. This leads to inefficiency and resource waste in the tuning process for new workloads. To address this issue, this embodiment employs an efficient tuning cold-start mechanism. It constructs a good set of configurations based on tuning experience with similar historical workloads as the starting point for the target workload tuning task, thereby reducing the number of iterations required in subsequent exploration processes and the cost of configuration metrics.
[0047] The historical tuning knowledge base stores multiple historical workload characteristics and their high-performance parameter configurations. When historical data accumulation is lacking, a small number of workloads are generated and several rounds of random searches are performed on each workload. The top K high-performance configurations are selected and added to the knowledge base to complete the knowledge base initialization. During the historical data accumulation process, the top K high-performance configurations discovered during the overall tuning of each workload are added to the knowledge base.
[0048] It should be noted that for OLTP (On-Line Transaction Processing) workloads, since the SQL statements executed are randomly generated according to a set transaction weight distribution, special handling is required during workload characterization and similarity matching. The method adopted in this embodiment is as follows: First, a benchmark test is repeatedly executed 10 times under a fixed transaction weight distribution. From each generated workload, 50 SQL statements are randomly selected as approximations of that workload group. Then, the 10 extracted workloads are characterized to obtain corresponding feature vectors. The average of these feature vectors yields the feature vector of the OLTP workload under the transaction weight distribution. Finally, this feature vector is used for similarity matching.
[0049] Step S3: The Latin hypercube sampling method is used to randomly and uniformly sample in the initial parameter configuration space to obtain the sampled parameter configuration space; the high-performance parameter configuration is generated based on the large model knowledge reasoning method.
[0050] This embodiment leverages historical workload tuning experience to cold-start the tuning process without having to tune every new workload from scratch. It also boasts high exploration efficiency, enabling it to readily explore potential high-performance configurations after the cold start, avoiding excessive workload execution due to inefficient exploration and thus reducing tuning costs. This embodiment can initiate the tuning process of the target workload from several favorable initial configuration points and efficiently explore potentially high-performance configurations within the configuration space, effectively improving the convergence speed of tuning and reducing time and resource consumption.
[0051] In one embodiment, step S1 involves encoding each SQL statement based on a statement-level encoder to generate statement-level semantic features, including:
[0052] First, break the SQL statement down into multiple tokens; for example, break the SQL statement q down into multiple tokens. i Split into multiple word units t i,1 ,t i,2 ,…t i,j ,…t i,M , t i,j The statement q i The j-th word, M is the sentence q i The number of tokens. Add the start and end position special symbols [CLS_i] and [SEP_i] to this.
[0053] Then, token embedding and positional encoding are performed on each token to obtain the input vector x of the SQL statement. i =(x i,1 ,x i,2 ,…x i,j …,x i,M );x i,j For word t i,j The result after word embedding and positional encoding.
[0054] Then, a statement-level encoder is used to process the SQL statement q. i input vector x i Encode the SQL statement q. i Statement-level semantic features i The statement-level encoder consists of L Transformer layers, where L can be 2.
[0055] In one embodiment, step S1, based on the load-level aggregator, fuses all statement-level semantic features to generate workload features, including:
[0056] A load-level aggregator is used to capture the dependencies between all statement-level semantic features, resulting in statement-level semantic features with dependencies. The load-level aggregator consists of L Transformer layers, where L can be 2.
[0057] The workload feature z is obtained by average pooling the statement-level semantic features with dependencies.
[0058] In one embodiment, the hierarchical Transformer encoder is a trained encoder, and the training process includes:
[0059] First, a training model is built, which includes a hierarchical Transformer encoder and a load reconstruction decoder. The load reconstruction encoder is used to generate each term of each SQL statement based on the workload features output by the hierarchical Transformer encoder.
[0060] Specifically, the load reconstruction decoder is used for:
[0061] The workload feature z output by the hierarchical Transformer encoder is expanded to a temporal dimension and used as a global memory vector z. ′ ;
[0062] Considering the global memory vector z ′ It is the same at every time step, lacks dynamism, therefore the global memory vector z ′ By introducing positional encoding, the keys and values at different time steps differ, resulting in a positionally encoded global memory vector; the positionally encoded global memory vector z at time step t. t ′ ;
[0063] The position-encoded global memory vector is input into the Transformer decoder, which includes a masked self-attention mechanism and a cross-attention mechanism. The position-encoded global memory vector is input into both the masked self-attention mechanism and the cross-attention mechanism. The output of the masked self-attention mechanism serves as the query for the cross-attention mechanism, and the position-encoded global memory vector serves as the key and value for the cross-attention mechanism. The output of the cross-attention mechanism includes each token of each SQL statement.
[0064] Then, obtain the training dataset; the samples in the training dataset are the database workload;
[0065] Then, the training model is trained based on the training dataset to obtain the trained hierarchical Transformer encoder.
[0066] Here, to train the designed encoder-decoder model so that the features output by the encoder accurately characterize the workload, different types of learning tasks can be designed. In this embodiment, to reduce the burden of data collection, a workload reconstruction task drives the encoder-decoder training. That is, the encoder inputs the SQL sequence contained in the workload, and the decoder regressively reconstructs the complete SQL statement sequence. During training, cross-entropy loss is calculated based on the predicted probability distribution and the true probability distribution.
[0067] To further verify the effectiveness of the method in this application, a comparative experiment was set up, using a MySQL database as the object of configuration parameter tuning, and comparing the method in this application (hereinafter referred to as EfficTune) with representative database system configuration parameter tuning algorithms.
[0068] Example
[0069] Experimental Datasets: Two representative benchmark datasets in the field of database performance testing were selected: TPC-H and TPC-C. (1) The TPC-H benchmark dataset is an OLAP workload that simulates a decision support system for the retail industry. It contains 8 related tables and 22 complex analytical SQL statements, covering typical OLAP operations such as aggregation, multi-table joins, subqueries, and expression calculations. The TPC-H database with a scale factor of 1 was used. (2) The TPC-C benchmark dataset is an online transaction processing (OLTP) database performance benchmark that simulates 5 core transaction types in a typical warehouse management business scenario. It can evaluate the transaction processing capabilities of the database system. The TPC-C database containing 5 warehouses was used.
[0070] Comparison Algorithms: (1) BestConfig: A search-based configuration parameter tuning method that proposes divide-and-diverge sampling and recursive bound-and-search search methods to improve the efficiency of searching for the best configuration parameters. (2) RGHOC: A classic machine learning-based configuration parameter optimization method that uses a random forest model as a performance prediction model and combines the performance prediction model with a genetic algorithm to find the configuration parameters that produce the highest performance. (3) SMAC: A parameter tuning method based on Bayesian optimization that uses a random forest as a surrogate model and is widely used in parameter tuning. (4) QTune: A database system configuration tuning method based on deep reinforcement learning that considers the rich features of the workload and inputs the query features into the DRL model to select the most suitable configuration.
[0071] Performance comparison metrics: (1) For OLAP type workloads TPC-H, the degree of latency reduction (LR) of database execution workloads after configuration parameter tuning is used as a performance comparison metric between different algorithms, which can be defined as:
[0072]
[0073] Where L baseline L represents the latency of the workload after different algorithm configurations and optimizations. defeult This is the latency of the workload under the default configuration of the database system.
[0074] (2) For OLTP-type workloads like TPC-C, the degree of database throughput improvement (TI) after configuration parameter tuning is used as a performance comparison indicator between different algorithms, and it is defined as:
[0075]
[0076] Where T baseline This refers to the database throughput (number of transactions per second, tx / s) after different algorithm configurations and optimizations. dafault This refers to the database throughput under the default database system configuration.
[0077] Experimental Results: The specific performance improvements obtained by different algorithms using three configuration metrics to optimize database configuration parameters for TPC-H and TPC-C workloads are shown in Tables 1 and 2.
[0078] Table 1 Comparison of algorithm performance under cost constraints for different TPC-H workload configurations
[0079]
[0080] Table 2 Comparison of algorithm performance under different sampling cost constraints for TPC-C workloads.
[0081]
[0082] Figure 2 The optimization results of each algorithm for TPC-H and TPC-C under different configuration metric costs are shown. Figure 2As can be seen, EfficTune achieved the highest latency reduction (LR) and throughput improvement (TI) on both TPC-H and TPC-C workloads. Calculations show that on TPC-H, EfficTune achieved average latency reductions of 13.90%, 21.43%, 10.13%, and 8.03% compared to BestConfig, RGHOC, QTune, and SMAC, respectively. On TPC-C, EfficTune achieved higher throughput reductions of 15.60%, 18.33%, 9.47%, and 9.80% compared to BestConfig, RGHOC, QTune, and SMAC, respectively. This demonstrates that EfficTune's overall tuning performance is superior to other methods.
[0083] On the other hand, from Figure 2 The results also show that EfficTune, on both the TPC-H and TPC-C workloads, achieves performance improvements close to or even exceeding those of other algorithms using a configuration metric cost of CM=600, with a configuration metric cost of CM=200. This indicates that EfficTune achieves the same tuning effect at a lower cost. Firstly, EfficTune's cold-start mechanism enables the tuning process to begin from several good initial configuration points, effectively reducing the number of iterations required in the exploration phase. In contrast, BestConfig, RFHOC, and SMAC all require tuning from scratch, lacking the utilization of historical tuning data, thus incurring higher tuning costs for new tuning tasks. Secondly, EfficTune also leverages the reasoning capabilities of a large language model to generate potentially promising configurations from already explored high-performance configurations, achieving higher exploration efficiency.
[0084] Employing the same inventive concept as the database configuration parameter tuning method based on load characterization and large model exploration, this embodiment also provides a corresponding database configuration parameter tuning device based on load characterization and large model exploration. Figure 3 The diagram shows the structural block diagram of a database configuration parameter tuning device based on load characterization and large model exploration, including:
[0085] The workload characterization module is used to encode the database workload using a hierarchical Transformer encoder to obtain a workload characterization. The database workload includes N SQL statements. The hierarchical Transformer encoder includes a statement-level encoder and a load-level aggregator. The statement-level encoder encodes each SQL statement to generate statement-level semantic features. The load-level aggregator merges all statement-level semantic features to generate workload features.
[0086] The cold start module is used to acquire the historical tuning knowledge base, which stores multiple historical workload features and their high-performance parameter configurations. It calculates the cosine similarity between the workload features and each historical workload feature in the historical tuning knowledge base, and selects the high-performance parameter configurations corresponding to the top K historical workload features with the highest cosine similarity to form the initial parameter configuration space.
[0087] The module explores the use of a Latin hypercube sampling method to randomly and uniformly sample parameters in the initial parameter configuration space to obtain a sampled parameter configuration space. Based on a large model knowledge reasoning method, a high-performance parameter configuration is generated from the sampled parameter configuration space.
[0088] The database configuration parameter tuning device based on load characterization and large model exploration in this embodiment has the same inventive concept as the database configuration parameter tuning method based on load characterization and large model exploration described above. Therefore, the specific implementation of this device can be found in the embodiment section of the database configuration parameter tuning method based on load characterization and large model exploration described above, and its technical effects correspond to the technical effects of the above method, so it will not be repeated here.
[0089] This application provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the above-described database configuration parameter tuning method based on load characterization and large model exploration.
[0090] This application provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the aforementioned database configuration parameter tuning method based on load characterization and large model exploration.
[0091] In summary, this application has the following technical effects:
[0092] This application leverages experience and data from past tuning tasks, combining exploration and exploitation principles with the natural language understanding and reasoning capabilities of a large language model to improve the efficiency of finding optimal configuration parameters for new workloads. First, using NLP representation learning techniques, a hierarchical Transformer encoder and workload reconstruction decoder are employed to transform complex workloads into low-dimensional feature vector representations. Then, a tuning cold start is performed, utilizing the similarity between workload feature representations to match new workloads with similar historical workloads. The optimal configurations of these similar historical workloads are used as the initial configuration set, avoiding tuning from scratch and saving time and resources. Next, a large model enhancement exploration and exploitation process is implemented. Combining exploration and exploitation principles, the configuration of new workloads is further optimized based on the initial configuration. The exploration phase uses a Latin hypercube sampling method to uniformly sample in the configuration space, while the exploitation phase generates potential high-performance configurations through large model knowledge reasoning. Through these approaches, this application can initiate the tuning process of a target workload from several favorable initial configuration points and efficiently explore potential configurations in the configuration space, effectively improving the convergence speed of tuning and reducing time and resource consumption.
[0093] The above descriptions are merely various embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
Claims
1. A database configuration parameter tuning method based on load characterization and large model exploration, characterized in that, include: The database workload is encoded using a hierarchical Transformer encoder to obtain a workload representation; the database workload includes N SQL statements; the hierarchical Transformer encoder includes a statement-level encoder and a load-level aggregator. Each SQL statement is encoded based on the statement-level encoder to generate statement-level semantic features, and all statement-level semantic features are merged based on the load-level aggregator to generate the workload feature. Obtain a historical tuning knowledge base, which stores multiple historical workload features and their high-performance parameter configurations; calculate the cosine similarity between the workload features and each historical workload feature in the historical tuning knowledge base; select the high-performance parameter configurations corresponding to the top K historical workload features with the highest cosine similarity to form an initial parameter configuration space. The Latin hypercube sampling method is used to randomly and uniformly sample in the initial parameter configuration space to obtain the sampling parameter configuration space; The sampling parameter configuration space is based on a large model knowledge reasoning method to generate high-performance parameter configurations; Each SQL statement is encoded based on the statement-level encoder to generate statement-level semantic features, including: The SQL statement is broken down into multiple tokens; Each lexical is embedded and positionally encoded to obtain the input vector of the SQL statement; The statement-level encoder is used to encode the input vector of the SQL statement to obtain the statement-level semantic features of the SQL statement; the statement-level encoder includes L Transformer layers; Based on the load-level aggregator, all statement-level semantic features are fused to generate the workload features, including: A load-level aggregator is used to capture the dependencies between all statement-level semantic features, resulting in statement-level semantic features with dependencies; the load-level aggregator includes L Transformer layers; The workload features are obtained by average pooling the statement-level semantic features with dependencies. The hierarchical Transformer encoder is a trained encoder, and the training process includes: A training model is constructed, which includes a hierarchical Transformer encoder and a load reconstruction decoder; the load reconstruction decoder is used to generate each terminology of each SQL statement based on the workload features output by the hierarchical Transformer encoder. Obtain the training dataset; the samples in the training dataset are the database workload; The training model is trained based on the training dataset to obtain the trained hierarchical Transformer encoder.
2. The method as described in claim 1, characterized in that, The load reconstruction decoder is used for: The workload features output by the hierarchical Transformer encoder are expanded into a temporal dimension and used as a global memory vector. The global memory vector is then encoded by position to obtain the position-encoded global memory vector. The position-encoded global memory vector is input into the Transformer decoder, which includes a masked self-attention mechanism and a cross-attention mechanism. The position-encoded global memory vector is input into the masked self-attention mechanism and the cross-attention mechanism, respectively. The output of the masked self-attention mechanism serves as the query of the cross-attention mechanism, and the position-encoded global memory vector serves as the key and value of the cross-attention mechanism. The output of the cross-attention mechanism includes each token of each SQL statement.
3. A database configuration parameter tuning device based on load characterization and large model exploration, characterized in that, include: The workload characterization module is used to encode the database workload using a hierarchical Transformer encoder to obtain a workload characterization; the database workload includes N SQL statements; the hierarchical Transformer encoder includes a statement-level encoder and a load-level aggregator, the statement-level encoder encodes each SQL statement to generate statement-level semantic features, and the load-level aggregator merges all statement-level semantic features to generate the workload features. The cold start module is used to acquire the historical tuning knowledge base, which stores multiple historical workload features and their high-performance parameter configurations; calculate the cosine similarity between the workload features and each historical workload feature in the historical tuning knowledge base; and select the high-performance parameter configurations corresponding to the top K historical workload features with the highest cosine similarity to form the initial parameter configuration space. Explore the use of a module to randomly and uniformly sample in the initial parameter configuration space using the Latin hypercube sampling method to obtain the sampling parameter configuration space; The sampling parameter configuration space is based on a large model knowledge reasoning method to generate high-performance parameter configurations; Each SQL statement is encoded based on the statement-level encoder to generate statement-level semantic features, including: The SQL statement is broken down into multiple tokens; Each lexical is embedded and positionally encoded to obtain the input vector of the SQL statement; The statement-level encoder is used to encode the input vector of the SQL statement to obtain the statement-level semantic features of the SQL statement; the statement-level encoder includes L Transformer layers; Based on the load-level aggregator, all statement-level semantic features are fused to generate the workload features, including: A load-level aggregator is used to capture the dependencies between all statement-level semantic features, resulting in statement-level semantic features with dependencies; the load-level aggregator includes L Transformer layers; The workload features are obtained by average pooling the statement-level semantic features with dependencies. The hierarchical Transformer encoder is a trained encoder, and the training process includes: A training model is constructed, which includes a hierarchical Transformer encoder and a load reconstruction decoder; the load reconstruction decoder is used to generate each terminology of each SQL statement based on the workload features output by the hierarchical Transformer encoder. Obtain the training dataset; the samples in the training dataset are the database workload; The training model is trained based on the training dataset to obtain the trained hierarchical Transformer encoder.
4. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the database configuration parameter tuning method based on load characterization and large model exploration as described in any one of claims 1-2.
5. A computer program product, characterized in that, Includes a computer program / instruction, which, when executed by a processor, implements the database configuration parameter tuning method based on load characterization and large model exploration as described in any one of claims 1-2.
Citation Information
Patent Citations
Database intelligent parameter tuning system based on big language model enhancement
CN118467496A
Intelligent parameter tuning method and system for end-to-end cloud native database
CN119025536A
