Database configuration parameter tuning method based on load characterization and large model exploration

Abstract

The invention relates to a database configuration parameter adjusting and optimizing method based on load characterization and large model exploration, and the method comprises the steps: carrying out the coding of a database working load, and obtaining working load characterization; calculating the cosine similarity between the workload feature and each historical workload feature, and selecting high-performance parameter configurations corresponding to the first K historical workload features with the highest cosine similarity to form an initial parameter configuration space; randomly and uniformly sampling in the initial parameter configuration space to obtain a sampling parameter configuration space; and generating high-performance parameter configuration for the sampling parameter configuration space based on a large model knowledge reasoning method. According to the method and the device, the cold start adjustment and optimization process can be realized by utilizing historical workload adjustment and optimization experience knowledge, adjustment and optimization are not required to be carried out from zero for each new workload, meanwhile, the method and the device have high exploration efficiency, potential high-performance configuration can be explored sensitively after cold start, excessive workloads caused by low-efficiency exploration are prevented from being executed, and the method and the device are suitable for popularization and application. Therefore, the tuning cost is reduced.

Description

Technical Field

[0001] The present application relates to the field of database system configuration parameter tuning, and in particular, to a database configuration parameter tuning method based on load characterization and large model exploration. Background Art

[0002] Modern database systems typically offer hundreds of configurable parameters to adapt to diverse business needs, hardware environments, and workload characteristics. Different combinations of these parameters can lead to differences in database system performance. Achieving high-performance database services is crucial for improving enterprise business efficiency and reducing resource consumption. Therefore, research on database system configuration parameter tuning has significant application value.

[0003] The goal of database system configuration parameter tuning is to find the optimal or near-optimal combination of configuration parameters from a high-dimensional and complex configuration space to meet the database performance requirements in specific business scenarios. Due to the continuous expansion of database system scale and the increasing complexity of application scenarios, traditional database tuning methods based on expert experience and rules have become difficult to meet practical needs. Researchers are committed to designing more efficient tuning methods. Based on the core ideas and technical means of tuning methods, existing research can be mainly divided into three categories: search-based methods, learning-based methods, and natural language processing-based methods. The following problems exist in existing technologies:

[0004] First, search-based database system configuration parameter tuning methods do not require building performance prediction models and are direct and effective. However, the huge configuration space results in a high consumption of time and resource costs. In addition, most methods do not consider the constraints between configuration parameters during the search process, resulting in the search of a large number of configuration samples that violate domain knowledge rules. Moreover, most methods cannot utilize historical tuning knowledge and require searching from scratch for new tuning tasks, which is inefficient.

[0005] Second, traditional machine learning-based database system configuration tuning methods predict database system performance by building performance models, replacing the actual execution of workloads. This significantly reduces the resource overhead incurred by repeated workload execution during the exploration process. However, existing methods also suffer from issues such as strong empirical parameter selection, the requirement for a large number of high-quality samples to build performance models, and dynamic workload changes. Database system configuration tuning methods based on deep reinforcement learning do not require the collection of large samples to train performance models, but suffer from slow convergence. Each round of tuning typically requires hundreds of iterations, making them only suitable for scenarios with long tuning times and less versatile than traditional machine learning-based methods.

[0006] Third, the database system configuration parameter optimization method based on natural language processing uses a large language model to extract effective information from domain knowledge, accelerate the convergence of the configuration parameter tuning algorithm, and improve the tuning performance. However, this direction is still in the early stages of exploration and still has some defects: (1) Some work requires fine-tuning of the model, which will lead to additional resource consumption in both the initialization and tuning stages; (2) Inadequate use of domain knowledge. Most work only extracts parameter default values, recommended values, and interval range information from it, but pays less attention to the dependencies and constraints between parameters. This will sample and explore a large number of configurations that violate the dependencies between parameters, resulting in resource waste. (3) The dynamic changes of workloads are not considered. The tuning process is restarted for each new workload, and the experience in historical tuning tasks cannot be effectively utilized. This is inefficient and leads to resource waste. (4) The natural language understanding and reasoning capabilities of large models are not fully utilized. Most work is limited to using large language models to complete knowledge extraction work, but does not use large language models for knowledge reasoning. However, the latest research progress on large language models has shown its great potential in reasoning. Summary of the Invention

[0007] In order to overcome at least one deficiency in the prior art, the present application provides a database configuration parameter tuning method based on load characterization and large model exploration.

[0008] First, a database configuration parameter tuning method based on load characterization and large model exploration is provided, including:

[0009] The database workload is encoded using a hierarchical Transformer encoder to obtain a workload representation. The database workload consists of N SQL statements. The hierarchical Transformer encoder includes a statement-level encoder and a workload-level aggregator. The statement-level encoder encodes each SQL statement to generate statement-level semantic features. The workload-level aggregator fuses all the statement-level semantic features to generate a workload feature.

[0010] Obtain a historical tuning knowledge base, which stores multiple historical workload features and their high-performance parameter configurations. Calculate the cosine similarity between the workload feature and each historical workload feature in the historical tuning knowledge base, and select the high-performance parameter configurations corresponding to the top K historical workload features with the highest cosine similarity to form an initial parameter configuration space.

[0011] The Latin hypercube sampling method is used to randomly and uniformly sample in the initial parameter configuration space to obtain the sampling parameter configuration space; based on the large model knowledge reasoning method, high-performance parameter configuration is generated for the sampling parameter configuration space.

[0012] In one embodiment, each SQL statement is encoded based on a statement-level encoder to generate statement-level semantic features, including:

[0013] Split the SQL statement into multiple tokens;

[0014] Perform word embedding and position encoding on each word to obtain the input vector of the SQL statement;

[0015] A sentence-level encoder is used to encode the input vector of the SQL statement to obtain the sentence-level semantic features of the SQL statement; the sentence-level encoder includes L Transformer layers.

[0016] In one embodiment, workload features are generated by fusing all sentence-level semantic features based on a workload-level aggregator, including:

[0017] A payload-level aggregator is used to capture the dependencies between all sentence-level semantic features, obtaining sentence-level semantic features with dependencies. The payload-level aggregator consists of L Transformer layers.

[0018] The sentence-level semantic features with dependencies are average-pooled to obtain workload features.

[0019] In one embodiment, the hierarchical Transformer encoder is a trained encoder, and the training process includes:

[0020] Build a training model, which includes a hierarchical Transformer encoder and a workload reconstruction decoder. The workload reconstruction encoder is used to generate each token of each SQL statement based on the workload characteristics output by the hierarchical Transformer encoder.

[0021] Obtain a training dataset; the samples in the training dataset are database workloads;

[0022] The training model is trained based on the training data set to obtain a trained hierarchical Transformer encoder.

[0023] In one embodiment, the payload reconstruction decoder is configured to:

[0024] Expand the workload features output by the hierarchical Transformer encoder into a temporal dimension as a global memory vector;

[0025] Introduce position encoding into global memory to obtain the position-encoded global memory vector;

[0026] The position-encoded global memory vector is input into the Transformer decoder, which includes a masked self-attention mechanism and a criss-cross attention mechanism. The position-encoded global memory vector is input into the masked self-attention mechanism and the criss-cross attention mechanism respectively. The output of the masked self-attention mechanism serves as the query of the criss-cross attention mechanism, and the position-encoded global memory vector serves as the key and value of the criss-cross attention mechanism. The output of the criss-cross attention mechanism includes each word of each SQL statement.

[0027] In a second aspect, a database configuration parameter tuning device based on load characterization and large model exploration is provided, comprising:

[0028] The workload characterization module is used to encode the database workload using a hierarchical Transformer encoder to obtain a workload representation. The database workload consists of N SQL statements. The hierarchical Transformer encoder includes a statement-level encoder and a workload-level aggregator. The statement-level encoder encodes each SQL statement to generate statement-level semantic features. The workload-level aggregator fuses all the statement-level semantic features to generate workload features.

[0029] The cold start module is used to obtain a historical tuning knowledge base that stores multiple historical workload characteristics and their high-performance parameter configurations. The module calculates the cosine similarity between the workload characteristics and each historical workload characteristic in the historical tuning knowledge base, and selects the high-performance parameter configurations corresponding to the top K historical workload characteristics with the highest cosine similarity to form the initial parameter configuration space.

[0030] The exploration and utilization module is used to randomly and uniformly sample in the initial parameter configuration space using the Latin hypercube sampling method to obtain the sampling parameter configuration space; based on the large model knowledge reasoning method, the sampling parameter configuration space generates high-performance parameter configuration.

[0031] In a third aspect, a computer-readable storage medium is provided, which stores a computer program. When the computer program is executed by a processor, it implements the above-mentioned database configuration parameter tuning method based on load characterization and large model exploration.

[0032] In a fourth aspect, a computer program product is provided, comprising a computer program / instruction, which, when executed by a processor, implements the above-mentioned database configuration parameter tuning method based on load characterization and large model exploration.

[0033] Compared with the existing technology, the present application has the following beneficial effects: the database configuration parameter tuning method based on load characterization and large model exploration of the present application can not only utilize historical workload tuning experience knowledge to cold-start the tuning process without having to tune from scratch for each new workload, but also has high exploration efficiency. After the cold start, it can keenly explore potential high-performance configurations and avoid executing too much workload due to inefficient exploration, thereby reducing the tuning cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The present application may be better understood by referring to the following description in conjunction with the accompanying drawings, which together with the following detailed description are incorporated into and form a part of this specification. In the drawings:

[0035] Figure 1 A flowchart of a database configuration parameter tuning method based on load characterization and large model exploration is shown;

[0036] Figure 2 The results of each algorithm's optimization on TPC-H and TPC-C under different configuration metric costs are shown;

[0037] Figure 3 The structural block diagram of the database configuration parameter tuning device based on load characterization and large model exploration is shown. DETAILED DESCRIPTION

[0038] Exemplary embodiments of the present application are described below with reference to the accompanying drawings. For the sake of clarity and conciseness, not all features of actual embodiments are described in this specification. However, it should be understood that in the process of developing any such actual embodiment, many implementation-specific decisions may be made to achieve the developer's specific goals, and these decisions may vary from one implementation to another.

[0039] It is also necessary to explain here that, in order to avoid obscuring the present application due to unnecessary details, the accompanying drawings only show the device structure closely related to the solution according to the present application, while other details that are not closely related to the present application are omitted.

[0040] It should be understood that the present application is not limited to the described embodiments due to the following description with reference to the accompanying drawings. In this document, where feasible, the embodiments may be combined with each other, features between different embodiments may be replaced or borrowed, and one or more features may be omitted in one embodiment.

[0041] The present invention provides a method for optimizing database configuration parameters based on load characterization and large-scale model exploration. Figure 1 A flowchart of a database configuration parameter tuning method based on load characterization and large model exploration is shown. Figure 1 , the method mainly includes the following steps:

[0042] Step S1: Encode the database workload using a hierarchical Transformer encoder to obtain a workload representation; the database workload W includes N SQL statements, that is, W = {q1, q2, ...q i ,…q N},q i Represents the i-th SQL statement; the hierarchical Transformer encoder includes a statement-level encoder and a workload-level aggregator. The statement-level encoder encodes each SQL statement to generate statement-level semantic features, and the workload-level aggregator fuses all statement-level semantic features to generate workload features.

[0043] A database workload typically consists of a set of SQL statements that describe database operations such as queries, updates, and inserts. In practical applications, workloads change dynamically, and different workloads often have different optimal database configurations. If each new workload is tuned from scratch, it will inevitably waste time and resources. Therefore, this embodiment uses the empirical knowledge gained from similar historical workload tuning tasks to improve the tuning efficiency of the target workload. In order to quickly and accurately find the most similar workload in historical data, it is necessary to characterize the workload.

[0044] The hierarchical Transformer encoder first encodes each SQL statement in the workload based on the statement-level encoder to generate statement-level semantic features, and then uses the workload-level aggregator to fuse all statement features to generate workload features.

[0045] Step S2: Obtain a historical tuning knowledge base, which stores multiple historical workload features and their high-performance parameter configurations. Calculate the cosine similarity between the workload feature and each historical workload feature in the historical tuning knowledge base. Select the high-performance parameter configurations corresponding to the top K (e.g., K = 3) historical workload features with the highest cosine similarity to form the initial parameter configuration space.

[0046] Most learning-based database configuration tuning algorithms assume a fixed workload, without considering that the workload may change during actual application. Therefore, most methods usually lack a cold start mechanism during the tuning process. For each new workload tuning task, they start exploring from scratch, such as using the default configuration of the database as the starting configuration point for tuning, or using Latin hypercube sampling to randomly and uniformly sample several sample points in the configuration space as a set of starting configuration points, and then combining a specific search strategy to find the configuration that achieves the best database performance. This will lead to inefficiency and waste of resources in the tuning process for new workloads. In order to solve this problem, this embodiment adopts an efficient tuning cold start mechanism, that is, based on the tuning experience of similar historical workloads, a set of good configuration sets are constructed for the tuning task of the target workload as the starting point for tuning, so as to reduce the number of iterations required for the subsequent exploration process and the cost of configuration measurement.

[0047] The historical tuning knowledge base stores multiple historical workload characteristics and their high-performance parameter configurations. When historical data is scarce, a small number of workloads are generated and subjected to several rounds of random search. The top K high-performance configurations are selected and added to the knowledge base to complete the initialization of the knowledge base. As historical data accumulates, the top K high-performance configurations discovered during the overall tuning of each workload are added to the knowledge base.

[0048] It should be noted that for OLTP (On-Line Transaction Processing) workloads, since the SQL statements executed are randomly generated according to the set transaction weight distribution, special processing is required during workload characterization and similarity matching. The method adopted in this embodiment is: first, the benchmark test is repeatedly executed 10 times under a fixed transaction weight distribution, and 50 SQL statements are randomly extracted from the workload generated each time as an approximation of the workload group; then, the 10 extracted workloads are characterized to obtain corresponding feature vectors, which are averaged to obtain the feature vector of the OLTP workload under the transaction weight distribution; and then, the feature vector is used for similarity matching.

[0049] In step S3, a Latin hypercube sampling method is used to randomly and uniformly sample in the initial parameter configuration space to obtain a sampling parameter configuration space; and a high-performance parameter configuration is generated for the sampling parameter configuration space based on a large model knowledge reasoning method.

[0050] This embodiment not only leverages historical workload tuning experience to cold-start the tuning process, eliminating the need to start tuning from scratch for each new workload, but also offers efficient exploration efficiency, enabling the rapid exploration of potential high-performance configurations after a cold start, avoiding the execution of excessive workloads due to inefficient exploration, thereby reducing tuning costs. This embodiment can initiate the tuning process for a target workload from several good initial configuration points and efficiently explore potential configurations within the configuration space, effectively improving the convergence speed of tuning and reducing time and resource consumption.

[0051] In one embodiment, step S1 encodes each SQL statement based on a statement-level encoder to generate statement-level semantic features, including:

[0052] First, split the SQL statement into multiple tokens; for example, i Split into multiple tokens t i,1 ,t i,2 ,…t i,j ,…t i,M , t i,j Represents statement q i The jth word of M is the sentence q i On this basis, special symbols [CLS_i] and [SEP_i] are added for the start and end positions.

[0053] Then, each word is embedded and positionally encoded to obtain the input vector x of the SQL statement. i =(x i,1 ,x i,2 ,…x i,j …,x i,M );x i,j t i,j The result after word embedding and position encoding.

[0054] Then, a sentence-level encoder is used to encode the SQL statement q i The input vector x i Encode and get the SQL statement q i Sentence-level semantic features s i ; The sentence-level encoder consists of L Transformer layers, where L can be 2.

[0055] In one embodiment, step S1 generates workload features by fusing all sentence-level semantic features based on a workload-level aggregator, including:

[0056] A payload-level aggregator is used to capture the dependencies between all sentence-level semantic features to obtain sentence-level semantic features with dependencies. The payload-level aggregator includes L Transformer layers, where L can be 2.

[0057] The sentence-level semantic features with dependencies are average-pooled to obtain the workload feature z.

[0058] In one embodiment, the hierarchical Transformer encoder is a trained encoder, and the training process includes:

[0059] First, a training model is built. The training model includes a hierarchical Transformer encoder and a workload reconstruction decoder. The workload reconstruction encoder is used to generate each token of each SQL statement based on the workload features output by the hierarchical Transformer encoder.

[0060] Specifically, the payload reconstruction decoder is used to:

[0061] Expand the workload feature z output by the hierarchical Transformer encoder into a time series dimension as the global memory vector z ′ ;

[0062] Considering the global memory vector z ′ It is the same at each time step and lacks dynamics, so the global memory vector z ′ Position encoding is introduced to make the key and value at different time steps different, and the global memory vector after position encoding is obtained; the global memory vector z after position encoding of time step t is t ′ ;

[0063] The position-encoded global memory vector is input into the Transformer decoder. The Transformer decoder includes a masked self-attention mechanism and a cross-attention mechanism. The position-encoded global memory vector is input into the masked self-attention mechanism and the cross-attention mechanism respectively. The output of the masked self-attention mechanism is used as the query of the cross-attention mechanism, and the position-encoded global memory vector is used as the key and value of the cross-attention mechanism. The output of the cross-attention mechanism includes each word element of each SQL statement.

[0064] Then, a training dataset is obtained; the samples in the training dataset are database workloads;

[0065] Then, the training model is trained based on the training dataset to obtain the trained hierarchical Transformer encoder.

[0066] To train the encoder-decoder model so that the features output by the encoder accurately represent the workload, different types of learning tasks can be designed. To reduce the burden of data collection, this embodiment uses a workload reconstruction task to drive encoder-decoder training. Specifically, the encoder inputs the SQL sequence contained in the workload, and the decoder autoregressively reconstructs the complete SQL statement sequence. During training, the cross-entropy loss is calculated based on the predicted and true probability distributions.

[0067] In order to further verify the effectiveness of the method of this application, a comparative experiment was set up, using the MySQL database as the configuration parameter tuning object, and comparing the method of this application (hereinafter referred to as EfficTune) with representative database system configuration parameter tuning algorithms.

[0068] Example

[0069] Experimental Datasets: We selected two representative benchmark datasets in the field of database performance testing, namely TPC-H and TPC-C. (1) The TPC-H benchmark dataset is an OLAP workload that simulates a decision support system for the retail industry. It contains 8 related tables and 22 complex analytical SQL statements, covering typical OLAP operations such as aggregation, multi-table joins, subqueries, and expression calculations. A TPC-H database with a scale factor of 1 was used. (2) The TPC-C benchmark dataset is an online transaction processing (OLTP) type database performance test benchmark that simulates five core transaction types in a typical warehouse management business scenario and can evaluate the transaction processing capabilities of the database system. A TPC-C database containing five warehouses was used.

[0070] Comparison algorithms: (1) BestConfig: A search-based configuration parameter tuning method that proposes divide-and-diverge sampling and recursive bound-and-search methods to improve the efficiency of searching for the best configuration parameters. (2) RFHOC: A classic machine learning-based configuration parameter optimization method that uses a random forest model as a performance prediction model and combines the performance prediction model with a genetic algorithm to find the configuration parameters that produce the highest performance. (3) SMAC: A parameter tuning method based on Bayesian optimization that uses random forest as a proxy model and is widely used in parameter tuning. (4) QTune: A database system configuration tuning method based on deep reinforcement learning that considers the rich characteristics of the workload and inputs query features into the DRL model to select the most appropriate configuration.

[0071] Performance comparison indicators: (1) For the OLAP workload TPC-H, the latency reduction (LR) of the database execution workload after configuration parameter tuning is used as the performance comparison indicator between different algorithms. It can be defined as:

[0072]

[0073] Among them L baseline is the workload latency after different algorithm configurations are tuned, L defeult is the workload latency using the database system's default configuration.

[0074] (2) For the OLTP workload TPC-C, the degree of database throughput improvement (TI) after configuration parameter tuning is used as the performance comparison indicator between different algorithms, which is defined as:

[0075]

[0076] Where T baseline is the database throughput (number of transactions per second, tx / s) after different algorithm configurations are optimized. dafault The throughput of the database using the default configuration of the database system

[0077] Experimental results: Tables 1 and 2 show the specific performance improvement results obtained by tuning database configuration parameters for TPC-H and TPC-C workloads using three configuration cost metrics for different algorithms.

[0078] Table 1 Comparison of algorithm performance under different configurations of TPC-H workload metric cost constraints

[0079]

[0080] Table 2 Comparison of algorithm performance under different sampling cost constraints in TPC-C workload

[0081]

[0082] Figure 2 The results of TPC-H and TPC-C optimization of each algorithm under different configuration metric costs are shown. Figure 2As can be seen in the results, EfficTune achieved the highest latency reduction (LR) and throughput improvement (TI) on both the TPC-H and TPC-C workloads, respectively. Calculations show that on TPC-H, EfficTune achieved an average of 13.90%, 21.43%, 10.13%, and 8.03% lower latency than BestConfig, RFHOC, QTune, and SMAC, respectively. On TPC-C, EfficTune achieved 15.60%, 18.33%, 9.47%, and 9.80% higher throughput than BestConfig, RFHOC, QTune, and SMAC, respectively. This demonstrates that EfficTune's overall tuning performance is superior to other methods.

[0083] On the other hand, from Figure 2 It can also be seen that, on both TPC-H and TPC-C workloads, EfficTune can achieve performance improvements close to or even exceeding those achieved by other algorithms using a configuration metric cost of CM = 600, using a configuration metric cost of CM = 200. This demonstrates that EfficTune achieves the same tuning effect at a lower cost. This is due, firstly, to the cold start mechanism in the EfficTune algorithm, which initiates the tuning process for the target workload from several good initial configuration points, effectively reducing the number of iterations required during the exploration phase. In contrast, BestConfig, RFHOC, and SMAC all require tuning from scratch, lacking the use of historical tuning data and therefore incurring higher tuning costs for new tuning tasks. Furthermore, the EfficTune algorithm leverages the reasoning capabilities of large language models, using previously explored high-performance configurations to generate promising configurations, achieving higher exploration efficiency.

[0084] Using the same inventive concept as the database configuration parameter tuning method based on load characterization and large model exploration, this embodiment also provides a corresponding database configuration parameter tuning device based on load characterization and large model exploration. Figure 3 The structure diagram of the database configuration parameter tuning device based on load characterization and large model exploration is shown, including:

[0085] The workload characterization module is used to encode the database workload using a hierarchical Transformer encoder to obtain a workload representation. The database workload consists of N SQL statements. The hierarchical Transformer encoder includes a statement-level encoder and a workload-level aggregator. The statement-level encoder encodes each SQL statement to generate statement-level semantic features. The workload-level aggregator fuses all the statement-level semantic features to generate workload features.

[0086] The cold start module is used to obtain a historical tuning knowledge base that stores multiple historical workload characteristics and their high-performance parameter configurations. The module calculates the cosine similarity between the workload characteristics and each historical workload characteristic in the historical tuning knowledge base, and selects the high-performance parameter configurations corresponding to the top K historical workload characteristics with the highest cosine similarity to form the initial parameter configuration space.

[0087] The exploration and utilization module is used to randomly and uniformly sample in the initial parameter configuration space using the Latin hypercube sampling method to obtain the sampling parameter configuration space; based on the large model knowledge reasoning method, the sampling parameter configuration space generates high-performance parameter configuration.

[0088] The database configuration parameter tuning device based on load characterization and large model exploration of this embodiment has the same inventive concept as the database configuration parameter tuning method based on load characterization and large model exploration mentioned above. Therefore, the specific implementation method of the device can be seen in the embodiment part of the database configuration parameter tuning method based on load characterization and large model exploration mentioned above, and its technical effect corresponds to the technical effect of the above method, which will not be repeated here.

[0089] An embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the above-mentioned database configuration parameter tuning method based on load characterization and large model exploration.

[0090] An embodiment of the present application provides a computer program product, including a computer program / instruction. When the computer program / instruction is executed by a processor, it implements the above-mentioned database configuration parameter tuning method based on load characterization and large model exploration.

[0091] In summary, this application has the following technical effects:

[0092] This application can use the experience and data obtained from past historical tuning tasks, combine the idea of ​​exploration and utilization with the natural language understanding and reasoning capabilities of large language models, and improve the efficiency of finding the best configuration parameters for new workloads. First, through NLP representation learning technology, a hierarchical Transformer encoder and a workload reconstruction decoder are used to convert complex workloads into low-dimensional feature vector representations; then, a tuning cold start is performed, and the similarity between workload feature representations is used to match similar historical workloads for the new workload, and the best configuration of similar historical workloads is used as the initial configuration set, avoiding tuning from scratch and saving time and resource consumption; then, a large model-enhanced exploration and utilization process is performed, combining the idea of ​​exploration and utilization to further optimize the configuration of the new workload based on the initial configuration. The exploration phase uses the Latin hypercube sampling method to uniformly sample in the configuration space, and in the utilization phase, potential high-performance configurations are generated through large model knowledge reasoning. Through the above approach, this application can start the tuning process of the target workload from several good initial configuration points and efficiently explore potential configurations in the configuration space, effectively improving the convergence speed of the tuning and reducing time and resource consumption.

[0093] The above descriptions are merely examples of various embodiments of the present application, but the scope of protection of the present application is not limited thereto. Any modifications or substitutions that can be readily conceived by a person skilled in the art within the technical scope disclosed in the present application should be included within the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A database configuration parameter tuning method based on load characterization and large model exploration, characterized in that: include: A database workload is encoded using a hierarchical Transformer encoder to obtain a workload representation; the database workload includes N SQL statements; the hierarchical Transformer encoder includes a statement-level encoder and a workload-level aggregator, each SQL statement is encoded based on the statement-level encoder to generate a statement-level semantic feature, and all the statement-level semantic features are fused based on the workload-level aggregator to generate the workload feature; Obtain a historical tuning knowledge base, wherein the historical tuning knowledge base stores a plurality of historical workload features and their high-performance parameter configurations; calculate the cosine similarity between the workload feature and each historical workload feature in the historical tuning knowledge base, and select the high-performance parameter configurations corresponding to the top K historical workload features with the highest cosine similarity to form an initial parameter configuration space; A Latin hypercube sampling method is used to randomly and uniformly sample in the initial parameter configuration space to obtain a sampling parameter configuration space; and a high-performance parameter configuration is generated for the sampling parameter configuration space based on a large model knowledge reasoning method.

2. The method according to claim 1, wherein in, Encoding each of the SQL statements based on the statement-level encoder to generate statement-level semantic features includes: Splitting the SQL statement into multiple tokens; Perform word embedding and position encoding on each word to obtain the input vector of the SQL statement; The sentence-level encoder is used to encode the input vector of the SQL statement to obtain the sentence-level semantic features of the SQL statement; the sentence-level encoder includes L Transformer layers.

3. The method according to claim 1, wherein in, The workload features are generated by fusing all the statement-level semantic features based on the workload-level aggregator, including: A payload-level aggregator is used to capture the dependencies between all sentence-level semantic features to obtain sentence-level semantic features with dependencies; the payload-level aggregator includes L Transformer layers; The sentence-level semantic features with dependencies are average-pooled to obtain the workload features.

4. The method according to claim 1, wherein The hierarchical Transformer encoder is a trained encoder. The training process includes: Constructing a training model, the training model including a hierarchical Transformer encoder and a workload reconstruction decoder; the workload reconstruction encoder is used to generate each token of each SQL statement based on the workload characteristics output by the hierarchical Transformer encoder; Obtaining a training data set; samples in the training data set are database workloads; The training model is trained based on the training data set to obtain a trained hierarchical Transformer encoder.

5. The method according to claim 4, wherein The load reconstruction decoder is used to: Expanding the workload features output by the hierarchical Transformer encoder into a temporal dimension as a global memory vector; Introducing position coding into the global memory to obtain a position-coded global memory vector; The position-encoded global memory vector is input into the Transformer decoder, which includes a masked self-attention mechanism and a crisscross attention mechanism. The position-encoded global memory vector is input into the masked self-attention mechanism and the crisscross attention mechanism respectively. The output of the masked self-attention mechanism serves as the query of the crisscross attention mechanism, and the position-encoded global memory vector serves as the key and value of the crisscross attention mechanism. The output of the crisscross attention mechanism includes each word of each SQL statement.

6. A database configuration parameter tuning device based on load characterization and large model exploration, characterized in that: include: A workload characterization module is configured to encode the database workload using a hierarchical Transformer encoder to obtain a workload representation; the database workload includes N SQL statements; the hierarchical Transformer encoder includes a statement-level encoder and a workload-level aggregator, each SQL statement is encoded based on the statement-level encoder to generate a statement-level semantic feature, and all statement-level semantic features are fused based on the workload-level aggregator to generate the workload feature; A cold start module is configured to obtain a historical tuning knowledge base, wherein the historical tuning knowledge base stores a plurality of historical workload characteristics and their high-performance parameter configurations; calculate the cosine similarity between the workload characteristics and each historical workload characteristic in the historical tuning knowledge base; and select the high-performance parameter configurations corresponding to the top K historical workload characteristics with the highest cosine similarity to form an initial parameter configuration space; The exploration and utilization module is used to randomly and uniformly sample in the initial parameter configuration space using the Latin hypercube sampling method to obtain a sampling parameter configuration space; and to generate a high-performance parameter configuration for the sampling parameter configuration space based on a large model knowledge reasoning method.

7. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it implements the database configuration parameter tuning method based on load characterization and large model exploration according to any one of claims 1 to 5.

8. A computer program product, characterized in that The method comprises a computer program / instruction, which, when executed by a processor, implements the database configuration parameter tuning method based on load characterization and large model exploration as described in any one of claims 1 to 5.

Images