Database configuration parameter optimization method and system based on large language model and deep reinforcement learning

Abstract

The present application relates to the technical field of database software debugging, more particularly to a database configuration parameter optimization method and system based on a large language model and deep reinforcement learning.The present application comprises: step one, obtaining the optimization source information of the target database software; step two, processing the optimization source information through a large language model; then using the question and answer method, obtaining the recommended parameter value returned by the large language model and arranging it into a parameter binary table; step three, testing the target database software according to the parameter binary table to obtain an initial transition data sequence; step four, training a DDPG for demonstration model based on the initial transition data sequence; after the DDPG for demonstration model converges, outputting the final parameter value.The present application integrates expert experience into the process of reinforcement learning, providing a feasible solution to reduce the cost of optimization and improve the accuracy of optimization as much as possible.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of database software debugging, and more particularly, to a database configuration parameter tuning method based on a large language model and deep reinforcement learning, and a database configuration parameter tuning system based on a large language model and deep reinforcement learning using the method. BACKGROUND

[0002] Database software is a type of database management system designed to allow users to customize the performance, security, availability, and other characteristics of a database according to specific needs and conditions by adjusting configuration parameters. This type of software provides users with a wide range of configuration options to adapt to different workloads, hardware environments, and business scenarios.

[0003] Common database software includes but is not limited to MySQL, PostgreSQL, MongoDB, etc. MySQL and PostgreSQL, as database software systems, both cover a large number of configuration parameters to adapt to diverse application scenarios. In MySQL, there are more than 100 parameters such as buffer pool size, connection number, etc., while PostgreSQL provides more than 500 configuration options including index configuration, query optimization, etc. These rich configuration parameters allow users to customize settings according to actual business needs, thus flexibly coping with different workloads and performance requirements.

[0004] However, this also brings challenges to configuration tuning - users need to make wise choices in the complex parameter space to balance performance, security, and other non-functional attributes. Because these configuration parameters not only affect the functional attributes of the software, but also involve non-functional attributes such as performance and cost.

[0005] That is, the complexity of the configuration space and the complex constraint relationships between configuration parameters make it difficult to adjust the configuration. Incorrect configuration can cause system crashes, abnormal outputs, or even serious performance degradation, thereby affecting user experience, response time, and system throughput, etc.

[0006] Existing tuning work is often done by database administrators (DBAs) through experience to adjust configuration parameters, but on the one hand it is difficult to find the optimal or near-optimal configuration, and on the other hand it is difficult to discover the relationship between configuration parameters. Today, there are also methods of using machine learning to search for configurations or establishing models to obtain desired configuration parameters, but the method of using ordinary reinforcement learning for configuration tuning needs to explore many meaningless points before reaching convergence, which causes a lot of waste of overhead. SUMMARY

[0007] Based on this, it is necessary to solve the problems of poor effect of manual configuration of database parameters and waste of overhead in existing ordinary reinforcement learning method configuration optimization, and provide a database configuration parameter optimization method and system based on large language model and deep reinforcement learning.

[0008] The application adopts the following technical solutions:

[0009] In a first aspect, the application discloses a database configuration parameter optimization method based on a large language model and deep reinforcement learning, comprising the following steps:

[0010] Step one, obtain the optimization source information of the target database software;

[0011] Among them, step one includes:

[0012] Step 1.1, search the parameter configuration suggestion webpage about the target database software in the Internet through the search engine;

[0013] Step 1.2, text processing is performed on the parameter configuration suggestion webpage, and the content description irrelevant to the parameter configuration is removed to obtain the optimization source information;

[0014] Step two, process the optimization source information through the large language model; then adopt the question and answer mode to obtain the recommended parameter value returned by the large language model and arrange it into a parameter binary table;

[0015] Step three, test the target database software according to the parameter binary table to obtain the initial transition data sequence;

[0016] Step four, train the DDPG for demonstration model based on the initial transition data sequence; after the DDPG for demonstration model converges, output the final parameter value.

[0017] The database configuration parameter optimization method based on the large language model and the deep reinforcement learning realizes the method or process according to the embodiments of the present disclosure.

[0018] In a second aspect, the application discloses a database configuration parameter optimization system based on a large language model and deep reinforcement learning, which uses the database configuration parameter optimization system based on a large language model and deep reinforcement learning disclosed in the first aspect.

[0019] The database configuration parameter optimization system based on a large language model and deep reinforcement learning comprises an information acquisition module, a large language processing module, a pre-test module and a DDPGFD processing module.

[0020] The information acquisition module is configured to acquire tuning source information of the target database software. The large language processing module is internally provided with a large language model, which is configured to process the tuning source information. Then, the large language model returns recommended parameter values in a question-and-answer manner, and the recommended parameter values are sorted into a parameter binary table. The pre-test module is configured to test the target database software according to the parameter binary table, and obtain an initial transition data sequence. The DDPGFD processing module is internally provided with a DDPG for demonstration model, and is configured to train the DDPG for demonstration model based on the initial transition data sequence. After the DDPG for demonstration model converges, the final parameter values are output.

[0021] The database configuration parameter tuning system based on the large language model and the deep reinforcement learning realizes the method or process according to the embodiments of the present disclosure.

[0022] In a third aspect, the present disclosure provides a readable storage medium. The readable storage medium stores computer program instructions. When the computer program instructions are read and executed by a processor, the steps of the database configuration parameter tuning method based on the large language model and the deep reinforcement learning are performed.

[0023] Compared with the prior art, the present disclosure has the following beneficial effects:

[0024] Based on the idea of imitation learning, the present disclosure integrates expert experience into the process of reinforcement learning. On the one hand, the expert experience is used to obtain an initial transition data sequence for adjusting the first round of weights of the DDPG for demonstration model, thereby shortening the time for exploring meaningless points. On the other hand, the DDPG for demonstration model is used for deep reinforcement learning of itself, and the model weights and the recommended parameters are periodically alternately adjusted to ensure the accuracy of tuning. The present disclosure provides a feasible solution to reduce the cost of tuning and improve the accuracy of tuning as much as possible. BRIEF DESCRIPTION OF DRAWINGS

[0025] Figure 1 The flowchart of the database configuration parameter tuning method based on the large language model and the deep reinforcement learning in the embodiment 1 of the present disclosure is shown.

[0026] Figure 2 The example chart of the parameter binary table is shown. Figure 1 DETAILED DESCRIPTION

[0027] ​With reference to the drawings of the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments of the present application, all the other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0028] It should be noted that when a component is referred to as being "mounted on" another component, it can be directly on the other component or there can be a middle component. When a component is referred to as being "disposed on" another component, it can be directly disposed on the other component or there can be a middle component. When a component is referred to as being "fixed on" another component, it can be directly fixed on the other component or there can be a middle component.

[0029] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in the description herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0030] Embodiment 1

[0031] Please refer to Figure 1 , Figure 1 The flowchart of the database configuration parameter tuning method based on a large language model and deep reinforcement learning provided for Embodiment 1 is shown in FIG. 1. As shown in FIG. 1, the database configuration parameter tuning method includes the following steps: Figure 1

[0032] Step 1, obtain tuning source information of the target database software.

[0033] It should be noted that the tuning source information obtained by the method can be regarded as expert experience from the Internet.

[0034] Specifically, step 1 includes:

[0035] Step 1.1, search the Internet for parameter configuration suggestion web pages about the target database software through a search engine; wherein the search engine includes but is not limited to Google search, Baidu search, and Bing search; and the related words of the parameter tuning of the target database software are used as keywords for searching.

[0036] ​Take the target database software Mysql as an example. You can search for keywords such as "mysql configuration tuning" on Google and obtain web pages related to Mysql configuration parameters: technical forum posts, official software documentation, etc. These web pages will record the recommended values of Mysql configuration parameters.

[0037] Step 1.2, text processing of parameter configuration suggestion web pages, removing irrelevant content descriptions, obtaining tuning source information.

[0038] Specifically, step 1.2 includes:

[0039] Save the content of the parameter configuration suggestion web page to the local text database using the beautiful soup tool; where the beautiful soup tool is a Python library that can extract data from HTML or XML files.

[0040] Then remove the content descriptions unrelated to the configuration parameters in the local text database using regular expressions to obtain the tuning source information; where regular expressions are a tool for matching and manipulating text.

[0041] Step two, process the tuning source information through a large language model; then use the question and answer method to obtain the recommended parameter values returned by the large language model and organize them into a parameter binary table.

[0042] Where the large language model (LLM) has strong understanding capabilities, such as GPT-4, Flan-T5, MOSS, etc. This step is to obtain structured tuning recommendations from tuning source information through a large language model - that is, for parameter X, the recommended value is Y.

[0043] Specifically, step two includes:

[0044] Step 2.1, divide the tuning source information into individual natural language paragraphs containing complete tuning descriptions using regular expressions.

[0045] This processing is to avoid exceeding the maximum input token limit of the large language model, and also to ensure the integrity of the tuning source information.

[0046] Step 2.2, input the natural language paragraphs into the large language model.

[0047] For example, one natural language paragraph is “As a common rule of thumb, you'll want to set shared_buffers at around 25% of the system's total RAM”, which means that the recommended value of shared_buffers is one quarter of the total available memory.

[0048] Alternatively, one natural language paragraph is “As a common rule of thumb, you'll want to set shared_buffers at 8GB”, which means that the recommended value of shared_buffers is 8GB.

[0049] Alternatively, one natural language paragraph is “As a common rule of thumb, you'll want to set shared_buffers from 8GB to 12GB”, which means that the recommended value of shared_buffers is any value within the range of 8GB to 12GB.

[0050] Then the large language model understands it according to the above meaning.

[0051] Step 2.3, according to the large language model preset question template, ask the large language model to optimize the parameters, so that the answer returned by the large language model contains the recommended parameter value.

[0052] Generally, the question template uses CoT prompt to ask questions, for example, input question: I want to get multiple binary tuple data structures from the text, that is, (the name of the configuration parameter, the value of the optimization). The recommended value can be: 1 an absolute value, such as 2gb, or a range [20, 25] 2 a relative value, relative to other parameters, or a certain index, such as 25% RAM. For example, ('shared_buffers', '0.25mem') If a parameter has multiple optimization suggestions, generate multiple binary tuples. The text is: “As a common rule of thumb, you'll want to set shared_buffers at around 25% of the system's total RAM”. Then the answer returned by the large language model is (shared_buffers, 0.25mem).

[0053] Similarly, if the above question text in Chinese is modified to "As a common rule of thumb, you'll want to set shared_buffers from 8GB to 12GB", then the answer returned by the large language model is (shared_buffers, 8GB ~ 12GB).

[0054] If the above question text in Chinese is modified to "As a common rule of thumb, you'll want to set shared_buffers at 8GB", then the answer returned by the large language model is (shared_buffers, 8GB).

[0055] The above answer shows the recommended parameter value of shared_buffers.

[0056] Step 2.4, convert the answer returned by the large language model into a parameter binary table through a regular expression; wherein the parameter binary table contains the parameter name and its corresponding recommended value.

[0057] As shown in Figure 2 , several commonly used parameter binary tables are shown - only one recommended value is shown.

[0058] Step three, test the target database software according to the parameter binary table to obtain an initial transition data sequence.

[0059] Generally, benchmarking tools such as YCSB, sysbench, LoadRunner, etc. are used to test the target database software. This embodiment recommends using YCSB, which can provide a variety of load parameters to generate different types of loads, thereby testing the running time and throughput of the target database under different loads.

[0060] Specifically, step three includes:

[0061] According to the parameter binary table, import the recommended parameter value into the target database software and perform testing to obtain an initial transition data sequence (transition0) and store it in the prioritized experience replay buffer (Prioritized experience replay, PER).

[0062] Wherein, the initial transition data sequence (transition0) includes: the state of the target database software before the recommended parameter value is imported (s0), the recommended parameter value (a0), the state of the target database software after the recommended parameter value is imported (s0'), the performance improvement reward of the target database software after the recommended parameter value is imported (r0).

[0063] Step four, training the DDPG for demonstration model based on the initial transition data sequence; after the DDPG for demonstration model converges, output the final parameter value.

[0064] In general, the DDPG for demonstration model (which can be abbreviated as DDPGFD) uses demonstration (i.e. the initial transition data sequence) and DDPG (i.e. the DDPG model) jointly to recommend configuration parameters.

[0065] Specifically, step four includes:

[0066] Step 4.1, initialize the DDPG for demonstration model.

[0067] It should be noted that the main architecture of the DDPG for demonstration model is similar to that of the DDPG model, except that it uses the initial transition data sequence as the input for model weight adjustment.

[0068] Step 4.2, select the initial transition data sequence from the prioritized experience replay buffer and input it into the DDPG for demonstration model for weight training to adjust the weight coefficients of the DDPG for demonstration model;

[0069] Step 4.3, after the weight training of step 4.2 is completed, the DDPG for demonstration model with adjusted weight coefficients is obtained.

[0070] Step 4.4, generate N groups of candidate configuration parameters through the DDPG for demonstration model with adjusted weight coefficients. Generally, N≥5. In this embodiment 1, N is 20.

[0071] Step 4.5, test the target database software according to the n groups of candidate configuration parameters respectively to obtain N candidate transition data sequences (transition1~transition N ), and store them in the prioritized experience replay buffer (Prioritized experience replay, PER);

[0072] Among them, the nth candidate transition data sequence (transition n ) includes: the state of the target database software before the import of the nth group of candidate configuration parameters (s n ), the nth group of candidate configuration parameters (a n), the state of the target database software after the import of the n-th set of candidate configuration parameters (s n ), the performance improvement reward (r n ) of the target database software after the import of the n-th set of candidate configuration parameters.

[0073] In this way, the priority experience replay buffer has the initial transition data sequence and the candidate transition data sequence.

[0074] Step 4.6, priority of all transition data sequences in the priority experience replay buffer is calculated; wherein all transition data sequences include the initial transition data sequence and the candidate transition data sequence;

[0075] And the m transition data sequences with high priority are input into the model in step 4.4 for weight training, and the weight coefficients of the model are adjusted again.

[0076] Generally, m is a hyperparameter and is 20.

[0077] The calculation formula of the priority is as follows:

[0078]

[0079] In the formula, P(i) is the priority of the i-th transition data sequence, that is, the probability of the i-th transition data sequence being selected from all transition data sequences in the priority experience replay buffer,

[0080] p i represents the priority probability of the i-th transition data sequence; represents the time difference error of the i-th transition data sequence; and λ3 is a hyperparameter; represents the action gradient of the Q function of the DDPG for demonstration model; s i represents the state before import in the i-th transition data sequence; a i represents the configuration parameter in the i-th transition data sequence; θ Q represents the weight coefficient of the DDPG for demonstration model; ∈ represents the minimum value to be selected, so as to ensure that each transition data sequence has a certain probability of being selected; and ∈ D represents the guaranteed selection probability of the initial transition data sequence, so as to increase the probability of the initial transition data sequence being selected.

[0081] It should be noted that although the initial transition data sequence represents the expert experience, its priority use is better than random exploration. However, according to the above formula, only the probability of the initial transition data sequence being in the m transition data sequences with high priority is increased, and it cannot be said that the m transition data sequences with high priority always have the initial transition data sequence each time.

[0082] Step 4.7, after the weight training of step 4.6 is completed, the DDPG for demonstration model with adjusted weight coefficients is obtained again.

[0083] Step 4.8, repeat steps 4.4-4.7 until the model converges, and the output candidate configuration parameters of the model are used as the final parameter values.

[0084] It should be noted that as steps 4.4-4.7 are repeated, the number of candidate transition data sequences in the experience replay buffer increases, but the same transition data sequence can be used multiple times. Although this may waste some newly generated transition data sequences, it can ensure the efficiency of using old data - because initial transition data sequences such as expert experience data are more likely to make the target database software achieve optimal performance.

[0085] In this way, the final parameter values of the target database software are obtained based on the above steps, and they can be applied to the target database software.

[0086] Embodiment 2

[0087] This embodiment 2 discloses a database configuration parameter optimization system based on a large language model and deep reinforcement learning, which uses the database configuration parameter optimization method based on a large language model and deep reinforcement learning disclosed in embodiment 1.

[0088] The database configuration parameter optimization system based on a large language model and deep reinforcement learning includes an information acquisition module, a large language processing module, a pre-test module, and a DDPGFD processing module.

[0089] The information acquisition module is configured to acquire optimization source information of a target database software.

[0090] The large language processing module has a large language model built-in and is configured to process the optimization source information; then uses the question and answer method to obtain the recommended parameter values returned by the large language model and organizes them into a parameter binary table.

[0091] The pre-test module is configured to test the target database software according to the parameter binary table and obtain initial transition data sequences.

[0092] The DDPGFD processing module has a DDPG for demonstration model built-in and is configured to train the DDPG for demonstration model based on the initial transition data sequences; after the DDPG for demonstration model converges, the final parameter values are output.

[0093] Embodiment 3

[0094] The embodiment 3 also discloses a readable storage medium, and the readable storage medium stores computer program instructions. When the computer program instructions are read and run by a processor, the method for optimizing database configuration parameters based on a large language model and deep reinforcement learning in the embodiment 1 is executed.

[0095] The method in the embodiment 1 can be applied in the form of software, for example, a program independently run by a computer readable storage medium, for example, a U disk, designed as a U disk, and the program is designed to start the whole method through external triggering.

[0096] The technical features of the above embodiments can be combined in any manner. To make the description simple, all possible combinations of the technical features in the above embodiments are not described, but as long as the combinations of the technical features do not exist, they should be considered as the scope of the description.

[0097] The above embodiments only express several implementation manners of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the patent. It should be pointed out that for ordinary skilled persons in the art, without departing from the concept of the present application, some modifications and improvements can be made, which are all within the protection scope of the present application. Therefore, the protection scope of the patent of the present application should be subject to the appended claims.

Claims

1. A method for optimizing database configuration parameters based on large language models and deep reinforcement learning, characterized in that, Includes the following steps: Step 1: Obtain the optimization source information for the target database software; Step one includes: Step 1.1: Search the internet for web pages offering parameter configuration suggestions for the target database software using a search engine; Step 1.2: Textify the parameter configuration suggestion webpage, remove content descriptions unrelated to parameter configuration, and obtain the optimization source information; Step two: Process the source information for optimization using a large language model; then, use a question-and-answer approach to obtain the recommended parameter values ​​returned by the large language model and organize them into a parameter binary table. Step 3: Test the target database software according to the parameter binary table to obtain the initial transformation data sequence; Step 4: Train the DDPG for demonstration model based on the initial transformation data sequence; after the DDPG for demonstration model converges, output the final parameter values. Step four includes: Step 4.1, initialize the DDPG for demonstration model; Step 4.2: Select the initial transition data sequence from the priority experience replay buffer and input it into the DDPG for demonstration model for weight training to adjust the weight coefficients of the DDPG for demonstration model; Step 4.3: After the weight training in step 4.2 is completed, the DDPG fordemonstration model with adjusted weight coefficients is obtained. Step 4.4: Generate the DDPG for demonstration model by adjusting the weight coefficients. N Group candidate configuration parameters; Step 4.5, according to N The candidate configuration parameters were tested on the target database software to obtain... N A candidate transition data sequence is generated and stored in the priority experience replay buffer; Among them, the n The candidate transition data sequences include: the target database software in the... n The status of the group candidate configuration parameters before import, the first n Group candidate configuration parameters, target database software in the first n The status of the group candidate configuration parameters after import, and the target database software in the first... n Performance improvement reward after importing group candidate configuration parameters; Step 4.6: Calculate the priority of all transition data sequences in the priority experience replay buffer; where all transition data sequences include: initial transition data sequence and candidate transition data sequences; and high priority m The transformed data sequence is input into the model in step 4.4 for weight training, and the weight coefficients of the model are adjusted again. Step 4.7: After the weight training in step 4.6 is completed, the DDPG fordemonstration model with adjusted weight coefficients is obtained. Step 4.8: Repeat steps 4.4 to 4.7 until the model converges, and use the candidate configuration parameters output as the final parameter values.

2. The database configuration parameter tuning method based on large language models and deep reinforcement learning according to claim 1, characterized in that, In step 1.1, the search engine includes, but is not limited to: Google Search, Baidu Search, and Bing Search.

3. The database configuration parameter tuning method based on large language models and deep reinforcement learning according to claim 1, characterized in that, In step 1.1, relevant terms for optimizing the target database software parameters are used as keywords for searching.

4. The database configuration parameter tuning method based on large language models and deep reinforcement learning according to claim 2, characterized in that, Step 1.2 includes: The Beautiful Soup tool is used to save the content of the parameter configuration suggestion webpage locally, forming a local text database. Then, regular expressions are used to remove content descriptions from the local text database that are irrelevant to the configuration parameters, thus obtaining the tuning source information.

5. The database configuration parameter tuning method based on large language models and deep reinforcement learning according to claim 1, characterized in that, Step two includes: Step 2.1: The optimization source information is segmented into natural language paragraphs containing complete optimization descriptions using regular expressions; Step 2.2: Input the natural language paragraphs into the large language model; Step 2.3: Use the pre-set question template of the large language model to perform parameter tuning questions so that the answer returned by the large language model includes recommended parameter values; Step 2.4: Convert the answer returned by the large language model into a parameter binary table using regular expressions; the parameter binary table contains parameter names and their corresponding recommended values.

6. The database configuration parameter tuning method based on large language models and deep reinforcement learning according to claim 1, characterized in that, In step three, Based on the parameter binary table, the recommended parameter values ​​are imported into the target database software and tested to obtain the initial transformation data sequence, which is then stored in the priority experience replay buffer. The initial transformation data sequence includes: the target database software's state before the recommended parameter values ​​were imported, the recommended parameter values, the target database software's state after the recommended parameter values ​​were imported, and the performance improvement reward of the target database software after the recommended parameter values ​​were imported.

7. The database configuration parameter tuning method based on large language models and deep reinforcement learning according to claim 6, characterized in that, Benchmarking tools were used to test the target database software.

8. A database configuration parameter tuning system based on large language models and deep reinforcement learning, characterized in that, It uses the database configuration parameter tuning method based on large language model and deep reinforcement learning as described in any one of claims 1-7; The database configuration parameter tuning system based on large language models and deep reinforcement learning includes: The information acquisition module is used to acquire the optimization source information of the target database software; The large language processing module has a built-in large language model for processing the source information for optimization; then, it uses a question-and-answer approach to obtain the recommended parameter values ​​returned by the large language model and organizes them into a parameter binary table. The pre-test module is used to test the target database software according to the parameter binary table to obtain the initial transformation data sequence; as well as The DDPGFD processing module has a built-in DDPG for demonstration model, which is used to train the DDPG for demonstration model based on the initial transformation data sequence; after the DDPG for demonstration model converges, it outputs the final parameter values.

9. A readable storage medium, characterized in that, The readable storage medium stores computer program instructions, which are read and executed by a processor to perform the steps of the database configuration parameter tuning method based on large language models and deep reinforcement learning as described in any one of claims 1-7.

Images