Reinforcement learning dynamic compression strategy adjusting and optimizing method for column storage database

Abstract

The invention relates to the technical field of intelligent database optimization, in particular to a reinforcement learning dynamic compression strategy tuning method for a column storage database, and the method comprises the steps: collecting the data feature information of a current column storage database; based on the data feature information, judging whether the current column storage database meets a preset compression condition or not; and if the current column storage database meets the preset compression condition, determining a target compression strategy according to the data feature information, and executing a corresponding compression action on the current column storage database based on the target compression strategy. Therefore, data features and loads are sensed in real time through the reinforcement learning algorithm, the optimal compression algorithm is intelligently selected, the problems of low storage efficiency, poor query performance and the like caused by the fact that traditional static compression cannot adapt to dynamic changes of data in the prior art are solved, and the comprehensive performance of an HTAP system is remarkably improved.

Description

Technical Field

[0001] The present application relates to the technical field of intelligent database optimization, and in particular to a reinforcement learning dynamic compression strategy tuning method for a column storage database. Background Art

[0002] With the advent of the big data era, data volumes are exploding, posing enormous challenges to data storage and management. Column-based storage technology, due to its efficiency and superiority in processing large amounts of data, has been widely adopted in database systems. However, increasing data storage costs and optimizing data access performance remain pressing challenges. Data compression technology, as a key approach to addressing storage costs and optimizing performance, has garnered widespread attention. Adaptive column-based storage compression technology, through dynamic selection of compression algorithms, balances storage efficiency and query performance across diverse data characteristics and access patterns, and has become a hot topic of research.

[0003] In related technologies, method (1) proposes an automated column-based storage adaptive data compression system. Through a metadata management module, a feature compression algorithm library, a full compression algorithm library, and an adaptive compression prediction engine, it analyzes and processes different data and dynamically selects the optimal compression algorithm. The system can optimize compression strategies based on data characteristics and historical processing information, effectively saving storage space and improving compression efficiency. Method (2) proposes an adaptive data compression method, system, device, and product for databases. It selects a compression algorithm by analyzing data characteristics (repetition rate, missing rate) through a machine learning model.

[0004] However, method (1) still has deficiencies in dynamic adjustment capabilities, mainly relying on preset data types and feature mappings, and has weak adaptability to runtime load changes; method (2) cannot respond to dynamic load changes in real time and is not linked to query performance indicators. Traditional static data compression algorithms may lead to frequent decompression operations when processing real-time dynamically updated databases, reducing data processing efficiency; adaptive data compression technologies in related technologies still have deficiencies in responding to changes in data features in databases. For example, the value range distribution, data repetition rate, and data repetition pattern between different data are quite different. Using the same compression algorithm cannot effectively cope with these differences, resulting in excessive storage space usage and poor data query performance. These problems need to be solved urgently. Summary of the Invention

[0005] This application provides a reinforcement learning dynamic compression strategy tuning method for column-store databases to address the problems of low storage efficiency and poor query performance caused by the inability of traditional static compression to adapt to dynamic data changes in related technologies, significantly improving the overall performance of HTAP systems.

[0006] The first embodiment of the present application provides a method for optimizing a dynamic compression strategy using reinforcement learning for a column-based database, including the following steps:

[0007] Collect data feature information of the current column storage database;

[0008] Based on the data feature information, determining whether the current column store database meets a preset compression condition;

[0009] If the current column store database meets the preset compression condition, a target compression strategy is determined according to the data feature information, and a corresponding compression action is performed on the current column store database based on the target compression strategy.

[0010] Optionally, the data characteristic information includes at least one of data type characteristics, statistical data quantity, data repetition rate, data value range distribution, data volatility, load access pattern and system resource load condition.

[0011] Optionally, the determining, based on the data feature information, whether the current column store database meets a preset compression condition includes:

[0012] Based on the data characteristic information, determining current CPU usage, current memory usage, current data duplication rate, current I / O throughput, current data update frequency, current data type characteristics, and current semi-structured data;

[0013] If the current CPU usage is greater than the first usage and the current data repetition rate is less than the first preset data repetition rate, or the current memory usage is greater than the second usage and the current data repetition rate is less than the first preset data repetition rate, or the current I / O throughput reaches the preset peak value, or the current data update frequency is greater than the preset frequency, or the current data type characteristics meet the preset restriction conditions, or the nesting level of the current semi-structured data is greater than the first preset level and the key name repetition rate of the current semi-structured data is less than the first preset key name repetition rate, then it is determined that the current column storage database does not meet the preset compression conditions.

[0014] Optionally, determining a target compression strategy according to the data feature information, and performing a corresponding compression action on the current column store database based on the target compression strategy includes:

[0015] For numerical data, when the current data repetition rate is greater than the second preset data repetition rate, if the current CPU usage is less than a third usage rate, or if the current memory usage is less than a fourth usage rate, then using the RLE compression algorithm to perform a corresponding compression action on the current column store database; if the current CPU usage is greater than the third usage rate, then using the null value compression algorithm to perform a corresponding compression action on the current column store database, wherein the third usage rate is less than the first usage rate;

[0016] And, for character data or text data, when the current data repetition rate is greater than a third preset data repetition rate, using an LZW compression algorithm to perform a corresponding compression action on the current column storage database;

[0017] And, for binary data, when the entropy value is less than a preset value, using the LZ77 compression algorithm to perform a corresponding compression action on the current column storage database;

[0018] Also, for semi-structured data, when the nesting level of the current semi-structured data is less than or equal to the second preset level and the key name repetition rate of the current semi-structured data is greater than the second preset key name repetition rate, the LZW compression algorithm is used to perform corresponding compression actions on the current column storage database; when the nesting level of the current semi-structured data is greater than the second preset level, the tree compression algorithm is used to perform corresponding compression actions on the current column storage database.

[0019] Optionally, after performing a corresponding compression action on the current column store database based on the target compression strategy, the method further includes:

[0020] Dynamically generate reward values ​​and benefits for executing the target compression strategy through a preset reward model;

[0021] Based on the preset Actor-Critic algorithm, the strategy network and value network in the preset Actor-Critic algorithm are updated according to the data feature information of the current column storage database after executing the corresponding compression action, the reward value and benefit of executing the target compression strategy.

[0022] A second embodiment of the present application provides a device for optimizing a dynamic compression strategy using reinforcement learning for a column-based database, including:

[0023] The acquisition module is used to collect data feature information of the current column storage database;

[0024] a judgment module, configured to judge whether the current column storage database meets a preset compression condition based on the data feature information;

[0025] A compression module is configured to determine a target compression strategy according to the data feature information if the current column store database meets a preset compression condition, and perform a corresponding compression action on the current column store database based on the target compression strategy.

[0026] Optionally, the data characteristic information includes at least one of data type characteristics, statistical data quantity, data repetition rate, data value range distribution, data volatility, load access pattern and system resource load condition.

[0027] Optionally, the judgment module is specifically configured to:

[0028] Based on the data characteristic information, determining current CPU usage, current memory usage, current data duplication rate, current I / O throughput, current data update frequency, current data type characteristics, and current semi-structured data;

[0029] If the current CPU usage is greater than the first usage and the current data repetition rate is less than the first preset data repetition rate, or the current memory usage is greater than the second usage and the current data repetition rate is less than the first preset data repetition rate, or the current I / O throughput reaches the preset peak value, or the current data update frequency is greater than the preset frequency, or the current data type characteristics meet the preset restriction conditions, or the nesting level of the current semi-structured data is greater than the first preset level and the key name repetition rate of the current semi-structured data is less than the first preset key name repetition rate, then it is determined that the current column storage database does not meet the preset compression conditions.

[0030] Optionally, the compression module is specifically configured to:

[0031] For numerical data, when the current data repetition rate is greater than the second preset data repetition rate, if the current CPU usage is less than a third usage rate, or if the current memory usage is less than a fourth usage rate, then using the RLE compression algorithm to perform a corresponding compression action on the current column store database; if the current CPU usage is greater than the third usage rate, then using the null value compression algorithm to perform a corresponding compression action on the current column store database, wherein the third usage rate is less than the first usage rate;

[0032] And, for character data or text data, when the current data repetition rate is greater than a third preset data repetition rate, using an LZW compression algorithm to perform a corresponding compression action on the current column storage database;

[0033] And, for binary data, when the entropy value is less than a preset value, using the LZ77 compression algorithm to perform a corresponding compression action on the current column storage database;

[0034] Also, for semi-structured data, when the nesting level of the current semi-structured data is less than or equal to the second preset level and the key name repetition rate of the current semi-structured data is greater than the second preset key name repetition rate, the LZW compression algorithm is used to perform corresponding compression actions on the current column storage database; when the nesting level of the current semi-structured data is greater than the second preset level, the tree compression algorithm is used to perform corresponding compression actions on the current column storage database.

[0035] Optionally, after performing a corresponding compression action on the current column store database based on the target compression strategy, the compression module is further configured to:

[0036] Dynamically generate reward values ​​and benefits for executing the target compression strategy through a preset reward model;

[0037] Based on the preset Actor-Critic algorithm, the strategy network and value network in the preset Actor-Critic algorithm are updated according to the data feature information of the current column storage database after executing the corresponding compression action, the reward value and benefit of executing the target compression strategy.

[0038] A third aspect of the present application provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are configured to execute the reinforcement learning dynamic compression strategy tuning method for a column storage database as described in the above embodiment.

[0039] The fourth aspect of the present application provides a computer-readable storage medium having a computer program stored thereon, which is executed by a processor to implement the reinforcement learning dynamic compression strategy tuning method for a column storage database as described in the above embodiment.

[0040] In a fifth aspect, an embodiment of the present application provides a computer program product, which stores a computer program. When the program is executed by a processor, the method for optimizing the dynamic compression strategy using reinforcement learning for a column storage database is implemented as described in the above embodiment.

[0041] Therefore, the embodiments of the present application collect data feature information from the current column storage database. Based on this data feature information, when determining whether the current column storage database meets preset compression conditions, the target compression strategy is determined based on the data feature information, and corresponding compression actions are performed on the current column storage database based on the target compression strategy. Thus, through the use of a reinforcement learning algorithm to perceive data features and load in real time and intelligently select the optimal compression algorithm, this solves the problems of low storage efficiency and poor query performance caused by traditional static compression in related technologies that cannot adapt to dynamic data changes, significantly improving the overall performance of HTAP systems.

[0042] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0044] Figure 1 This is a flowchart of a method for optimizing a dynamic compression strategy using reinforcement learning for a column-based database according to an embodiment of the present application.

[0045] Figure 2 A flow chart of a reinforcement learning process for a reinforcement learning dynamic compression strategy tuning method for a column storage database provided according to one embodiment of the present application;

[0046] Figure 3 A schematic diagram of a reward model training process for a reinforcement learning dynamic compression strategy tuning method for a column-based database according to one embodiment of the present application;

[0047] Figure 4 A schematic diagram of a device for optimizing a dynamic compression strategy using reinforcement learning for a column-based database according to an embodiment of the present application;

[0048] Figure 5 A schematic diagram of the structure of an electronic device provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0049] The following describes in detail embodiments of the present application, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0050] The following describes the reinforcement learning dynamic compression strategy tuning method for column storage databases according to an embodiment of the present application with reference to the accompanying drawings. In response to the problem that the traditional static compression in the related art mentioned in the above background technology cannot adapt to the low storage efficiency and poor query performance caused by dynamic changes in data, the present application provides a reinforcement learning dynamic compression strategy tuning method for column storage databases. In this method, the embodiment of the present application collects data feature information of the current column storage database, and based on the data feature information, determines that when the current column storage database meets the preset compression conditions, it determines the target compression strategy according to the data feature information, and performs the corresponding compression action on the current column storage database based on the target compression strategy. Thus, the data features and load are perceived in real time by the reinforcement learning algorithm, and the optimal compression algorithm is intelligently selected, which solves the problem of low storage efficiency and poor query performance caused by the traditional static compression in the related art that cannot adapt to dynamic changes in data, and significantly improves the comprehensive performance of the HTAP (Hybrid Transactional / Analytical Processing, a database system architecture that can simultaneously process transactional and analytical workloads) system.

[0051] Specifically, Figure 1 A flowchart of a reinforcement learning dynamic compression strategy tuning method for a column storage database provided in an embodiment of the present application.

[0052] like Figure 1 As shown, the reinforcement learning dynamic compression strategy tuning method for column storage database includes the following steps:

[0053] In step S101 , data feature information of the current column store database is collected.

[0054] Optionally, in some embodiments, the data characteristic information includes at least one of data type characteristics, statistical data quantity, data repetition rate, data value range distribution, data volatility, load access pattern and system resource load conditions.

[0055] Specifically, the agent (an entity that can interact with the environment and take actions to maximize cumulative rewards) collects the status information of the current column storage database (the state of the environment at a certain moment, usually described by a set of features) through an interactive interface with the current column storage database Environment (the external world in which the agent is located, and the agent learns the optimal strategy through interaction with the environment); metadata acquisition and classification involve obtaining table structure information, including table name, field name, and data type; automatic classification according to data type, for example, numeric types include integer (INT) and floating point type (FLOAT); character types include string (VARCHAR) and text (TEXT); binary types include BLOB and binary stream; semi-structured data includes JSON and XML fields. Acquiring data features involves calculating statistical quantity, namely the original size of the data; repetition rate, namely calculating the proportion of unique values ​​for numeric / character fields; calculating entropy values ​​for binary data blocks (such as 1024 bytes) to detect local repetitive patterns; detecting key name repetition rate and label redundancy for semi-structured data; value range distribution, namely calculating the minimum, maximum, and range of numeric fields; volatility, namely the standard deviation or sliding variance within a time window (such as every hour), and the incremental change trend of numeric fields. Acquiring load and access pattern features involves data operation frequency, namely counting the number of operations per minute on a field and distinguishing between cold data (low frequency) and hot data (high frequency); query complexity, namely parsing SQL statements, counting the number of JOIN operations and the nesting level of subqueries; and read-write ratio, namely the ratio of transactional (OLTP) write operations to analytical (OLAP) read operations. Collect system resource usage, including indicators such as CPU utilization and memory utilization.

[0056] In step S102 , based on the data feature information, it is determined whether the current column store database meets the preset compression condition.

[0057] Specifically, the intelligent agent determines whether the current column storage database meets the preset compression conditions based on the collected data feature information and the preset threshold.

[0058] Optionally, in some embodiments, based on data feature information, whether the current column storage database meets the preset compression conditions is determined, including: based on the data feature information, determining the current CPU usage, current memory usage, current data repetition rate, current I / O throughput, current data update frequency, current data type characteristics and current semi-structured data; if the current CPU usage is greater than the first usage and the current data repetition rate is less than the first preset data repetition rate, or the current memory usage is greater than the second usage and the current data repetition rate is less than the first preset data repetition rate, or the current I / O throughput reaches a preset peak, or the current data update frequency is greater than the preset frequency, or the current data type characteristics meet the preset restriction conditions, or the nesting level of the current semi-structured data is greater than the first preset level and the key name repetition rate of the current semi-structured data is less than the first preset key name repetition rate, then it is determined that the current column storage database does not meet the preset compression conditions.

[0059] Among them, the first usage rate, the first preset data repetition rate, the second usage rate, the preset peak value, the preset frequency, the preset restriction condition, the first preset level and the first preset key name repetition rate can be thresholds set in advance by the user, can be thresholds obtained through a limited number of experiments, or can be thresholds obtained through a limited number of computer simulations, and no specific limitation is made here.

[0060] It is understandable that if the following scenarios occur, it is judged that the current column storage database does not meet the preset compression conditions, that is, system resources are overloaded, for example, when CPU utilization is >90% or memory utilization is >95%, and data duplication rate is <10%; I / O throughput reaches the preset peak to avoid compression operations exacerbating resource competition; compression benefits are insufficient, for example, data update frequency is >100 times / second, and compression-decompression overhead exceeds benefits; data type feature limitations, such as highly dynamic binary streams (such as real-time video streams), compression takes too long; semi-structured data (such as randomized JSON), nesting level >5 and key name duplication rate <20%.

[0061] In step S103 , if the current column store database meets the preset compression conditions, a target compression strategy is determined according to the data feature information, and a corresponding compression action is performed on the current column store database based on the target compression strategy.

[0062] Specifically, when the current column storage database meets the preset compression conditions, an independent thread is created to perform the compression task to avoid blocking the main database process. CPU resources are dynamically allocated according to load priority (such as OLTP transactions take precedence over compression tasks). The embodiment of the present application only compresses cold data or historical archived data, and hot data is not compressed for the time being. Read the original data, process the data according to the selected strategy, and if compression is performed, generate a compressed data file and store the compressed data file in a specified location. If the current column storage database does not meet the preset compression conditions, no operation is performed.

[0063] Optionally, in some embodiments, a target compression strategy is determined according to data feature information, and a corresponding compression action is performed on the current column storage database based on the target compression strategy, including: for numerical data, when the current data repetition rate is greater than the second preset data repetition rate, if the current CPU usage is less than the third usage rate, or the current memory usage is less than the fourth usage rate, then the RLE compression algorithm is used to perform the corresponding compression action on the current column storage database; if the current CPU usage is greater than the third usage rate, then the null value compression algorithm is used to perform the corresponding compression action on the current column storage database, wherein the third usage rate is less than the first usage rate; and for character data or text data, when the current data When the repetition rate is greater than the third preset data repetition rate, the LZW compression algorithm is used to perform corresponding compression actions on the current column storage database; and, for binary data, when the entropy value is less than the preset value, the LZ77 compression algorithm is used to perform corresponding compression actions on the current column storage database; and, for semi-structured data, when the nesting level of the current semi-structured data is less than or equal to the second preset level and the key name repetition rate of the current semi-structured data is greater than the second preset key name repetition rate, the LZW compression algorithm is used to perform corresponding compression actions on the current column storage database; when the nesting level of the current semi-structured data is greater than the second preset level, the tree compression algorithm is used to perform corresponding compression actions on the current column storage database.

[0064] Among them, the second preset data repetition rate, the third usage rate, the fourth usage rate, the third preset data repetition rate, the second preset level, and the second preset key name repetition rate can be thresholds pre-set by the user, can be thresholds obtained through a limited number of experiments, or can be thresholds obtained through a limited number of computer simulations, and are not specifically limited here. The RLE compression algorithm is a simple lossless compression algorithm that achieves data compression by replacing consecutive repeated elements with element values ​​and the number of repetitions; the null value compression algorithm is a compression algorithm for data containing a large number of null values, which achieves data compression by only storing non-null values ​​and their positions; the LZW compression algorithm is a lossless compression algorithm that achieves data compression by establishing a dictionary to store frequently occurring character strings; the LZ77 compression algorithm is a lossless compression algorithm that achieves data compression by searching for previously occurring character string sequences and replacing them with pointers.

[0065] It is understandable that the target compression strategy is determined based on data feature information. For numerical data, in high-repetition scenarios (repetition rate ≥ 70%): If system resources are sufficient (CPU usage < 40%, memory usage < 50%), the RLE compression algorithm is selected to perform the corresponding compression action on the current column storage database, significantly reducing storage space by encoding consecutive repeated values. If resources are tight (CPU usage > 70%), the null value compression algorithm is selected to only store non-null values ​​and their positions, reducing computing overhead. For character / text data, when the repetition rate is > 50%, the LZW compression algorithm is used to perform the corresponding compression action on the current column storage database; when the repetition rate is < 30% and resources are tight, no compression is selected. For binary data, when the entropy value is low, the LZ77 compression algorithm is used to perform the corresponding compression action on the current column storage database; when the entropy value is high and compression takes too long, no compression is selected. For semi-structured data, when the nesting level is ≤ 3 and the key name repetition rate is > 60%, the LZW compression algorithm is used to perform the corresponding compression action on the current column storage database; when the level is > 3, the tree compression algorithm is used.

[0066] It should be noted that, depending on the different data stored in the database and the different scenarios, the types and number of compression algorithms in the compression algorithm set may change, and the strategies used may also be different.

[0067] Therefore, the embodiments of the present application aim to address the defects of existing column storage database compression technology, and provide a reinforcement learning dynamic compression strategy tuning method for column storage databases. Through the reinforcement learning algorithm, adaptive tuning of the dynamic compression strategy of the column storage database is achieved, thereby improving the storage efficiency and query performance of the column storage database and reducing storage costs.

[0068] Optionally, in some embodiments, after performing a corresponding compression action on the current column storage database based on the target compression strategy, it also includes: dynamically generating a reward value and benefit for executing the target compression strategy through a preset reward model; based on a preset Actor-Critic algorithm, updating a strategy network and a value network in the preset Actor-Critic algorithm according to the data feature information of the current column storage database after executing the corresponding compression action, and the reward value and benefit for executing the target compression strategy.

[0069] Among them, the preset Actor-Critic algorithm is a reinforcement learning algorithm that combines the advantages of the policy gradient method and the value function method, and optimizes the strategy through the collaborative learning of two networks (Actor and Critic); the reward value is the feedback obtained by the agent from the environment after performing an action, which is used to evaluate the quality of the action.

[0070] Specifically, if Figure 2 and Figure 3 As shown, Figure 2 This is a flow chart of a reinforcement learning process for a reinforcement learning dynamic compression strategy tuning method for a column storage database according to one embodiment of the present application. Figure 3This is a schematic diagram of the reward model training process of a reinforcement learning dynamic compression strategy tuning method for a column storage database according to an embodiment of the present application; the preset reward model is trained separately, independently of the policy network and value network in the Actor-Critic algorithm, and is obtained through training with historical data and an online real-time experience pool; wherein, the input of the preset reward model includes storage space related, i.e., data repetition rate, storage occupancy before compression, field value range distribution, etc.; query speed related, i.e., query complexity (such as the number of JOIN operations), data block access frequency, etc.; action information, i.e., the compression strategy selected by the agent (including specific algorithms or "no compression"); historical performance baseline, i.e., the difference in storage occupancy before and after compression (ΔStorage), query delay change (ΔQuerySpeed). The output of the preset reward model includes a dynamic reward value, which comprehensively considers the benefits of reduced storage space and improved query speed. The preset reward model is trained, recording changes in storage usage (ΔStorage) and query latency (ΔQuerySpeed) after each action. This includes a historical experience pool, which stores the past 100,000 interaction data in a circular experience replay buffer to ensure data timeliness. Data normalization also standardizes ΔStorage (as a percentage of total storage) and ΔQuerySpeed ​​(as a percentage of query latency reduction) to eliminate dimensional differences. The preset reward model is trained and updated by continuously updating the experience pool with historical data and real-time data obtained online during operation. The architecture of the preset reward model adopts a feedforward neural network structure. The input layer receives state features and action information, the hidden layer uses the ReLU activation function, and the output layer uses the Tanh activation function. The reward value is constrained to the [-1,1] range. Negative values ​​indicate performance degradation, and positive values ​​indicate performance improvement.

[0071] Furthermore, the preset reward model training process begins with data preparation. Samples containing ΔStorage and ΔQuerySpeed ​​records are extracted from historical database logs and normalized. The network weights and bias parameters are initialized based on the input feature dimensions and the preset network structure. The preprocessed training data is fed into the network, and forward propagation is performed to obtain the model-predicted reward value. A loss function, such as the mean squared error (MSE), is defined to measure the difference between the model-predicted reward value and the actual observed reward value. The gradient of the loss function with respect to the network parameters is calculated using a backpropagation algorithm, and an optimization algorithm (such as Adam) is used to update the network parameters to minimize the loss function. Supervised learning is performed using historical database logs to minimize the mean squared error (MSE) between the predicted reward and the actual observed reward. After accumulating 100 new experiences, batches of data are randomly sampled from the buffer, and the model parameters are fine-tuned based on real-time performance metrics. The reward model then generates a composite reward value based on the acquired core state features and the executed actions.

[0072] Furthermore, based on the obtained comprehensive reward, the executed action, and the new state, the policy network and value network are updated using a pre-defined actor-critic algorithm, thereby updating the target compression policy. The actor is responsible for selecting actions based on the current state, i.e., generating the policy; the critic is responsible for evaluating the selected actions, i.e., evaluating the state-value function. Through collaborative learning between the two, the agent can more effectively update its policy to adapt to the dynamically changing database environment. First, the target compression policy network (actor) and the value network (critic) are initialized. The target compression policy network is constructed by constructing a feedforward neural network with the number of neurons in the input layer equal to the dimension of the environment state and the number of neurons in the output layer equal to the size of the action space. The number of neurons and layers in the hidden layer can be adjusted based on the complexity of the task. The commonly used hidden layer activation function is Reinforced Lu (ReLU). The weight matrix and bias vector in the network are initialized. Common methods include Glorot uniform initialization or He normal initialization to ensure that the network initial parameters are reasonable and avoid vanishing or exploding gradients. The output layer typically uses a softmax activation function to convert the network output into a probability distribution, representing the probability of selecting each action in a given state. The value network is constructed by constructing a feedforward neural network. The number of neurons in the input layer is equal to the dimension of the environment state, and the output layer consists of a single neuron, which is used to estimate the state-value function. Similar to the policy network, the weight matrix and bias vector are initialized using appropriate initialization methods. The mean squared error (MSE) is selected as the loss function for the value network to measure the difference between the estimated value and the true value.

[0073] Furthermore, at each time step, the agent selects an action based on its current state through the target compression policy network. After executing the action, it obtains a reward and the next state. The agent obtains information about the current state of the environment through an interactive interface with the environment. This state can include features of database data, load access patterns, system resource usage, and more. The obtained state undergoes preprocessing, such as normalization and standardization, to ensure that the data input to the network falls within an appropriate numerical range, improving network training effectiveness. Then, the agent selects an action and inputs the preprocessed state into the policy network to obtain an action probability distribution. Based on this probability distribution, a specific action is selected using a sampling method (such as probability-based random sampling). Finally, the action is executed and feedback is obtained. The selected action is then executed in the environment, such as applying a specific compression strategy to compress data in a database. After executing the action, the agent obtains a reward and the next state from the environment. The reward is dynamically calculated based on the reward model.

[0074] Furthermore, based on the reward value and the value function of the next state, the parameters of the value network are updated to reduce the estimation error of the value function. The current state is input into the value network to obtain the estimated value of the current state; the next state is input into the value network to obtain the estimated value of the next state. Based on the reward value and the estimated value of the next state, the Temporal Difference (TD) target is calculated; it is used to balance the importance of immediate rewards and future rewards. Finally, the mean squared error loss function is used to calculate the loss between the estimated value of the value network and the TD target. The gradient of the loss with respect to the value network parameters is calculated through the backpropagation algorithm, and the parameters are updated using an optimization algorithm (such as Adam) to minimize the loss and reduce the estimation error of the value function.

[0075] Furthermore, based on the updated value network, the gradient of the target compression policy network is calculated, and the parameters of the policy network are updated to maximize the long-term cumulative reward. Using the updated value network, the advantage function value of the current action is calculated. The current state is then input into the policy network, and the gradient of the action probability distribution with respect to the network parameters is calculated. The advantage function value is then multiplied by the probability gradient to obtain the gradient of the target compression policy. This step incorporates the value information into the policy update, guiding the target compression policy to adjust in the direction of improving the long-term cumulative reward. Finally, an optimization algorithm (such as Adam) is used to update the parameters of the target compression policy network based on the target compression policy gradient. The goal of the update is to maximize the long-term cumulative reward, that is, to adjust the parameters so that actions with higher advantage values ​​are more likely to be selected under similar states.

[0076] Repeat the above steps until the target compression policy converges; at each time step, the agent continuously interacts with the environment, updating the parameters of the policy network and value network, and updating the preset reward model. Convergence is determined when the change in the policy network parameters is less than a certain threshold (for example, a parameter change of less than 0.001), or when the actions selected by the policy remain stable over multiple consecutive time steps. When the convergence criteria are met, training is terminated, and the agent is deemed to have learned a relatively stable optimal policy.

[0077] It should be noted that in addition to the Actor-Critic algorithm, the embodiments of the present application can also use other reinforcement learning algorithms, such as DQN, A3C and DDPG, to achieve dynamic tuning of the compression strategy; the embodiments of the present application can select different state feature combinations according to actual needs and database characteristics, such as adding data access frequency, data update frequency and other features to more comprehensively describe the state of the database; in addition to compression algorithms such as LZW, RLE, null value, LZ77, and differential, the embodiments of the present application can also select other suitable compression algorithms according to data characteristics and application scenarios, such as Huffman coding, arithmetic coding, etc.

[0078] Therefore, the embodiments of the present application are highly flexible and can adjust actions according to real-time status characteristics to adapt to the diversity of data characteristics and load access patterns; the embodiments of the present application are efficient and intelligent, and automatically learn the optimal compression strategy through the optimization of the reinforcement learning algorithm without the need for a large amount of manual intervention; the embodiments of the present application have excellent performance, and calculate the reward value based on multiple performance indicators to ensure that the database achieves the best compression effect; the embodiments of the present application are highly versatile and applicable to various types of databases and data characteristics, have good versatility and adaptability, and can meet the database compression needs in different scenarios.

[0079] According to the reinforcement learning dynamic compression strategy tuning method for column-store databases proposed in the embodiments of this application, the embodiments of this application collect data feature information of the current column-store database. Based on this data feature information, when it determines whether the current column-store database meets preset compression conditions, it determines a target compression strategy based on the data feature information and performs corresponding compression actions on the current column-store database based on the target compression strategy. Thus, through the reinforcement learning algorithm, real-time perception of data features and load is achieved, intelligent selection of the optimal compression algorithm is achieved, solving the problem of low storage efficiency and poor query performance caused by traditional static compression in related technologies that cannot adapt to dynamic data changes, significantly improving the overall performance of HTAP systems.

[0080] Next, a reinforcement learning dynamic compression strategy tuning device for a column storage database proposed in an embodiment of the present application is described with reference to the accompanying drawings.

[0081] Figure 4 It is a block diagram of a device for optimizing a dynamic compression strategy using reinforcement learning for a column storage database according to an embodiment of the present application.

[0082] like Figure 4 As shown, the reinforcement learning dynamic compression strategy tuning device 10 for column storage database includes: a collection module 100, a judgment module 200 and a compression module 300.

[0083] The acquisition module 100 is used to acquire data feature information of the current column storage database;

[0084] A determination module 200 is configured to determine whether the current column store database meets a preset compression condition based on data feature information;

[0085] The compression module 300 is configured to determine a target compression strategy based on data feature information if the current column store database meets preset compression conditions, and perform corresponding compression actions on the current column store database based on the target compression strategy.

[0086] Optionally, the data characteristic information includes at least one of data type characteristics, statistical data quantity, data repetition rate, data value range distribution, data volatility, load access pattern and system resource load condition.

[0087] Optionally, the judgment module 200 is specifically used to: determine the current CPU usage, current memory usage, current data repetition rate, current I / O throughput, current data update frequency, current data type characteristics and current semi-structured data based on data feature information; if the current CPU usage is greater than the first usage and the current data repetition rate is less than the first preset data repetition rate, or, the current memory usage is greater than the second usage and the current data repetition rate is less than the first preset data repetition rate, or, the current I / O throughput reaches a preset peak value, or, the current data update frequency is greater than the preset frequency, or, the current data type characteristics meet the preset restriction conditions, or, the nesting level of the current semi-structured data is greater than the first preset level and the key name repetition rate of the current semi-structured data is less than the first preset key name repetition rate, then it is determined that the current column storage database does not meet the preset compression conditions.

[0088] Optionally, the compression module 300 is specifically configured to: for numerical data, when the current data repetition rate is greater than the second preset data repetition rate, if the current CPU usage rate is less than the third usage rate, or the current memory usage rate is less than the fourth usage rate, use the RLE compression algorithm to perform a corresponding compression action on the current column storage database; if the current CPU usage rate is greater than the third usage rate, use the null value compression algorithm to perform a corresponding compression action on the current column storage database, wherein the third usage rate is less than the first usage rate; and for character data or text data, when the current data repetition rate is greater than the third preset data repetition rate, use the LZW compression algorithm to perform a corresponding compression action on the current column storage database. The compression algorithm performs corresponding compression actions on the current column storage database; and, for binary data, when the entropy value is less than a preset value, the LZ77 compression algorithm is used to perform corresponding compression actions on the current column storage database; and, for semi-structured data, when the nesting level of the current semi-structured data is less than or equal to the second preset level and the key name repetition rate of the current semi-structured data is greater than the second preset key name repetition rate, the LZW compression algorithm is used to perform corresponding compression actions on the current column storage database; when the nesting level of the current semi-structured data is greater than the second preset level, the tree compression algorithm is used to perform corresponding compression actions on the current column storage database.

[0089] Optionally, after executing the corresponding compression action on the current column storage database based on the target compression strategy, the compression module 300 is further used to: dynamically generate the reward value and benefit of executing the target compression strategy through a preset reward model; based on the preset Actor-Critic algorithm, update the strategy network and value network in the preset Actor-Critic algorithm according to the data feature information of the current column storage database after executing the corresponding compression action, the reward value and benefit of executing the target compression strategy.

[0090] It should be noted that the above explanation of the embodiment of the reinforcement learning dynamic compression strategy tuning method for a column storage database is also applicable to the reinforcement learning dynamic compression strategy tuning device for a column storage database in this embodiment, and will not be repeated here.

[0091] According to the reinforcement learning dynamic compression strategy tuning device for column-store databases proposed in an embodiment of the present application, the embodiment of the present application collects data feature information of the current column-store database. Based on this data feature information, when it determines that the current column-store database meets preset compression conditions, it determines a target compression strategy based on the data feature information and performs corresponding compression actions on the current column-store database based on the target compression strategy. Thus, through the reinforcement learning algorithm, real-time perception of data features and load is achieved, intelligent selection of the optimal compression algorithm is achieved, solving the problem of low storage efficiency and poor query performance caused by traditional static compression in related technologies that cannot adapt to dynamic data changes, significantly improving the overall performance of HTAP systems.

[0092] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device may include:

[0093] Memory 501 , processor 502 , and computer programs stored in the memory 501 and executable on the processor 502 .

[0094] When the processor 502 executes the program, the reinforcement learning dynamic compression strategy tuning method for a column store database provided in the above embodiment is implemented.

[0095] Furthermore, the electronic device further includes:

[0096] The communication interface 503 is used for communication between the memory 501 and the processor 502 .

[0097] The memory 501 is used to store computer programs that can be run on the processor 502 .

[0098] The memory 501 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0099] If the memory 501, processor 502, and communication interface 503 are implemented independently, the communication interface 503, memory 501, and processor 502 can be connected to each other via a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0100] Optionally, in a specific implementation, if the memory 501, the processor 502 and the communication interface 503 are integrated on a chip, the memory 501, the processor 502 and the communication interface 503 can communicate with each other through an internal interface.

[0101] The processor 502 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0102] An embodiment of the present application further provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the program implements the above-mentioned reinforcement learning dynamic compression strategy tuning method for a column storage database.

[0103] An embodiment of the present application further provides a computer program product, which stores a computer program. When the program is executed by a processor, the program implements the above-mentioned reinforcement learning dynamic compression strategy tuning method for a column storage database.

[0104] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0105] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of technical features indicated. Thus, a feature specified as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of this application, "N" means at least two, for example, two, three, etc., unless otherwise specifically defined.

[0106] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may be performed out of the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present application belong.

[0107] It should be understood that various parts of the present application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiment, the N steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used to implement: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0108] Those skilled in the art will understand that all or part of the steps in the method of the above embodiment can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiment.

Claims

1. A reinforcement learning dynamic compression strategy tuning method for column storage database, characterized in that: The following steps are involved: Collect data feature information of the current column storage database; Based on the data feature information, determining whether the current column store database meets a preset compression condition; If the current column store database meets the preset compression condition, a target compression strategy is determined according to the data feature information, and a corresponding compression action is performed on the current column store database based on the target compression strategy.

2. The method according to claim 1, characterized in that The data characteristic information includes at least one of data type characteristics, statistical data quantity, data repetition rate, data value range distribution, data volatility, load access pattern and system resource load condition.

3. The method according to claim 2, characterized in that The determining, based on the data feature information, whether the current column store database satisfies a preset compression condition includes: Based on the data characteristic information, determining current CPU usage, current memory usage, current data duplication rate, current I / O throughput, current data update frequency, current data type characteristics, and current semi-structured data; If the current CPU usage is greater than the first usage and the current data repetition rate is less than the first preset data repetition rate, or the current memory usage is greater than the second usage and the current data repetition rate is less than the first preset data repetition rate, or the current I / O throughput reaches the preset peak value, or the current data update frequency is greater than the preset frequency, or the current data type characteristics meet the preset restriction conditions, or the nesting level of the current semi-structured data is greater than the first preset level and the key name repetition rate of the current semi-structured data is less than the first preset key name repetition rate, then it is determined that the current column storage database does not meet the preset compression conditions.

4. The method according to claim 1, wherein The determining a target compression strategy according to the data feature information, and performing a corresponding compression action on the current column store database based on the target compression strategy, includes: For numerical data, when the current data repetition rate is greater than the second preset data repetition rate, if the current CPU usage is less than a third usage rate, or if the current memory usage is less than a fourth usage rate, then using the RLE compression algorithm to perform a corresponding compression action on the current column store database; if the current CPU usage is greater than the third usage rate, then using the null value compression algorithm to perform a corresponding compression action on the current column store database, wherein the third usage rate is less than the first usage rate; And, for character data or text data, when the current data repetition rate is greater than a third preset data repetition rate, using an LZW compression algorithm to perform a corresponding compression action on the current column storage database; And, for binary data, when the entropy value is less than a preset value, using the LZ77 compression algorithm to perform a corresponding compression action on the current column storage database; Also, for semi-structured data, when the nesting level of the current semi-structured data is less than or equal to the second preset level and the key name repetition rate of the current semi-structured data is greater than the second preset key name repetition rate, the LZW compression algorithm is used to perform corresponding compression actions on the current column storage database; when the nesting level of the current semi-structured data is greater than the second preset level, the tree compression algorithm is used to perform corresponding compression actions on the current column storage database.

5. The method according to claim 1, wherein After performing a corresponding compression action on the current column store database based on the target compression strategy, the method further includes: Dynamically generate reward values ​​and benefits for executing the target compression strategy through a preset reward model; Based on the preset Actor-Critic algorithm, the strategy network and value network in the preset Actor-Critic algorithm are updated according to the data feature information of the current column storage database after executing the corresponding compression action, the reward value and benefit of executing the target compression strategy.

6. A reinforcement learning dynamic compression strategy tuning device for column storage database, characterized in that: include: The acquisition module is used to collect data feature information of the current column storage database; a judgment module, configured to judge whether the current column storage database meets a preset compression condition based on the data feature information; A compression module is configured to determine a target compression strategy according to the data feature information if the current column store database meets a preset compression condition, and perform a corresponding compression action on the current column store database based on the target compression strategy.

7. The device according to claim 6, characterized in that The data characteristic information includes at least one of data type characteristics, statistical data quantity, data repetition rate, data value range distribution, data volatility, load access pattern and system resource load condition.

8. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the reinforcement learning dynamic compression strategy tuning method for a column storage database according to any one of claims 1 to 5.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement the reinforcement learning dynamic compression strategy tuning method for a column storage database according to any one of claims 1 to 5.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the method for optimizing a dynamic compression strategy using reinforcement learning for a column storage database is implemented.

Images