Self-adaptive storage time partition implementation method for time sequence database
By deploying the master and standby nodes in the timing database and building a deep convolutional neural network algorithm model, adaptive storage time partitioning is realized, the problem of timing data occupies resources, and the database performance and storage efficiency are improved.
Patent Information
- Application Number
- CN202410949040.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-16
- Publication Date
- 2024-12-06
AI Technical Summary
The huge amount of time series data affects database performance and occupies a lot of storage space, resulting in waste of resources.
Adaptive storage time partitioning method is adopted, by deploying the main node and backup node, building a deep convolutional neural network algorithm model, data segmentation and optimization are performed according to the storage time partition parameters, and data switching of the main and backup nodes are regularly adjusted, and backup nodes are queried first. Meet query needs with merged data.
Improve database performance, reduce storage resource usage, and dynamically adjust the storage of primary and secondary nodes to flexibly optimize the storage and query efficiency of timing data.
Smart Images

Figure CN119088876A_ABST
Abstract
Description
Technical Field
[0001] The invention discloses a method, and relates to the technical field of database management, in particular to a method for realizing adaptive storage time partitioning for a time series database. Background Art
[0002] Time series data refers to time series data. Time series data is a data column recorded in chronological order for the same indicator, such as data generated by equipment monitoring sensors, IT system monitoring data, etc. The amount of time series data is often very large. Using time series data for trend display will affect the performance of the display program, and a particularly large amount of time series data will take up a lot of storage space, resulting in a waste of resources. Summary of the invention
[0003] In view of the problems of the prior art, the present invention provides a method for implementing adaptive storage time partitioning for a time series database, which can improve database performance and reduce storage resource occupancy.
[0004] The specific scheme proposed by the present invention is:
[0005] The present invention provides a method for implementing adaptive storage time partitioning for a time series database, comprising:
[0006] Step 1: Deploy the master node and slave node of the time series database, and perform bidirectional data synchronization between the master node and the slave node.
[0007] Step 2: Set the parameters of each storage time partition.
[0008] Step 3: Build an algorithm model for adaptive storage time partitioning, obtain the parameters of each storage time partition through the algorithm model, use the time series data written to the master node and related data to train the algorithm model, and divide the data according to different storage time partition parameters to compare and obtain the optimal storage time partition.
[0009] Step 4: Use the standby node to import the time series data of the primary node according to the optimal storage time partition, and switch the primary and standby nodes.
[0010] Step 5: Regularly select the optimal storage time partition through the algorithm model, repeat the process of the standby node importing the time series data of the primary node according to the optimal storage time partition, and perform the master-standby node switching process.
[0011] Furthermore, in the adaptive storage time partition implementation method for a time series database, a storage time partition is selected in step 3, the time series data is written to the write master node, and the time series data and related business reading and other operation data are passed to the algorithm model for algorithm model training.
[0012] Furthermore, in step 3 of the method for implementing adaptive storage time partitioning for a time series database, a deep convolutional neural network is used to build an algorithm model for adaptive storage time partitioning, and the deep convolutional neural network of the algorithm model is trained using the time series data and related data written to the master node.
[0013] Furthermore, the adaptive storage time partitioning implementation method for a time series database also includes step 6: when performing a time series data query, the data of the standby node is queried first. If the time series data span in the standby node does not meet the query time span, the master node is queried for the time series data within the time span missing from the standby node, the data obtained by the standby node query and the data obtained by the master node query are merged, and the result is returned.
[0014] The present invention also provides an adaptive storage time partitioning implementation device for a time series database, comprising a deployment module, a parameter management module, a model management module and a storage management module.
[0015] The deployment module deploys the master node and the standby node of the time series database, and performs bidirectional data synchronization writing between the master node and the standby node.
[0016] The parameter management module sets the parameters of each storage time partition.
[0017] The model management module builds an algorithm model for adaptive storage time partitioning, obtains the parameters of each storage time partition through the algorithm model, and uses the time series data and related data written to the master node to train the algorithm model. The algorithm model divides the data according to different storage time partition parameters, and compares and obtains the optimal storage time partition.
[0018] The storage management module imports the time series data of the master node through the standby node according to the optimal storage time partition, and switches the master and standby nodes.
[0019] The model management module regularly selects the optimal storage time partition through the algorithm model, and the storage management module repeats the process of the standby node importing the time series data of the primary node according to the optimal storage time partition to switch the primary and standby nodes.
[0020] Furthermore, the model management module of the adaptive storage time partition implementation device for a time series database selects a storage time partition, writes time series data to the write master node, and passes the time series data and related business reading and other operation data to the algorithm model for algorithm model training.
[0021] Furthermore, the model management module of the adaptive storage time partitioning implementation device for a time series database uses a deep convolutional neural network to build an algorithm model for adaptive storage time partitioning, and uses the time series data and related data written into the main node to train the deep convolutional neural network of the algorithm model.
[0022] Furthermore, the adaptive storage time partitioning implementation device for a time series database also includes a query module. When the query module performs a time series data query, it gives priority to querying the data of the standby node. If the time series data span in the standby node does not meet the query time span, the main node is queried for the time series data within the time span missing from the standby node, the data obtained by the standby node query and the data obtained by the main node query are merged, and the result is returned.
[0023] The benefits of the present invention are:
[0024] The present invention deploys the master node and standby node of the time series database, sets the parameters of each storage time partition, and builds an algorithm model for adaptive storage time partition. The algorithm model can be used to perform data segmentation according to different storage time partition parameters, compare and obtain the optimal storage time partition, and the standby node imports the time series data of the master node according to the optimal storage time partition, and switches the master and standby nodes, which can improve database performance and reduce storage resource occupancy. At the same time, the algorithm model can also be used to regularly screen the optimal storage time partition, and the standby node can repeat the process of importing the time series data of the master node according to the optimal storage time partition, and switching the master and standby nodes, so as to flexibly adjust the storage of the master and standby nodes. BRIEF DESCRIPTION OF THE DRAWINGS
[0025] Figure 1 It is a schematic flow chart of the method of the present invention. DETAILED DESCRIPTION
[0026] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it, but the embodiments are not intended to limit the present invention.
[0027] Example 1
[0028] The present invention provides a method for implementing adaptive storage time partitioning for a time series database, comprising:
[0029] Step 1: Deploy the master node and slave node of the time series database, and perform bidirectional data synchronization between the master node and the slave node.
[0030] Step 2: Set the parameters of each storage time partition.
[0031] Step 3: Build an algorithm model for adaptive storage time partitioning, obtain the parameters of each storage time partition through the algorithm model, use the time series data written to the master node and related data to train the algorithm model, and divide the data according to different storage time partition parameters to compare and obtain the optimal storage time partition.
[0032] The deep convolutional neural network can be further used to build an algorithm model for adaptive storage time partitioning, and the deep convolutional neural network of the algorithm model can be trained using the time series data and related data written to the master node.
[0033] For example, initially select a storage time partition, write time series data to the write master node, pass the time series data and related business reading and other operation data to the algorithm model, use the data written to the model to perform deep convolutional neural network training of the storage time partition algorithm model, use different storage time partition parameters to split the data, and perform business reading and other operations, compare which storage time partition database has better performance and occupies less resources, and predict future data conditions based on existing data to select the optimal storage time partition.
[0034] Step 4: Use the standby node to import the time series data of the master node according to the optimal storage time partition, and switch the master and standby nodes. When switching the standby node to the master node, you can clean up the data written in the original master node.
[0035] Step 5: Regularly select the optimal storage time partition through the algorithm model, repeat the process of the standby node importing the time series data of the primary node according to the optimal storage time partition, and perform the master-standby node switching process.
[0036] It may further include step 6: when querying time series data, give priority to querying the data of the standby node. If the time series data span in the standby node does not meet the time span of the query, query the main node for the time series data within the time span missing from the standby node, merge the data obtained from the standby node query and the data obtained from the main node query, and return the result.
[0037] Example 2
[0038] The present invention also provides an adaptive storage time partitioning implementation device for a time series database, comprising a deployment module, a parameter management module, a model management module and a storage management module.
[0039] The deployment module deploys the master node and the standby node of the time series database, and performs bidirectional data synchronization writing between the master node and the standby node.
[0040] The parameter management module sets the parameters of each storage time partition.
[0041] The model management module builds an algorithm model for adaptive storage time partitioning, obtains the parameters of each storage time partition through the algorithm model, and uses the time series data and related data written to the master node to train the algorithm model. The algorithm model divides the data according to different storage time partition parameters, and compares and obtains the optimal storage time partition.
[0042] The storage management module imports the time series data of the master node through the standby node according to the optimal storage time partition, and switches the master and standby nodes.
[0043] The model management module regularly selects the optimal storage time partition through the algorithm model, and the storage management module repeats the process of the standby node importing the time series data of the primary node according to the optimal storage time partition to switch the primary and standby nodes.
[0044] As the information interaction, execution process and other contents between the modules in the above-mentioned device are based on the same concept as the embodiment of the method of the present invention, the specific contents can be found in the description of the embodiment of the method of the present invention and will not be repeated here.
[0045] Likewise, the benefits of the device of the present invention are:
[0046] Deploy the master node and standby node of the time series database, set the parameters of each storage time partition, and build an algorithm model for adaptive storage time partition. The algorithm model can be used to segment data according to different storage time partition parameters, compare and obtain the optimal storage time partition, and import the time series data of the master node according to the optimal storage time partition through the standby node, and switch the master and standby nodes, which can improve database performance and reduce storage resource usage. At the same time, the algorithm model can be used to regularly select the optimal storage time partition, and the standby node can be repeated to import the time series data of the master node according to the optimal storage time partition, and the master and standby node switching process can be performed to flexibly adjust the storage of the master and standby nodes.
[0047] It should be noted that not all steps and modules in the above-mentioned processes and device structures are necessary, and some steps or modules can be ignored according to actual needs. The execution order of each step is not fixed and can be adjusted as needed. The system structure described in the above-mentioned embodiments can be a physical structure or a logical structure, that is, some modules may be implemented by the same physical entity, or some modules may be implemented by multiple physical entities, or some components in multiple independent devices may be implemented together.
[0048] The above-described embodiments are only preferred embodiments for fully illustrating the present invention, and the protection scope of the present invention is not limited thereto. Equivalent substitutions or changes made by those skilled in the art based on the present invention are within the protection scope of the present invention. The protection scope of the present invention shall be subject to the claims.
Claims
1. A method for implementing adaptive storage time partitioning for a time series database, characterized in that include: Step 1: Deploy the master node and slave node of the time series database, and perform bidirectional data synchronization between the master node and the slave node. Step 2: Set the parameters of each storage time partition. Step 3: Build an algorithm model for adaptive storage time partitioning, obtain the parameters of each storage time partition through the algorithm model, use the time series data written to the master node and related data to train the algorithm model, and divide the data according to different storage time partition parameters to compare and obtain the optimal storage time partition. Step 4: Use the standby node to import the time series data of the primary node according to the optimal storage time partition, and switch the primary and standby nodes. Step 5: Regularly select the optimal storage time partition through the algorithm model, repeat the process of the standby node importing the time series data of the primary node according to the optimal storage time partition, and perform the master-standby node switching process.
2. According to claim 1, a method for implementing adaptive storage time partitioning for a time series database is characterized by: In step 3, select a storage time partition, write the time series data to the write master node, and pass the time series data and related business reading and other operation data to the algorithm model for algorithm model training.
3. The method for implementing adaptive storage time partitioning for a time series database according to claim 1 or 2, characterized in that In step 3, a deep convolutional neural network is used to build an algorithm model for adaptive storage time partitioning, and the deep convolutional neural network of the algorithm model is trained using the time series data and related data written to the master node.
4. According to claim 1, a method for implementing adaptive storage time partitioning for a time series database is characterized by: It also includes step 6: when querying time series data, the data of the standby node is queried first. If the time series data span in the standby node does not meet the query time span, the master node is queried for the time series data within the time span that the standby node lacks, the data obtained by the standby node query and the data obtained by the master node query are merged, and the result is returned.
5. An adaptive storage time partitioning implementation device for a time series database, characterized in that Including deployment module, parameter management module, model management module and storage management module, The deployment module deploys the master node and the standby node of the time series database, and performs bidirectional data synchronization writing between the master node and the standby node. The parameter management module sets the parameters of each storage time partition. The model management module builds an algorithm model for adaptive storage time partitioning, obtains the parameters of each storage time partition through the algorithm model, and uses the time series data and related data written to the master node to train the algorithm model. The algorithm model divides the data according to different storage time partition parameters, and compares and obtains the optimal storage time partition. The storage management module imports the time series data of the master node through the standby node according to the optimal storage time partition, and switches the master and standby nodes. The model management module regularly selects the optimal storage time partition through the algorithm model, and the storage management module repeats the process of the standby node importing the time series data of the primary node according to the optimal storage time partition to switch the primary and standby nodes.
6. The device for implementing adaptive storage time partitioning for a time series database according to claim 5, characterized in that The model management module selects a storage time partition, writes time series data to the write master node, and passes the time series data and related business reading and other operation data to the algorithm model for algorithm model training.
7. The device for implementing adaptive storage time partitioning for a time series database according to claim 5 or 6, characterized in that The model management module uses a deep convolutional neural network to build an algorithm model for adaptive storage time partitioning, and uses the time series data and related data written to the master node to train the deep convolutional neural network of the algorithm model.
8. The device for implementing adaptive storage time partitioning for a time series database according to claim 5, characterized in that It also includes a query module. When the query module performs a time series data query, it gives priority to querying the data of the standby node. If the time series data span in the standby node does not meet the query time span, the main node is queried for the time series data within the time span that the standby node lacks, the data obtained by the standby node query and the data obtained by the main node query are merged, and the result is returned.
