Adaptive Time Series Data Management System and Method for Time Series Databases
With the adaptive time-series data management system and methodology, users can customize partition parameters and select migration types, which solves the limitations of time-series databases in data analysis, backup and migration, and improves operational efficiency and system adaptability.
Patent Information
- Application Number
- CN202211012938.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-08-23
- Publication Date
- 2025-10-31
- Estimated Expiration
- 2042-08-23
AI Technical Summary
Existing time-series databases have limitations in data analysis, backup, and migration, failing to meet diverse user needs, and the backup and migration processes need improvement.
An adaptive time-series data management system and method are provided. Through user-configurable parsing modules, storage and computing modules, data upload modules, and data migration modules, the system supports users to customize partitioning parameters and select migration types, thereby optimizing the data partitioning, backup, and migration processes.
It improves user operational efficiency, adapts to the needs of different industries, optimizes data backup and migration processes, and increases the system's flexibility and disaster recovery capabilities.
Smart Images

Figure CN115422152B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of time-series database technology, specifically to an adaptive time-series data management system and method for time-series databases. Background Technology
[0002] A time series database is a database that primarily processes data with time labels (data that changes in chronological order, i.e., time-series data). Data with time labels is also called time series data.
[0003] Time series data are mainly collected and generated by various types of real-time monitoring, inspection and analysis equipment in industries such as power, chemical, meteorology and geographic information. The typical characteristics of this industrial data are: high generation frequency (multiple data can be generated per second at each monitoring point), heavy dependence on acquisition time (each data requires a unique time), and large amount of information from many measurement points.
[0004] In terms of time-series data management, traditional time-series databases all adopt the form of partitioning by time. The efficiency of querying by time is greatly improved compared with other relational databases, which is also one of the characteristics of time-series databases. In terms of data backup and migration, time-series databases are similar to other databases.
[0005] However, with the development of various industries, in some scenarios, partitioning by time alone can no longer meet the needs of users. The partitioning method needs to be optimized, and the data backup and migration also needs to be improved accordingly.
[0006] The technical problems that need to be solved are how to improve the limitations of time-series databases in analyzing time-series data and optimize data backup and migration processes. Summary of the Invention
[0007] The technical objective of this invention is to address the above-mentioned shortcomings by providing an adaptive time-series data management system and method for time-series databases, thereby solving the technical problems of how to improve the limitations of time-series databases in analyzing time-series data and optimize data backup and data migration processes.
[0008] In a first aspect, the present invention provides an adaptive time-series data management system for a time-series database, used for managing data partitioning, data backup, and data migration of time-series data, the system comprising:
[0009] The user configuration parsing module interacts with the user through the user configuration interface. It supports users in setting partition parameters, parsing partition parameters into partition objects, and constructing a system table. It is used to configure partition rules by combining timestamps and partition objects in the system table, and to support users in selecting data migration types.
[0010] The storage and computing module interacts with the user through a storage and computing interface. It supports the user in creating partition objects. For each partition object, it selects a primary node and a backup node for the partition object based on the usage of the database cluster nodes and allocates storage space. It also records the partition object and its corresponding object data into a system table.
[0011] The data upload module interacts with the user through a data upload interface to support user-input data. The data specifies a partition object and is used to store the data to the corresponding database node based on the partition object obtained from parsing the data and the partitioning rules.
[0012] The data migration module interacts with the user through a data migration interface. It is used to detect the status of each node. For nodes whose data storage exceeds a threshold, node migration is triggered. For nodes that trigger node migration, the module calculates the partition objects to be migrated and the target node after migration based on the selected migration type. The module then migrates the partition objects and their corresponding object data to the target node and updates the system tables.
[0013] Preferably, the user configuration parsing module is used to check the partition parameters set by the user, including checking whether the parameter format is correct. After passing the check, it is used to parse the partition parameters into partition objects and build a system table.
[0014] Preferably, the migration type includes:
[0015] Migrate master node data. For each database node, the data to be migrated is the partition object and the corresponding object data of the database node as the master node.
[0016] Migrate backup node data. For each database node, the data to be migrated is the partition object and the corresponding object data of the database node as the backup node.
[0017] For dynamic migration, the migration backup node data for each database consists of: partition objects and corresponding object data with the database node as the master node, and partition objects and corresponding object data with the database node as the backup node. The partition objects and corresponding object data with smaller data volumes are the data to be migrated.
[0018] Preferably, the data upload module is used to support user-configured data format, wherein the data format is used to limit the data uploaded by the user to include the data itself and the partition object;
[0019] The data upload module performs format verification on the user-uploaded data based on the data format. For data that conforms to the data format, it parses the data to obtain the corresponding partition object, and stores the data itself to the corresponding database node according to the partition object and the partitioning rules.
[0020] Preferably, the data migration module is used to call a scheduled task to periodically check the node status of each node. If the data storage volume of a node exceeds the configured threshold, the node migration is triggered.
[0021] In a second aspect, the present invention provides an adaptive time-series data management method for a time-series database, which performs data uploading, data backup, and data migration of time-series data through an adaptive time-series data management system for a time-series database as described in any one of the first aspects. The method includes the following steps:
[0022] Set partition parameters, parse the partition objects from the partition parameters, and build a system table. Configure partition rules by combining the timestamp and the partition objects in the system table as partition combinations; and select the data migration type.
[0023] Create partition objects. For each partition object, select a primary node and a backup node for the partition object according to the database cluster node usage, allocate storage space, and record the partition object and its corresponding object data in the system table.
[0024] The incoming data specifies a partition object. Based on the partition object obtained from parsing the data and the partitioning rules, the data is stored in the corresponding database node.
[0025] The status of each node is detected. For nodes whose data storage exceeds the threshold, node migration is triggered. For nodes that trigger migration, the partition objects to be migrated and the target node after migration are calculated based on the selected migration type. The partition objects and their corresponding object data are migrated to the target node, and the system tables are updated.
[0026] As a preferred method, the partition parameters are checked, including whether the parameter format is correct. If the check is successful, the partition parameters are parsed into partition objects and a system table is constructed.
[0027] Preferably, the migration type includes:
[0028] Migrate master node data. For each database node, the data to be migrated is the partition object and the corresponding object data of the database node as the master node.
[0029] Migrate backup node data. For each database node, the data to be migrated is the partition object and the corresponding object data of the database node as the backup node.
[0030] For dynamic migration, the migration backup node data for each database consists of: partition objects and corresponding object data with the database node as the master node, and partition objects and corresponding object data with the database node as the backup node. The partition objects and corresponding object data with smaller data volumes are the data to be migrated.
[0031] Preferably, before transmitting data, the data format is configured, which is used to limit the data uploaded by the user to include the data itself and the partition object;
[0032] After the data is input, the uploaded data is validated based on the data format. For data that conforms to the data format, the data is parsed to obtain the corresponding partition object. Based on the partition object and the partitioning rules, the data itself is stored in the corresponding database node.
[0033] Preferably, the status of each node is detected by the following steps: a scheduled task is invoked to periodically check the status of each node. If the data storage of a node exceeds the configured threshold, node migration of the node is triggered.
[0034] The adaptive time-series data management system and method for time-series databases of the present invention have the following advantages:
[0035] 1. It can partition according to the partition parameters set by the user, improve the user's efficiency and facilitate the user's operation. At the same time, backup and migration operations also provide corresponding convenience for users, making it more suitable for users in different industries.
[0036] 2. Due to phased statistics, the applicability of the product can be continuously increased and the user experience improved through subsequent optimization and improvement of the calculation algorithm. Attached Figure Description
[0037] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0038] The invention will be further described below with reference to the accompanying drawings.
[0039] Figure 1 The flowchart shows the user configuration parsing module in the adaptive time-series data management system for time-series databases in Example 1.
[0040] Figure 2 This is a flowchart illustrating the workflow of the storage and computation modules in the adaptive time-series data management system for the time-series database in Example 1.
[0041] Figure 3 This is a flowchart illustrating the workflow of the data upload module in the adaptive time-series data management system for time-series databases in Example 1.
[0042] Figure 4 This is a flowchart illustrating the workflow of the data migration module in the adaptive time-series data management system for time-series databases in Example 1.
[0043] Figure 5 This is a flowchart of the adaptive time-series data management method for time-series databases in Example 1. Detailed Implementation
[0044] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, so that those skilled in the art can better understand and implement the present invention. However, the embodiments are not intended to limit the present invention. In the absence of conflict, the embodiments of the present invention and the technical features in the embodiments can be combined with each other.
[0045] This invention provides an adaptive time-series data management system and method for time-series databases, which addresses the technical problems of how to improve the limitations of time-series databases in analyzing time-series data, and optimize data backup and data migration processes.
[0046] In this embodiment, time series data refers to time sequence data, which is a data column recorded in chronological order according to the same unified indicator.
[0047] A time-series database is a data warehouse specifically designed to store time-series data.
[0048] The master node is the database node for uploading and storing time-series data, and it is the node that is mainly used in normal times.
[0049] Backup node: It is a backup of the time-series data of the master node. It is not used normally, but is used for disaster recovery when the master node fails. The data source is replicated from the master node.
[0050] Example 1:
[0051] The present invention discloses an adaptive time-series data management system for time-series databases, comprising a user configuration parsing module, a storage and computing module, a data upload module, and a data migration module, for managing data partitioning, data backup, and data migration of time-series data.
[0052] The user configuration parsing module interacts with users through the user configuration interface. It supports users in setting partition parameters, parsing partition parameters into partition objects, and constructing system tables. It is used to configure partition rules by combining timestamps and partition objects in the system tables, and to support users in selecting data migration types.
[0053] As an improvement, in this embodiment, the user configuration parsing module is also used to check the partition parameters set by the user, including checking whether the parameter format is correct. After passing the check, it is used to parse the partition parameters into partition objects and build a system table.
[0054] The workflow of the user configuration parsing module is as follows: Figure 1 As shown, after the user enters the partition parameters through the user configuration interface, it is determined whether the partition parameters are correct. If they are correct, the partition parameters are parsed to obtain the partition object and a system table is constructed based on the partition object. The partition rules are configured by combining the timestamp and the partition object corresponding to the system table. The partition rules are used to implement the storage of subsequent data uploads.
[0055] Meanwhile, users can set the migration type through the user configuration interface. The system supports three migration types: migrating primary node data, migrating standby node data, and dynamic migration. For each database node, migrating primary node data involves migrating the partition objects and corresponding object data of the database node as the primary node; migrating standby node data involves migrating the partition objects and corresponding object data of the database node as the standby node; migrating standby node data involves migrating the partition objects and corresponding object data of the database node as the primary node, and the partition objects and corresponding object data of the database node as the standby node, with the smaller partition objects and corresponding object data being the data to be migrated.
[0056] When the user does not set an adaptive partition object, the data migration will be performed using the normal random migration method.
[0057] As a specific example, if a user selects a region as the partitioning parameter, the user-configured parsing module will parse the partitioning parameter "region" and construct a system table based on the partitioning object. This system table is initially empty; the specific field content is added and configured by the subsequent storage and calculation modules. Taking electricity consumption data in the power industry as an example, if the region is limited to North China, the corresponding partitioning objects could be Beijing, Tianjin, Hebei, Shanxi, and Inner Mongolia. In the system table, each partitioning object is recorded as a field.
[0058] The storage and computing module interacts with the user through the storage and computing interface, which supports the user in creating partition objects. For each partition object, it selects the primary and backup nodes for the partition object according to the usage of the database cluster nodes and allocates storage space. It also records the partition object and its corresponding object data into the system table.
[0059] In this embodiment, the storage and computing module is mainly used to create partition objects and compute primary and backup nodes. When a user creates a new partition object, the storage, computing, and allocation module creates the partition for it, selects suitable primary and backup nodes for the partition object based on the cluster node usage, allocates storage space for it, and records the data in the system table.
[0060] The workflow of the storage and computing module is as follows: Figure 2 As shown, after a user creates a new partition object, the system selects a primary node and a backup node for the partition object based on the database cluster node usage, allocates storage space, and records the partition object and its corresponding data into the system table based on the table structure of the previous system table.
[0061] As a specific example, based on the North China region, users sequentially create partition objects for Beijing, Tianjin, Hebei, Shanxi, and Inner Mongolia Autonomous Region, and record each partition object as a field in the system table, such as the name and ID of each region.
[0062] The data upload module interacts with the user through a data upload interface to support user-input data. The data specifies a partition object, which is used to store the data to the corresponding database node based on the partition object obtained from parsing the data and the partitioning rules.
[0063] In a specific implementation, the data upload module is used to support user-configured data formats, which are used to limit the data uploaded by the user to include the data itself and partition objects; and to perform format verification on the data uploaded by the user based on the data format. For data that conforms to the data format, the module is used to parse the data to obtain the corresponding partition object, and to store the data itself to the corresponding database node according to the partition object and partitioning rules.
[0064] The workflow of the data upload module in this embodiment is as follows: Figure 3 As shown, user-uploaded data must be uploaded according to the defined structure. When data is uploaded to the system, the data upload module determines whether the data format is correct. If correct, the data is parsed to obtain partition objects, and the data is stored in the corresponding nodes according to the partition objects. Specifically, the data format is defined as follows:
[0065] {
[0066] objectId:1
[0067] timestamp: 2022-06-27 00:00:00
[0068] {
[0069] {
[0070] key:022-06-27 00:00:00
[0071] value:test1
[0072] }
[0073] {
[0074] key:022-06-27 00:00:00
[0075] value:test2
[0076] }
[0077] }
[0078] }
[0079] The data migration module interacts with users through the data migration interface to detect the status of each node. For nodes whose data storage exceeds the threshold, node migration is triggered. For nodes that trigger migration, the module calculates the partition objects to be migrated and the target node after migration based on the selected migration type. The partition objects and their corresponding object data are then migrated to the target node, and the system tables are updated.
[0080] In its specific implementation, the data migration module includes a calculation module and a data assistance module. This module calls a scheduled task to periodically check the node status of each node. If the data storage volume of a node exceeds a configured threshold, it triggers node migration. For nodes requiring migration, the calculation module selects the object data to be migrated to the new node based on the chosen data migration type and sends this data to the data replication module. The data replication module executes the data migration, migrating the object data to the new node. After migration, it synchronizes the partitioned objects and their data changes to the system tables.
[0081] The workflow of the data migration module in this embodiment is as follows: Figure 4As shown, the cluster has a background process that monitors the status of each node at regular intervals. When the data storage of a certain node reaches the pressure value, a data migration operation will be triggered. Based on the selected data migration type, the object data to be migrated will be selected and migrated to the new node. The changed data will be sent to the target node. After the migration is completed, the partition object and the data changes of the object data will be synchronized to the system table.
[0082] The workflow of the system in this embodiment is as follows:
[0083] Operation 1: User enters partition parameter configuration;
[0084] Operation 2: The user configuration parsing module accepts the partition parameters entered by the user;
[0085] Operation 3: Parse the partition parameters into objects and create a system table;
[0086] Operation 4: When creating a partition object, the storage and compute modules create partitions for it based on the parsed parameters and select primary and backup nodes;
[0087] Operation 5: During data upload, the data upload module uploads the data to the corresponding node based on the parsed partition object;
[0088] Operation 6: The cluster backend has a scheduled task that checks the node status periodically;
[0089] Operation 7: When a node's storage reaches the pressure value, the computing module will calculate the nodes and partition objects that need to be migrated and send them to the data replication module;
[0090] Operation 8: Data migration is performed by the data replication module, and data changes are made after the migration is complete.
[0091] This system allows users to partition the system according to their own choices, improving user efficiency and making operation more convenient. At the same time, backup and migration operations also provide users with corresponding convenience, making it more suitable for users in different industries.
[0092] Example 2:
[0093] This invention discloses an adaptive time-series data management method for time-series databases. Using the adaptive time-series data management system for time-series databases disclosed in Example 1, the method performs data uploading, data backup, and data migration of time-series data. The method includes the following steps:
[0094] S100. Set partition parameters, parse the partition objects from the partition parameters, and build a system table. Configure partition rules by combining the timestamp and the partition objects in the system table as partitions; and select the data migration type.
[0095] S200. Create partition objects. For each partition object, select a primary node and a backup node for the partition object according to the usage of the database cluster nodes, allocate storage space, and record the partition object and the corresponding object data in the system table.
[0096] S300, Input data, the data specifies a partition object, and the data is stored in the corresponding database node based on the partition object obtained from parsing the data and the partitioning rules;
[0097] S400: Detect the status of each node. For nodes whose data storage exceeds the threshold, trigger node migration. For nodes that trigger node migration, calculate the partition objects to be migrated and the target node after migration based on the selected migration type. Migrate the partition objects and their corresponding object data to the target node and update the system tables.
[0098] In step S100 of this embodiment, after the user sets the partition parameters, the partition parameters are checked to determine whether the parameter format is correct. If the check is qualified, the partition parameters are parsed into partition objects and a system table is constructed.
[0099] As a specific implementation, step S100, based on the system disclosed in the embodiment, performs the following operations: After the user enters the partition parameters through the user configuration interface, it is determined whether the partition parameters are correct. If correct, the partition parameters are parsed to obtain the partition object and a system table is constructed based on the partition object. The partition rules are configured by combining the timestamp and the partition object corresponding to the system table. The partition rules are used to realize the storage of subsequent data uploads.
[0100] Meanwhile, users can set the migration type through the user configuration interface. In this implementation, three migration types are configured: migrating primary node data, migrating standby node data, and dynamic migration. For each database node, migrating primary node data involves migrating the partition objects and corresponding object data of the database node as the primary node; migrating standby node data involves migrating the partition objects and corresponding object data of the database node as the standby node; migrating standby node data involves migrating the partition objects and corresponding object data of the database node as the primary node, and the partition objects and corresponding object data of the database node as the standby node, with the smaller partition objects and corresponding object data being the data to be migrated.
[0101] When the user does not set an adaptive partition object, the data migration will be performed using the normal random migration method.
[0102] Step S200: Create partition objects and compute primary / standby nodes. When a user creates a new partition object, the partition is created through the storage, computation, and allocation module in the system of Example 1. Based on the cluster node usage, a suitable primary / standby node is selected for the partition object, storage space is allocated to it, and the data is recorded in the system table.
[0103] The specific workflow of step S200 is as follows: After the user creates a new partition object, the primary node and backup node are selected for the partition object according to the usage of the database cluster nodes, and storage space is allocated. Based on the table structure of the previous system table, the partition object and the corresponding object data are recorded in the system table.
[0104] Step S300 is implemented through the data upload module in the system of Example 1. The data uploaded by the user must meet a preset data format, which limits the uploaded data to include the data itself and a partition object. After the user uploads the data, the data is validated based on the data format. For data that meets the data format, the data is parsed to obtain the corresponding partition object, and the data itself is stored in the corresponding database node according to the partition object and partitioning rules.
[0105] As a specific implementation, the data format is defined as follows:
[0106] {
[0107] objectId:1
[0108] timestamp: 2022-06-27 00:00:00
[0109] {
[0110] {
[0111] key:022-06-27 00:00:00
[0112] value:test1
[0113] }
[0114] {
[0115] key:022-06-27 00:00:00
[0116] value:test2
[0117] }
[0118] }
[0119] }
[0120] Step S400 is implemented based on the data migration module in the system. The cluster has a background process that periodically monitors the status of each node. When the data storage of a node reaches a pressure threshold, a data migration operation is triggered. The execution flow in this embodiment is as follows: First, a scheduled task is invoked to periodically check the node status of each node. If the data storage volume of a node exceeds the configured threshold, node migration is triggered. For nodes requiring migration, the object data to be migrated is selected according to the chosen data migration type and migrated to the new node. After the migration is complete, the partition objects and the data changes of the object data are synchronized to the system tables.
[0121] The method in this embodiment changes the traditional rule that time-series databases generally use timestamps as the partitioning standard. Users can customize and select special fields as the partitioning standard, and partitioning is performed according to the user's selection. Each scenario is handled separately, which is more convenient for users. In terms of backup, each piece of data is stored on two nodes, primary and backup, which increases the disaster recovery capability of the system. If the data storage pressure on the nodes is too high, data migration is required. Users can choose to perform primary or backup migration, and the migration process also has a set of independent rules.
[0122] The present invention has been shown and described in detail above with reference to the accompanying drawings and preferred embodiments. However, the present invention is not limited to these disclosed embodiments. Based on the above embodiments, those skilled in the art will know that more embodiments of the present invention can be obtained by combining the code review methods in the different embodiments. These embodiments are also within the protection scope of the present invention.
Claims
1. An adaptive time-series data management system for time-series databases, characterized in that... The system is used to manage data partitioning, data backup, and data migration of time-series data, and includes: The user configuration parsing module interacts with the user through the user configuration interface. It supports users in setting partition parameters, parsing partition parameters into partition objects, and constructing a system table. It is used to configure partition rules by combining timestamps and partition objects in the system table, and to support users in selecting data migration types. The storage and computing module interacts with the user through a storage and computing interface. It supports the user in creating partition objects. For each partition object, it selects a primary node and a backup node for the partition object based on the usage of the database cluster nodes and allocates storage space. It also records the partition object and its corresponding object data into a system table. The data upload module interacts with the user through a data upload interface to support user-input data. The data specifies a partition object and is used to store the data to the corresponding database node based on the partition object obtained from parsing the data and the partitioning rules. The data migration module interacts with the user through a data migration interface. It is used to detect the status of each node. For nodes whose data storage exceeds a threshold, node migration is triggered. For nodes that trigger node migration, the module calculates the partition objects to be migrated and the target node after migration based on the selected migration type. The module then migrates the partition objects and their corresponding object data to the target node and updates the system tables.
2. The adaptive time-series data management system for time-series databases according to claim 1, characterized in that... The user configuration parsing module is used to check the partition parameters set by the user, including checking whether the parameter format is correct. After passing the check, it is used to parse the partition parameters into partition objects and build system tables.
3. The adaptive time-series data management system for time-series databases according to claim 1, characterized in that... The migration types include: Migrate master node data. For each database node, the data to be migrated is the partition object and the corresponding object data of the database node as the master node. Migrate backup node data. For each database node, the data to be migrated is the partition object and the corresponding object data of the database node as the backup node. For dynamic migration, the migration backup node data for each database consists of: partition objects and corresponding object data with the database node as the master node, and partition objects and corresponding object data with the database node as the backup node. The partition objects and corresponding object data with smaller data volumes are the data to be migrated.
4. The adaptive time-series data management system for time-series databases according to claim 1, characterized in that... The data upload module is used to support user-configured data formats, and the data formats are used to limit the data uploaded by the user to include the data itself and the partition object. The data upload module is used to perform format verification on the data uploaded by the user based on the data format. For data that conforms to the data format, it is used to parse the data to obtain the corresponding partition object, and store the data itself to the corresponding database node according to the partition object and the partition rules.
5. The adaptive time-series data management system for time-series databases according to claim 1, characterized in that... The data migration module is used to call a scheduled task to periodically check the node status of each node. If the data storage volume of a node exceeds the configured threshold, the node migration is triggered.
6. An adaptive time-series data management method for a time-series database, characterized in that... The adaptive time-series data management system for time-series databases as described in any one of claims 1-5 performs data uploading, data backup, and data migration of time-series data, the method comprising the following steps: Set partition parameters, parse the partition objects from the partition parameters, and build a system table. Configure partition rules by combining the timestamp and the partition objects in the system table as partitions; and select the data migration type. Create partition objects. For each partition object, select a primary node and a backup node for the partition object according to the database cluster node usage, allocate storage space, and record the partition object and its corresponding object data in the system table. The incoming data specifies a partition object. Based on the partition object obtained from parsing the data and the partitioning rules, the data is stored in the corresponding database node. The status of each node is detected. For nodes whose data storage exceeds the threshold, node migration is triggered. For nodes that trigger migration, the partition objects to be migrated and the target node after migration are calculated based on the selected migration type. The partition objects and their corresponding object data are migrated to the target node, and the system tables are updated.
7. The adaptive time-series data management method for time-series databases according to claim 6, characterized in that... The partition parameters are checked, including whether the parameter format is correct. If the check is successful, the partition parameters are parsed into partition objects and system tables are constructed.
8. The adaptive time-series data management method for time-series databases according to claim 6, characterized in that... The migration types include: Migrate master node data. For each database node, the data to be migrated is the partition object and the corresponding object data of the database node as the master node. Migrate backup node data. For each database node, the data to be migrated is the partition object and the corresponding object data of the database node as the backup node. For dynamic migration, the migration backup node data for each database consists of: partition objects and corresponding object data with the database node as the master node, and partition objects and corresponding object data with the database node as the backup node. The partition objects and corresponding object data with smaller data volumes are the data to be migrated.
9. The adaptive time-series data management method for time-series databases according to claim 6, characterized in that... Before transmitting data, configure the data format, which is used to limit the data uploaded by the user to include the data itself and the partition object; After the data is input, the uploaded data is validated based on the data format. For data that conforms to the data format, the data is parsed to obtain the corresponding partition object. Based on the partition object and the partitioning rules, the data itself is stored in the corresponding database node.
10. The adaptive time-series data management method for time-series databases according to claim 6, characterized in that... The status of each node is detected by the following steps: a scheduled task is invoked to periodically check the status of each node. If the data storage of a node exceeds the configured threshold, node migration is triggered.
Citation Information
Patent Citations
Industrial time series database-oriented adaptive database partitioning method and system
CN108446329A
Method and system for realizing materialized view performance optimization based on Hive partition table
CN111984680A