A time series database tag management method
By storing tag data as metadata in a time-series database and using a tag caching structure, the high complexity of tag data maintenance in existing technologies is solved, and more efficient insertion and query performance is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- 山东浪潮数据库技术有限公司
- Filing Date
- 2023-11-27
- Publication Date
- 2026-06-23
AI Technical Summary
Existing time-series databases require additional maintenance of tag data as physical tables, resulting in high computational complexity and low efficiency in insertion and querying.
Tag data is stored as metadata in a system table, and a tag cache is created using a locked map data structure. The tag cache structure accelerates the tag filtering process during compilation and avoids multi-table join calculations.
It reduces computational complexity, improves the insertion and query efficiency of time-series database systems, and reduces the cost of tag data maintenance.
Smart Images
Figure CN117668044B_ABST
Abstract
Description
Technical Field
[0001] This invention discloses a method relating to the field of time-series database technology, specifically a method for managing tags in a time-series database. Background Technology
[0002] A time-series database is a database system specifically designed for storing and processing time-series data, offering efficient data access, querying, and analysis capabilities. Time-series data refers to data sets that change over time, such as sensor data, log data, monitoring data, and transaction data. In a time-series database, a tag is metadata used to describe and classify time-series data. It can be a string, number, or other data type, used to identify and describe the attributes, categories, or keywords of the time-series data. Tags in a time-series database are used for classifying, archiving, and querying time-series data. By adding tags to time-series data, data can be grouped, filtered, and aggregated for easier data analysis and querying. However, currently, time-series databases require additional maintenance of tag data as physical tables, necessitating the maintenance of multiple tag tables and multi-table join calculations, which hinders the improvement of insertion and query efficiency. Summary of the Invention
[0003] This invention addresses the problems of existing technologies by providing a time-series database tag management method. It stores and manages tag data in the time-series database as metadata, accelerates the tag filtering process during compilation through a tag data caching structure, eliminates the need for additional tag data maintenance, reduces computational complexity, and improves the insertion and query efficiency of the time-series database system.
[0004] The specific solution proposed in this invention is as follows:
[0005] This invention provides a time-series database tag management method, which creates a tag data system table in the database system to store tag data. The tag data system table includes column fields, including the time-series table ID, the instance ID under the time-series table, and the tag value after serialization of the instance's tag data.
[0006] A tag cache structure is created using a locked map data structure. The key of the tag cache is the time series table ID, and the value of the tag cache is a data structure containing all instance IDs under a certain time series table and all the serialized tag values of the tag data of the instances under that time series table.
[0007] The system loads tag data from the tag data system table and generates a corresponding tag cache. The generated tag cache is then used to manage the tag data.
[0008] Furthermore, the method for managing time-series database tags, which involves loading tag data according to the tag data system table and generating a corresponding tag cache, includes: obtaining the instance ID under the time-series table; using the time-series table ID to obtain the corresponding tag cache; traversing all tag values in order; determining whether all tag values exist and belong to the same instance; if so, obtaining the instance ID under the time-series table; otherwise, writing a new set of tag data to the tag data system table.
[0009] Furthermore, the management of tag data in the aforementioned time-series database tag management method includes: adding or deleting tag data.
[0010] When adding or deleting a set of tag data, the corresponding column field position in the tag data system table is directly modified to perform the addition or deletion operation, and the corresponding tag cache data is invalidated.
[0011] Furthermore, the management of tag data in the aforementioned time-series database tag management method includes: inserting or querying tag data, wherein the tag data to be inserted or queried is determined by the time-series table ID, the values of all or some data columns in the time-series table, and the values of all or some tag columns in the time-series table. The tag cache is used to find all tag values under a certain time-series table through an operation with a time complexity of O(1), and then an operation with a time complexity of O(n) is used to find the tag values and locate the time-series data block to be inserted or queried.
[0012] This invention provides a time-series database tag management device, including a system table management module, a tag cache module, and a tag management module.
[0013] The system table management module creates a tag data system table in the database system to store tag data. This tag data system table contains column fields, including the time series table ID, the instance ID under the time series table, and the serialized tag value of the instance's tag data.
[0014] The tag caching module uses a locked map data structure to create the tag cache structure. The key of the tag cache is the time series table ID, and the value of the tag cache is a data structure. The data structure contains all instance IDs under a certain time series table and all the serialized tag values of the tag data of the instances under that time series table.
[0015] The tag management module loads tag data from the tag data system table and generates corresponding tag caches, and manages the tag data based on the generated tag caches.
[0016] Furthermore, in the aforementioned time-series database tag management device, the tag management module loads tag data according to the tag data system table and generates corresponding tag caches, including: obtaining the instance ID under the time-series table, using the time-series table ID to obtain the corresponding tag cache, traversing all tag values in order, determining whether all tag values exist and belong to the same instance, if so, obtaining a certain instance ID under the time-series table, otherwise writing a new set of tag data to the tag data system table.
[0017] Furthermore, in the aforementioned time-series database tag management device, the tag management module manages tag data, including: adding or deleting tag data.
[0018] When adding or deleting a set of tag data, the corresponding column field position in the tag data system table is directly modified to perform the addition or deletion operation, and the corresponding tag cache data is invalidated.
[0019] Furthermore, the tag management module in the aforementioned time-series database tag management device manages tag data, including: inserting or querying tag data, wherein the tag data to be inserted or queried is determined by the time-series table ID, the values of all or some data columns in the time-series table, and the values of all or some tag columns in the time-series table. The tag cache is used to find all tag values under a certain time-series table through an operation with a time complexity of O(1), and then an operation with a time complexity of O(n) is used to find the tag values and locate the time-series data block to be inserted or queried.
[0020] The advantages of this invention are:
[0021] This invention provides a tag management method for time-series databases. It stores and manages tag data in the time-series database as metadata, and accelerates the tag filtering process during compilation through a tag data caching structure. Compared to current time-series databases that store tag data and time-series data in separate tables, this invention eliminates the need for additional maintenance of tag data as a physical table. Furthermore, all tag data is stored in a single system table, avoiding the maintenance costs of multiple tag tables found in other solutions. Tag data filtering can be implemented at the database compilation layer. Compared to multi-table join calculations in other solutions, this invention reduces computational complexity and improves the insertion and query efficiency of the time-series database system. Attached Figure Description
[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0023] Figure 1 This is a schematic diagram of the process of loading tag data using the method of the present invention. Detailed Implementation
[0024] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, so that those skilled in the art can better understand and implement the present invention. However, the embodiments described are not intended to limit the present invention.
[0025] This invention provides a time-series database tag management method, which creates a tag data system table in the database system to store tag data. The tag data system table includes column fields, including the time-series table ID, the instance ID under the time-series table, and the tag value after serialization of the instance's tag data.
[0026] A tag cache structure is created using a locked map data structure. The key of the tag cache is the time series table ID, and the value of the tag cache is a data structure containing all instance IDs under a certain time series table and all the serialized tag values of the tag data of the instances under that time series table.
[0027] The system loads tag data from the tag data system table and generates a corresponding tag cache. The generated tag cache is then used to manage the tag data.
[0028] The method of this invention is applied to tag management in time series databases, playing a crucial role in the effective organization, classification, and querying of time series data.
[0029] Data organization and classification: Using tags, time series data can be organized and classified according to different attributes, categories, or keywords. This allows for better understanding and management of the data, facilitating subsequent queries, analysis, and applications.
[0030] Data filtering and selection: Tags can be used to quickly filter and select time-series data that meets specific conditions. Tag-based queries allow for quick location and retrieval of data of interest, improving data access efficiency.
[0031] Data aggregation and summarization: Labels can be used to aggregate time series data based on specific attributes. By grouping and performing statistical calculations according to label attributes, summarized data such as average, maximum, and minimum values can be obtained for higher-level data analysis and decision-making.
[0032] Query optimization and performance improvement: Time-series databases can utilize tags to create indexes, accelerating queries based on specific tags. Tag indexes allow for quick location and access to data that meets specific criteria, improving query performance and response speed.
[0033] Tag data can help organize, classify, and query large amounts of time-series data, improving data access efficiency, query performance, and data analysis flexibility. Properly planning and utilizing tag management functions can fully leverage the advantages of time-series databases, supporting data processing and decision-making needs across various application scenarios.
[0034] In specific applications, in some embodiments of the method of the present invention, time-series database tag management is performed, as follows:
[0035] Step 1: Taking the multi-model database KaiwuDB as an example, create a tag_value system table in the database system to store tag data. This table contains columns, including table_id (time series table ID), entity_id (instance ID under the time series table), and the serialized tag value of the tag_value instance. See Table 1 for reference.
[0036] Table 1
[0037]
[0038] Step 2: Create a tag cache structure using a locked map data structure. The key of the tag cache is the time series table ID, and the value of the tag cache is a data structure. The data structure contains all instance IDs under a given time series table and the serialized tag values of all instances under that time series table. The data structure can be referenced as follows:
[0039]
[0040] Step 3: Load tag data from the tag data system table and generate a corresponding tag cache. Manage tag data based on the generated tag cache. Specifically, when the database system starts, it automatically queries the `tag_value` tag data system table, loads the tag data from the system table, and generates the cache structure shown in Step 2. (Refer to...) Figure 1 When loading tag data, obtain the instance ID under the time series table, use the time series table ID to obtain the corresponding tag cache, traverse all tag values in order, and determine whether all tag values exist and belong to the same instance. If so, obtain the instance ID under the time series table; otherwise, write a new set of tag data to the tag data system table.
[0041] Furthermore, it involves managing tag data, including adding or deleting tag data.
[0042] When adding or deleting a set of tag data, the corresponding column field position in the tag data system table is directly modified to perform the addition or deletion operation, and the corresponding tag cache data is invalidated.
[0043] Furthermore, the management of tag data also includes: inserting or querying tag data, wherein the tag data to be inserted or queried is determined by the time series table ID, the values of all or some data columns in the time series table, and the values of all or some tag columns in the time series table. The tag cache is used to find all tag values under a certain time series table through an operation with a time complexity of O(1), and then an operation with a time complexity of O(n) is used to find the tag values and locate the time series data block to be inserted or queried.
[0044] This invention provides a time-series database tag management device, including a system table management module, a tag cache module, and a tag management module.
[0045] The system table management module creates a tag data system table in the database system to store tag data. This tag data system table contains column fields, including the time series table ID, the instance ID under the time series table, and the serialized tag value of the instance's tag data.
[0046] The tag caching module uses a locked map data structure to create the tag cache structure. The key of the tag cache is the time series table ID, and the value of the tag cache is a data structure. The data structure contains all instance IDs under a certain time series table and all the serialized tag values of the tag data of the instances under that time series table.
[0047] The tag management module loads tag data from the tag data system table and generates corresponding tag caches, and manages the tag data based on the generated tag caches.
[0048] The information interaction and execution process between the modules in the above-mentioned device are based on the same concept as the method embodiment of the present invention, and the specific details can be found in the description in the method embodiment of the present invention, and will not be repeated here.
[0049] Similarly, the device of this invention stores and manages tag data in the time-series database as metadata, and accelerates the tag filtering process during compilation through a tag data caching structure. Compared to current time-series databases that store tag data and time-series data in separate tables, the method of this invention does not require additional maintenance of tag data as a physical table, and all tag data is stored in the same system table, avoiding the maintenance costs of multiple tag tables in other solutions. Furthermore, tag data filtering can be implemented at the database compilation layer. Compared to the multi-table join calculations in other solutions, the method of this invention reduces computational complexity to a certain extent and improves the insertion and query efficiency of the time-series database system.
[0050] It should be noted that not all steps and modules in the above processes and device structures are mandatory; some steps or modules can be omitted as needed. The execution order of each step is not fixed and can be adjusted as required. The system structure described in the above embodiments can be a physical structure or a logical structure. That is, some modules may be implemented by the same physical entity, or some modules may be implemented by multiple physical entities, or they may be jointly implemented by certain components in multiple independent devices.
[0051] The above-described embodiments are merely preferred embodiments provided to fully illustrate the present invention, and the scope of protection of the present invention is not limited thereto. Equivalent substitutions or modifications made by those skilled in the art based on the present invention are all within the scope of protection of the present invention. The scope of protection of the present invention is defined by the claims.
Claims
1. A method for managing tags in a time-series database, characterized in that... A tag data system table is created in the database system to store tag data. The tag data system table contains column fields, including the time series table ID, the instance ID under the time series table, and the serialized tag value of the instance's tag data. A tag cache structure is created using a locked map data structure. The key of the tag cache is the time series table ID, and the value of the tag cache is a data structure containing all instance IDs under a certain time series table and all the serialized tag values of the tag data of the instances under that time series table. The system loads tag data from the tag data system table and generates a corresponding tag cache. The generated tag cache is then used to manage the tag data.
2. The time-series database tag management method according to claim 1, characterized in that: The step of loading tag data according to the tag data system table and generating the corresponding tag cache includes: Get the instance ID under the time series table, use the time series table ID to get the corresponding tag cache, traverse all tag values in order, and determine whether all tag values exist and belong to the same instance. If they do, get the instance ID under the time series table; otherwise, write a new set of tag data to the tag data system table.
3. The time-series database tag management method according to claim 1, characterized in that: The management of tag data includes: adding or deleting tag data. When adding or deleting a set of tag data, the corresponding column field position in the tag data system table is directly modified to perform the addition or deletion operation, and the corresponding tag cache data is invalidated.
4. The time-series database tag management method according to claim 1, characterized in that: The management of tag data includes: inserting or querying tag data, wherein the tag data to be inserted or queried is determined by the time series table ID, the values of all or some data columns in the time series table, and the values of all or some tag columns in the time series table. The tag cache is used to find all tag values under a certain time series table through an operation with a time complexity of O(1), and then an operation with a time complexity of O(n) is used to find the tag values and locate the time series data block to be inserted or queried.
5. A time-series database tag management device, characterized in that: It includes a system table management module, a tag caching module, and a tag management module. The system table management module creates a tag data system table in the database system to store tag data. This tag data system table contains column fields, including the time series table ID, the instance ID under the time series table, and the serialized tag value of the instance's tag data. The tag caching module uses a locked map data structure to create the tag cache structure. The key of the tag cache is the time series table ID, and the value of the tag cache is a data structure. The data structure contains all instance IDs under a certain time series table and all the serialized tag values of the tag data of the instances under that time series table. The tag management module loads tag data from the tag data system table and generates corresponding tag caches, and manages the tag data based on the generated tag caches.
6. A time-series database tag management device according to claim 5, characterized in that: The tag management module loads tag data according to the tag data system table and generates corresponding tag caches, including: obtaining the instance ID under the time series table, using the time series table ID to obtain the corresponding tag cache, traversing all tag values in order, determining whether all tag values exist and belong to the same instance, if so, obtaining a certain instance ID under the time series table, otherwise writing a new set of tag data to the tag data system table.
7. A time-series database tag management device according to claim 5, characterized in that: The tag management module manages tag data, including adding or deleting tag data. When adding or deleting a set of tag data, the corresponding column field position in the tag data system table is directly modified to perform the addition or deletion operation, and the corresponding tag cache data is invalidated.
8. A time-series database tag management device according to claim 5, characterized in that: The tag management module manages tag data, including inserting or querying tag data. Specifically, it uses the tag cache to find all tag values under a certain time series table through an operation with a time complexity of O(1), and then uses an operation with a time complexity of O(n) to find the tag values and locate the time series data block that needs to be inserted or queried.