Method and system for adding time series query in kv-type database

Abstract

This invention discloses a method and system for adding time-series queries to a key-value (KV) database, belonging to the field of database query technology. The technical problem it addresses is how to query time-series data based on a KV database. The method includes the following steps: adding a time-series query differentiation syntax, which distinguishes between time-series queries and KV queries; adding time-series query structures corresponding to database, schema, table, and column, and adding methods for parsing these structures; adding a parsing function to parse time-series query collection points, where each collection point is a time-series query structure corresponding to a column; adding a time-series query parsing function to parse time-series query conditions; and parsing the semantically parsed time-series query logical plan into a physical plan and executing the physical plan.

Description

Technical Field

[0001] This invention relates to the field of database query technology, specifically to a method and system for adding time-series queries to a key-value database. Background Technology

[0002] In the era of the Internet of Things (IoT), the amount of data generated by the Industrial Internet of Things (IIoT) is thousands or even tens of thousands of times greater than that of traditional information systems. This data is collected in real-time, at high frequency and density, and its dynamic data models are constantly changing. This data is time-series data. Time-series data (TSD) is a series of data indexed by the time dimension. Simply put, this type of data describes the measured values ​​of a measured entity at each point in time within a time range. It is ubiquitous in IT infrastructure, operation and maintenance monitoring systems, and the IoT. For example, collecting performance data for a server, such as CPU, memory, and I / O at a certain point in time, constitutes a series of time-series data. Queries on this data require time-series queries.

[0003] Compared to key-value (KV) data, time-series data refers to the storage and indexing of data point sequences based on time. Typically, time-series data is generated based on stable frequencies; for example, wind data collected at fixed time intervals when monitoring weather conditions constitutes a time series. Time-series data has the following basic properties:

[0004] Metrics: Data indicators collected. For example, wind speed in a city, similar to a table in a relational database.

[0005] Tag: A dimension column indicating the data's origin. Examples include city names and region names. These generally do not change over time and are used for querying.

[0006] field: The indicator column, representing the measured value of the data. For example, wind force and wind speed typically store values ​​that change over time. An indicator can have multiple fields.

[0007] timestamp: The point in time when the data measurement value was generated;

[0008] A data point is a value of a specific indicator collected at a specific time interval for a particular indicator of the object being tested. It is similar to a row in a relational table.

[0009] Key-value databases typically cannot perform real-time and effective queries for these attributes, so time-series queries need to be added to meet the current requirements.

[0010] How to query time-series data based on a key-value database is a technical problem that needs to be solved. Summary of the Invention

[0011] The technical objective of this invention is to address the above-mentioned shortcomings by providing a method and system for adding time-series queries to a key-value (kv) database, thereby solving the technical problem of how to query time-series data based on a kv database.

[0012] In a first aspect, the present invention provides a method for adding time-series queries to a key-value (kv) database, comprising the following steps:

[0013] Add a time-series query differentiation syntax, which is a query syntax that distinguishes between time-series queries and key-value queries;

[0014] Add time-series query structures corresponding to database, schema, table, and column respectively, and add time-series query structure parsing methods. Use the time-series query structure parsing methods to parse the time-series query structures corresponding to database, schema, and table to obtain metadata.

[0015] Add a parsing function to parse the time series query collection points, where the time series query collection points are the time series query structures corresponding to the columns;

[0016] Add a time-series query parsing function to parse time-series query conditions;

[0017] Based on the time-series query distinguishing syntax, adding a time-series query structure parsing method, parsing the time-series query collection point parsing function, and configuring the time-series query logical plan using the time-series query parsing function, the semantically parsed time-series query logical plan is parsed into a physical plan and then the physical plan is executed.

[0018] As a preferred option, add a time-series query differentiation syntax, including the following operations: use TIMESERIES SELECT as the identifier for time-series queries, and add TIMESERIES as the identifier for semantic parsing in the SELECTCLUSE structure after syntax parsing.

[0019] Preferably, the metadata is obtained through the following steps:

[0020] The time-series query structures corresponding to database, schema, and table are parsed using the time-series query structure parsing method to obtain the corresponding parsed information.

[0021] For the time-series query structure corresponding to the database, the parsed information includes the database name;

[0022] For the time-series query structure corresponding to scehma, the parsed information includes the name of scehma;

[0023] For the time-series query structure corresponding to the table, the parsed information includes the table's name;

[0024] Obtain the corresponding IDs for the database, scehma, and table names respectively, and construct a key based on the IDs of the database, scehma, and table. Use the constructed key to query metadata in the specified system table.

[0025] Preferably, the time-series query structure corresponding to the column is a datapoint, and the parsing function for parsing the time-series query collection points is used to parse the datapoint into a scopedatapoint, which is used to store the column.

[0026] Preferably, the time-series query parsing function is used to parse the query condition expression using the scopedatapoint.

[0027] In a second aspect, the present invention provides a system for adding time-series queries to a key-value (kV) database, used to add time-series queries to a kV database using a method for adding time-series queries to a kV database as described in any one of the first aspects, the system comprising:

[0028] The query syntax differentiation module is used to add time-series query differentiation syntax, which is a query syntax that distinguishes between time-series queries and key-value queries.

[0029] A structure parsing module is added. This module is used to add time-series query structures corresponding to database, schema, table, and column, and to add time-series query structure parsing methods. The time-series query structure parsing methods are used to parse the time-series query structures corresponding to database, schema, and table to obtain metadata.

[0030] The query parsing module is used to add parsing functions for parsing time-series query collection points, where the time-series query collection point is the time-series query structure corresponding to the column.

[0031] The query condition parsing module is used to add time-series query parsing functions to parse time-series query conditions;

[0032] The plan parsing and execution module is used to configure the time-series query logical plan based on the time-series query differentiation syntax, add a time-series query structure parsing method, parse the parsing function of the time-series query collection points, and the time-series query parsing function. It parses the semantically parsed time-series query logical plan into a physical plan and executes the physical plan.

[0033] Preferably, the query syntax differentiation module is used to add time-series query differentiation syntax by the following steps: using TIMESERIES SELECT as the identifier of time-series queries, and adding TIMESERIES as the identifier for semantic parsing in the SELECTCLUSE structure after syntax parsing.

[0034] Preferably, the plan parsing and execution module is used to obtain metadata through the following steps:

[0035] The time-series query structures corresponding to database, schema, and table are parsed using the time-series query structure parsing method to obtain the corresponding parsed information.

[0036] For the time-series query structure corresponding to the database, the parsed information includes the database name;

[0037] For the time-series query structure corresponding to scehma, the parsed information includes the name of scehma;

[0038] For the time-series query structure corresponding to the table, the parsed information includes the table's name;

[0039] Obtain the corresponding IDs for the database, scehma, and table names respectively, and construct a key based on the IDs of the database, scehma, and table. Use the constructed key to query metadata in the specified system table.

[0040] Preferably, the time-series query structure corresponding to the column is a datapoint, and the parsing function for parsing the time-series query collection points is used to parse the datapoint into a scopedatapoint, which is used to store the column.

[0041] Preferably, the time-series query parsing function is used to parse the query condition expression using the scopedatapoint.

[0042] The method and system for adding time-series queries to a key-value database according to the present invention have the following advantages:

[0043] 1. Adding time-series queries to the KV database allows for the reuse of some methods from the KV database without affecting the original functionality of the KV database, thus reducing development difficulty compared to developing a new time-series database.

[0044] 2. Although time-series and key-value queries are distinguished, key-value query conditions can also be used when querying time-series data, making it more feature-rich than a simple time-series database. Attached Figure Description

[0045] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0046] The invention will be further described below with reference to the accompanying drawings.

[0047] Figure 1 This is a flowchart illustrating a method for adding time-series queries to a key-value database. Detailed Implementation

[0048] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, so that those skilled in the art can better understand and implement the present invention. However, the embodiments are not intended to limit the present invention. In the absence of conflict, the embodiments of the present invention and the technical features in the embodiments can be combined with each other.

[0049] This invention provides a method and system for adding time-series queries to a key-value (kv) database, which solves the technical problem of how to query time-series data based on a kv database.

[0050] Example 1:

[0051] This invention discloses a method for adding time-series queries to a key-value (kv) database, comprising the following steps:

[0052] S100. Add a time-series query differentiation syntax, which is a query syntax that distinguishes between time-series queries and key-value queries;

[0053] S200. Add time-series query structures corresponding to database, schema, table and column respectively, and add time-series query structure parsing methods. Use the time-series query structure parsing methods to parse the time-series query structures corresponding to database, schema and table to obtain metadata.

[0054] S300. Add a parsing function to parse the time series query collection point, where the time series query collection point is the time series query structure corresponding to the column;

[0055] S400, Add a time-series query parsing function to parse time-series query conditions;

[0056] S500, based on time-series query differentiation syntax, adds time-series query structure parsing methods, parses the parsing function of time-series query collection points, and configures the time-series query logical plan using the time-series query parsing function, parses the semantically parsed time-series query logical plan into a physical plan, and executes the physical plan.

[0057] Step S100 adds a time-series query differentiation syntax to ensure that the original key-value query is not affected and that the two types of queries are distinguished during semantic parsing. Specifically, a time-series query differentiation syntax is added to distinguish between time-series and key-value queries. TIMESERIES SELECT is used as the identifier for time-series queries, and TIMESERIES is added to the SELECTCLUSE structure after syntax parsing as an identifier for semantic parsing.

[0058] In step S200, add the structures corresponding to database, schema, table, and column respectively, such as the datapoint structure corresponding to column; then, add a method to parse the time-series query structure to obtain metadata.

[0059] In a typical key-value query, the FROM clause is parsed to obtain the database, schema, and table IDs. Then, these IDs are used to construct a key to query metadata in the specified system table. However, since time-series data does not have database, schema, and table in the usual sense, but only the corresponding structures, it is necessary to first query the corresponding database, schema, and table name by the structure name before proceeding with the previous steps. Therefore, a method needs to be added during semantic parsing to parse the structure into a name and return the structure that stores this information.

[0060] In practice, the metadata is obtained through the following steps during subsequent execution:

[0061] (1) The time-series query structures corresponding to database, schema, and table are parsed using the time-series query structure parsing method to obtain the corresponding parsed information;

[0062] For the time-series query structure corresponding to the database, the parsed information includes the database name;

[0063] For the time-series query structure corresponding to scehma, the parsed information includes the name of scehma;

[0064] For the time-series query structure corresponding to the table, the parsed information includes the table's name;

[0065] (2) Obtain the corresponding IDs of database, scehma and table by name respectively, and construct a key based on the IDs of database, scehma and table. Query metadata in the system table based on the constructed key.

[0066] Step S300 adds a parsing function for the time-series query datapoint. When parsing the query list of a key-value query, the function parses the columns using the structure returned in step S300 and the query list structure, returning a `scopecolumn` structure. However, time-series queries do not have specific columns, only corresponding time-series query structures called `datapoint`. Therefore, a method needs to be added to parse `datapoint` into `scopedatapoint`, which is used to store the time-series query condition columns.

[0067] Step S400 adds a time-series query parsing function to parse query conditions. When parsing query conditions, the key-value query uses the scopecolumn structure from step 5 to parse the query condition expression. Therefore, a method for parsing the query condition expression using scopedatapoint needs to be added to the time-series query semantic parsing.

[0068] Step S500 parses the semantically parsed time-series query logical plan into a physical plan and executes it.

[0069] Example 2:

[0070] The present invention discloses a system for adding time-series queries to a key-value database, comprising a query syntax differentiation module, a structure addition and parsing module, a query parsing module, a query condition parsing module, and a plan parsing and execution module. This system can add time-series queries to a key-value database using the method disclosed in Embodiment 1.

[0071] The query syntax differentiation module is used to add time-series query differentiation syntax, which is a query syntax that distinguishes between time-series queries and key-value queries. Specifically, the query syntax differentiation module adds time-series query differentiation syntax through the following steps: using TIMESERIES SELECT as the identifier for time-series queries, and adding TIMESERIES as the identifier for semantic parsing in the SELECTCLUSE structure after syntax parsing.

[0072] The structure parsing module is used to add time-series query structures corresponding to database, schema, table, and column, and to add time-series query structure parsing methods. The time-series query structure parsing methods are used to parse the time-series query structures corresponding to database, schema, and table to obtain metadata.

[0073] In a typical key-value query, the FROM clause is parsed to obtain the database, schema, and table IDs. These IDs are then used to construct a key to query metadata in the specified system table. However, since time-series data does not have the usual database, schema, and table names, but only corresponding structures, it is necessary to first query the corresponding database, schema, and table names using the structure names before proceeding with the previous steps. Therefore, a time-series query structure graph parsing method needs to be added during semantic parsing. This method parses the structure into names to parse the time-series query structure corresponding to the database, schema, and table, and returns the parsed information to be stored in a structure.

[0074] The query parsing module is used to add parsing functions for parsing time-series query collection points, where each time-series query collection point is a time-series query structure corresponding to a column.

[0075] When parsing the query list of a key-value query, the key-value (KV) query parses the columns using the returned structure and the query list structure, returning a `scopecolumn` structure. Time-series queries, however, do not have specific columns; they only have a corresponding time-series query structure, `datapoint`. Therefore, a method needs to be added to parse `datapoint` into `scopedatapoint`, which is used to store the time-series query condition columns.

[0076] The query condition parsing module is used to add time-series query parsing functions to parse time-series query conditions.

[0077] When parsing query conditions, key-value queries use the scopecolumn structure from step 5 to parse the query condition expressions. Therefore, in time-series query semantic parsing, it is necessary to add a method that uses scopedatapoint to parse the query condition expressions.

[0078] The plan parsing and execution module is used to distinguish syntax based on time-series queries, add time-series query structure parsing methods, parse parsing functions for time-series query collection points, and configure time-series query logical plans using time-series query parsing functions. It then parses the semantically parsed time-series query logical plans into physical plans and executes the physical plans.

[0079] During execution, the plan parsing and execution module obtains metadata through the following steps:

[0080] (1) The time-series query structures corresponding to database, schema, and table are parsed using the time-series query structure parsing method to obtain the corresponding parsed information;

[0081] For the time-series query structure corresponding to the database, the parsed information includes the database name;

[0082] For the time-series query structure corresponding to scehma, the parsed information includes the name of scehma;

[0083] For the time-series query structure corresponding to the table, the parsed information includes the table's name;

[0084] (2) Obtain the corresponding IDs of database, scehma and table by name respectively, and construct a key based on the IDs of database, scehma and table. Query metadata in the system table based on the constructed key.

[0085] The present invention has been shown and described in detail above with reference to the accompanying drawings and preferred embodiments. However, the present invention is not limited to these disclosed embodiments. Based on the above embodiments, those skilled in the art will know that more embodiments of the present invention can be obtained by combining the means in the different embodiments described above, and these embodiments are also within the protection scope of the present invention.

Claims

1. A method for adding time-series queries to a key-value database, characterized in that... Includes the following steps: Add a time-series query differentiation syntax, use TIMESERIES SELECT as the identifier for time-series queries, and add TIMESERIES as the identifier for semantic parsing in the SELECTCLUSE structure after syntax parsing. The time-series query differentiation syntax is a query syntax that distinguishes between time-series queries and key-value queries. Add time-series query structures corresponding to database, schema, table, and column respectively, and add time-series query structure parsing methods. Use the time-series query structure parsing methods to parse the time-series query structures corresponding to database, schema, and table to obtain metadata. Add a parsing function to parse the time-series query collection point, where the time-series query collection point is the time-series query structure corresponding to the column, and the time-series query structure corresponding to the column is a datapoint. The parsing function to parse the time-series query collection point is used to parse the datapoint into a scopedatapoint, and the scopedatapoint is used to store the column. Add a time-series query parsing function to parse time-series query conditions. The time-series query parsing function is used to parse query condition expressions using scopedatapoint. Based on the time-series query distinguishing syntax, adding a time-series query structure parsing method, parsing the parsing function of the time-series query collection points, and configuring the time-series query logical plan using the time-series query parsing function, the semantically parsed time-series query logical plan is parsed into a physical plan and the physical plan is executed. The metadata is obtained through the following steps: The time-series query structures corresponding to database, schema, and table are parsed using the time-series query structure parsing method to obtain the corresponding parsed information. For the time-series query structure corresponding to the database, the parsed information includes the database name; For the time-series query structure corresponding to the schema, the parsed information includes the schema's name; For the time-series query structure corresponding to the table, the parsed information includes the table's name; Obtain the corresponding IDs for the database, schema, and table names respectively, and construct a key based on the database, schema, and table IDs. Use the constructed key to query metadata in the specified system table.

2. A system for adding time-series queries to a key-value database, characterized in that... The system is used to add time-series queries to a key-value database using the method for adding time-series queries to a key-value database as described in claim 1. The system includes: The query syntax differentiation module is used to add time-series query differentiation syntax, using TIMESERIES SELECT as the identifier of time-series queries, and adding TIMESERIES as the identifier for semantic parsing in the SELECTCLUSE structure after syntax parsing. The time-series query differentiation syntax is a query syntax that distinguishes between time-series queries and key-value queries. A structure parsing module is added. This module is used to add time-series query structures corresponding to database, schema, table, and column, and to add time-series query structure parsing methods. The time-series query structure parsing methods are used to parse the time-series query structures corresponding to database, schema, and table to obtain metadata. The query parsing module is used to add a parsing function to parse the time-series query collection point. The time-series query collection point is the time-series query structure corresponding to the column. The time-series query structure corresponding to the column is a datapoint. The parsing function to parse the time-series query collection point is used to parse the datapoint into a scopedatapoint. The scopedatapoint is used to store the column. The query condition parsing module is used to add a time-series query parsing function to parse time-series query conditions. The time-series query parsing function is used to parse query condition expressions using scopedatapoint. The plan parsing and execution module is used to distinguish the syntax of time-series queries, add a time-series query structure parsing method, parse the parsing function of time-series query collection points, and configure the time-series query logical plan based on the time-series query parsing function. It parses the semantically parsed time-series query logical plan into a physical plan and executes the physical plan. The plan parsing and execution module is used to obtain metadata through the following steps: The time-series query structures corresponding to database, schema, and table are parsed using the time-series query structure parsing method to obtain the corresponding parsed information. For the time-series query structure corresponding to the database, the parsed information includes the database name; For the time-series query structure corresponding to the schema, the parsed information includes the schema's name; For the time-series query structure corresponding to the table, the parsed information includes the table's name; Obtain the corresponding IDs for the database, schema, and table names respectively, and construct a key based on the database, schema, and table IDs. Use the constructed key to query metadata in the specified system table.

Images