Query optimization method and device based on time sequence data tag index

Abstract

The invention relates to the technical field of time series data query, in particular to a query optimization method and device based on a time series data tag index. The method comprises the steps of obtaining a query statement written by a user according to distribution characteristics of label data, analyzing the query statement to obtain an index prompt, and constructing a plurality of index lists based on the index prompt; obtaining a use sequence of the label indexes corresponding to the index object names in the index list; executing query by using the tag index with the highest priority to obtain an initial query result; executing query by using other tag indexes in sequence to obtain an intermediate query result corresponding to each tag index; and performing operation on the initial query result and all the intermediate query results to obtain time series data. The method can support the user to suggest the tag index for querying the time series data and the use sequence thereof, expand the optimization space of time series data query, and improve the overall query performance of a time series data processing system.

Description

Technical Field

[0001] This invention relates to the field of time-series data query technology, and in particular to a query optimization method and apparatus based on time-series data tag index. Background Technology

[0002] Time series data is a sequence of data recorded in chronological order, consisting of multiple data points arranged chronologically. Each data point, as an element in the sequence, contains a timestamp, and the data points exhibit temporal continuity and correlation. A typical time series dataset consists of several labels, at least one measurement data point, and a timestamp. Labels describe the data source, such as device number, sensor unique identifier, service name, etc.; measurement data represents specific observations or collected data, such as temperature, voltage, stock price, etc.; and the timestamp identifies the moment the time series data was generated or collected. In querying and analyzing time series data, labels are a key dimension for data filtering, aggregation, and location. To improve the efficiency of label-based queries and avoid scanning all irrelevant data, modern time series databases generally build indexing mechanisms for label data.

[0003] Traditional time-series databases primarily employ two methods to store and index tag data: The first method follows the design principles of traditional relational databases, storing tag data in a separate table and linking it to the main time-series table via foreign keys. This approach offers the advantage of allowing users to flexibly define the tag index structure and provides high scalability. However, its drawbacks are also significant: explicit table combination operations are required during queries, increasing query complexity; and when writing data, users must split the time-series data into tag and measurement data on the application side and write them to different tables, significantly raising the barrier to entry and development costs. The second method addresses the high cardinality and selectivity of tag data by designing dedicated storage and index structures (such as inverted indexes and log-structured merged tree indexes), integrating tags and time series data. While this method improves the performance of queries for specific tags, its index structure is typically predefined or statically configured, preventing users from dynamically adding or adjusting index strategies based on actual business needs. In addition, such systems often centrally manage all time-series data logically, resulting in complex table structures and difficult metadata management. The maintenance cost is extremely high, especially when facing large-scale label changes or dynamic schema expansions.

[0004] To overcome the shortcomings of the two methods mentioned above in terms of data management complexity and ease of use, a new storage paradigm is proposed: storing all time-series data and their tags in a single logical wide table. This method eliminates the need for cross-table joins by flattening the data model, greatly simplifying the data writing and querying process and improving system usability and management efficiency. However, while this single logical table storage method brings management convenience, it also introduces new performance challenges, especially in high-cardinality tag scenarios, i.e., applications containing a large number of unique tag values. How to efficiently utilize indexes for data filtering becomes a key issue. High-cardinality tag scenarios are characterized by a smaller average amount of data generated by a single data source compared to ordinary scenarios, but with a high degree of tag uniqueness. Therefore, in this case, the proportion of index access overhead in the total query overhead increases significantly, making query optimization based on tag indexes particularly important.

[0005] Therefore, overcoming the shortcomings of the existing technology is an urgent problem to be solved in this technical field. Summary of the Invention

[0006] To address the aforementioned deficiencies or improvement needs of existing technologies, this invention proposes a query optimization method and apparatus based on time-series data tag indexes. This method supports user prompts for selecting query tag indexes, increases the optimization space for time-series data queries based on tag indexes, and thereby improves the query performance of time-series data processing systems.

[0007] The embodiments of the present invention adopt the following technical solutions: In a first aspect, the present invention provides a query optimization method based on time-series data tag index, specifically: obtaining a query statement written by a user based on the distribution characteristics of tag data, parsing the query statement to obtain index hints, and constructing multiple index lists based on the index hints; Obtain the usage order of the tag indexes corresponding to the index object names in the index list; The query is executed using the highest priority tag index to obtain the initial query results; the other tag indexes are then executed in sequence to obtain the intermediate query results for each tag index. The initial query results and all intermediate query results are processed to obtain time series data.

[0008] Preferably, the index list includes an enabled index list, an ignored index list, a priority index list, and a delayed index list, and the step of obtaining the usage order of the tag indexes corresponding to the index object names in the index list includes: Retrieve the names of all index objects in the index list; By combining the enabled index list and the ignored index list, all index object names are filtered, and the index object names that meet the criteria are added to the intermediate results list. By combining the priority index list and the deferred index list, the name of each index object in the intermediate result list is sorted to obtain the usage order of the tag index corresponding to the index object name.

[0009] Preferably, the step of combining the enabled index list and the ignored index list to filter all index object names and adding the index object names that meet the conditions to the intermediate result list includes: For each index object name, determine whether the index object name exists in both the enabled index list and the ignored index list; If not, determine whether the index object name exists alone in the enabled index list; If so, add the index object name to the intermediate results list.

[0010] Preferably, the step of combining the priority index list and the delayed index list to sort each index object name in the intermediate result list to obtain the usage order of the tag index corresponding to the index object name includes: For each index object name in the intermediate results list, if the index object name is in the preferred index list, then add the index object name to the preferred list in order; If the index object name is in the deferred index list, then add the index object name to the deferred list in order; Otherwise, add the indexed object names to the regular list in order; By concatenating the priority list, the ordinary list, and the deferred list in sequence, the usage order of the tag index corresponding to the index object name is obtained.

[0011] Preferably, the enable index clause in the index hint is parsed; Retrieve the tag names corresponding to all tags in the enabled index clause in sequence; For each tag, obtain the tag index corresponding to each tag based on the tag name, and add the index object name of the tag index to the enabled index list.

[0012] Preferably, the method for constructing the enabled index list further includes: Parse the enable index clause in the index suggestion; Retrieve the index object names corresponding to all tag indexes in the enabled index clause in sequence; Add all indexed object names to the enabled index list.

[0013] Preferably, the method for constructing the priority index list includes: Parse the preferred index clause in the index hint; Retrieve the tag names corresponding to all tags in the priority index clause in order; For each tag, obtain the tag index corresponding to each tag based on the tag name, and add the index object name of the tag index to the priority index list.

[0014] Preferably, the step of performing calculations on the initial query results and all intermediate query results to obtain time-series data includes: Parse the join operators of the filtering conditions in the query statement; When the join operator is AND, the initial query result and all intermediate query results are intersected to obtain time series data; When the join operator is OR, the initial query result and all intermediate query results are combined to obtain time series data.

[0015] In a second aspect, the present invention provides a query optimization apparatus based on time-series data tag index, specifically comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the processor to perform the query optimization method based on time-series data tag index in the first aspect.

[0016] Thirdly, the present invention also provides a non-volatile computer storage medium storing computer-executable instructions that are executed by one or more processors to perform the method described in the first aspect.

[0017] Compared with existing technologies, the advantages of this invention are as follows: users are more familiar with the distribution characteristics of tag data. Therefore, when writing query statements, users can explicitly specify the tag indexes used to query time-series data and the order in which they are used through index hints. The query optimizer parses the query statement, identifies and extracts the index hints, obtains the user-suggested tag indexes and their usage order, and prioritizes the user-suggested tag indexes based on their filtering capabilities for time-series data. When accessing the time-series database to obtain time-series data, tag indexes with good filtering effects and low overhead are used preferentially. This method can support user-suggested tag indexes for querying time-series data and their usage order, expanding the optimization space for time-series data queries and improving the overall query performance of the time-series data processing system. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments of the present invention will be briefly described below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0019] Figure 1 This is a flowchart illustrating a query optimization method based on time-series data tag index provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of a process for constructing an enabled index list provided by an embodiment of the present invention; Figure 3 This is a flowchart illustrating the specific implementation method of step 102 provided in this embodiment of the invention; Figure 4 A schematic diagram of the structure of a query optimization device based on time-series data tag index provided in an embodiment of the present invention; The reference numerals in the attached figures are as follows: 21: Processor; 22: Memory. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0021] Unless the context otherwise requires, throughout the specification and claims, the term "comprising" is interpreted as openly inclusive, meaning "including, but not limited to." In the description of the specification, terms such as "one embodiment," "some embodiments," "exemplary embodiment," "example," "specific example," or "some examples" are intended to indicate that a particular feature, structure, material, or characteristic associated with that embodiment or example is included in at least one embodiment or example of this disclosure. The illustrative representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics mentioned may be included in any suitable manner in any one or more embodiments or examples; that is, although they may be incorporated into embodiments or examples using the above terms for reasons such as order and position, it does not limit them to be incorporated in combination by a single embodiment or example.

[0022] In the description of this invention, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined with "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of embodiments of this disclosure, unless otherwise stated, "a plurality of" means two or more. Furthermore, for example, the description may use the prefix "A" or "B" to describe the same type of nouns as two independent entities. In this case, the corresponding features defined with "A" and "B" are used only to distinguish between similar entities and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features.

[0023] In the description of this invention, the expression “A and / or B” (where A and B are used to formally represent specific features) will be used. The corresponding expression includes the following three combinations: only A, only B, and a combination of A and B.

[0024] As used in this invention, “about,” “approximately,” or “approximately” includes the stated value and the average value within an acceptable range of deviation from a particular value, wherein the acceptable range of deviation is determined by a person skilled in the art taking into account the measurement under discussion and the error associated with the measurement of the particular quantity (i.e., the limitations of the measurement system).

[0025] Furthermore, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0026] To facilitate understanding of this solution, let's first explain the terms used in this article. A tag is a key-value pair used to distinguish time-series data. For example, `region='us-west'` is a tag; `region` is the tag name. When storing tags in a time-series database, one tag name may correspond to multiple tag values ​​(such as `us-west`). To improve retrieval efficiency, time-series databases typically create dedicated tag indexes for tags. Specifically, this involves sorting the tag values ​​contained in a tag to generate a sorted tag index (stored as a table in the time-series database). For example, sorting by gender (male and female) will place all data with the gender of male before all data with the gender of female in the table. To facilitate the management of tag indexes, time-series databases create a unique index object name for each tag index. In this case, the index object name generally corresponds to the tag name. For example, if the tag name is `tag1`, the index object name is `tag1_idx`. Another method for building a tag index is to sort based on the tag values ​​contained in multiple tags. For example, first sort by gender (male and female), then sort the sorted data by age. At this time, all data with the gender of male will also be sorted by age from largest to smallest or smallest to largest, thus obtaining the tag index.

[0027] Example 1: Time-series databases typically rely on query optimizers to parse user-submitted queries, identify tags and related conditional expressions, and optimize the query execution plan based on statistical information. In typical tag-filtering queries, users often specify a set of tag matching conditions using the WHERE clause (e.g., region='us-west' AND device_type = 'sensor'). The database quickly retrieves the desired time-series data based on these conditions. Furthermore, to accelerate tag-based filtering, databases usually create dedicated tag indexes. The query optimizer, by understanding the query and considering the structural characteristics and underlying data distribution of the tag indexes, dynamically decides the order and combination strategy for selecting tag indexes. The database then retrieves time-series data that matches the matching conditions based on this selection and combination strategy. However, tag data has strong business relevance, and tags vary significantly in data type, cardinality (number of unique values), and distribution density, making simple tag index selection strategies difficult to adapt to complex and variable query loads.

[0028] Specifically, there are two core challenges: First, the effectiveness of tag indexes is affected by the distribution of tag data. For example, for a conditional expression containing a tag A with many unique values ​​(e.g., tag A is age) and a tag B with fewer unique values ​​(e.g., tag B is gender), the smallest tag index containing tag A can be considered. However, if the expression containing tag A actually contains the vast majority of tag A's values, then the tag index for tag A will not be able to filter effectively, and accessing the time-series database may even lead to performance degradation. The more unique values ​​tag A contains, the greater the performance overhead of using the tag index for tag A. Second, the order in which tag indexes are used affects overall query performance. For example, for a conditional expression containing a tag A with many unique values ​​and a tag B with fewer unique values, if the expression containing tag B can filter out a lot of time-series data, and tag A uses an index whose access overhead to the time-series database increases with the amount of data (such as a hash index), prioritizing the use of the tag index for tag A will lead to scanning more data when using the tag index for tag B subsequently, thus increasing the access overhead to the time-series database.

[0029] Therefore, the choice of which tag indexes to use for queries and the execution order of these tag indexes directly impact query efficiency. To address the aforementioned issues, such as... Figure 1 As shown, Embodiment 1 of the present invention provides a query optimization method based on time-series data tag index, which specifically includes the following steps: Step 101: Obtain the query statement written by the user based on the distribution characteristics of the tag data, parse the query statement to obtain index hints, and construct multiple index lists based on the index hints.

[0030] In one embodiment, since tags describe the source information of time-series data (such as device unique identifiers, service names, regions, etc.), they are relatively static and have stable patterns compared to frequently changing measurement data. Through long-term use and business practice, users often have a deeper understanding of the cardinality, value distribution, and selectivity of tags. Therefore, in certain complex or specific query scenarios, the optimization knowledge possessed by users may exceed the automatic judgment based on statistical information within the database. In this case, the query optimizer can parse the index hints in the query statement to obtain user suggestions. These suggestions include the tag indexes used when querying time-series data that meets preset conditions and their order of use, using relevant tag indexes to accelerate retrieval efficiency. The preset conditions refer to conditions based on tag filtering and / or conditions based on time filtering.

[0031] Step 102: Obtain the usage order of the tag indexes corresponding to the index object names in the index list.

[0032] In one embodiment, the query optimizer can construct a query plan and adjust the tag indexes used in the query plan and their order of use based on user suggestions. Specifically, based on user suggestions, the query optimizer first filters all tag indexes in the index list to obtain the tag indexes recommended by the user. Then, based on the filtering capabilities of the tag indexes for time-series data, it prioritizes the tag indexes, that is, it determines which tag indexes to use first to retrieve time-series data and which tag indexes to use later. To speed up retrieval efficiency, users, based on their understanding of the distribution characteristics of tag data, tend to assign higher priority to tag indexes containing fewer unique values ​​(compared to other tag indexes in the index list). In this scenario, using the tag index can filter out more time-series data that does not meet the conditions.

[0033] Optionally, for tagged indexes not specified by the user in the index hints, the query optimizer shall still decide whether to use them and the order in which they are used, without violating the user hints.

[0034] Optionally, the query optimizer can use the user's index suggestions as a basis for optimizing the tag indexes, and then consider the use and order of other tag indexes.

[0035] In addition, time-series data also has application scenarios such as high real-time requirements, high-frequency queries of small amounts of data, and write-while-querying. The method in this embodiment can further improve query performance in such scenarios and better meet the performance needs of users.

[0036] Step 103: Execute the query using the highest priority tag index to obtain the initial query result; then execute the query using the other tag indexes in sequence to obtain the intermediate query result corresponding to each tag index.

[0037] In one embodiment, the query optimizer obtains the usage order of the tag index and retrieves the time-series data according to the usage order.

[0038] Furthermore, since the highest priority tag index contains fewer unique values, when using this tag index, most of the time series data that does not meet the conditions can be filtered out, resulting in a smaller amount of time series data that meets the conditions. At this point, based on the judgment of retrieval efficiency, the query optimizer will not use the remaining tag indexes for retrieval even if there are still unused tag indexes. Instead, it will change the index query to condition matching, and use condition matching to find the time series data that meets the conditions in the initial query results, further accelerating the retrieval efficiency.

[0039] Step 104: Perform calculations on the initial query results and all intermediate query results to obtain time series data that meets all conditions.

[0040] In one embodiment, filtering conditions in a query statement can be connected using Boolean logical operators (such as AND or OR) to form compound query conditions, supporting complex multi-dimensional filtering needs. When a user enters a query statement, the query optimizer parses the connection operators of the filtering conditions in the query statement. When the connection operator is AND, that is, when multiple filtering conditions are connected using the "AND" Boolean logical operator, the query optimizer performs an intersection operation on the initial query result and all intermediate query results to obtain time-series data. When the connection operator is OR, that is, when multiple filtering conditions are connected using the "OR" Boolean logical operator, the query optimizer performs a union operation on the initial query result and all intermediate query results to obtain time-series data.

[0041] Optionally, if the query statement does not contain index hints, it indicates that the user believes that using condition matching to retrieve time-series data is more efficient than using tag indexes. In this case, the index list is empty, and the query optimizer directly scans the entire data table to perform condition matching to obtain time-series data that meets the conditions.

[0042] In this embodiment, users are more familiar with the distribution characteristics of tag data. Therefore, when writing queries, users can explicitly specify the tag indexes used to query time-series data and the order in which they are used through index hints. The query optimizer parses the query statement, identifies and extracts the index hints, obtains the user-suggested tag indexes and their usage order, and prioritizes the user-suggested tag indexes based on their filtering capabilities for time-series data. When accessing the time-series database to obtain time-series data, tag indexes with good filtering effects and low overhead are used first. This method supports the hint function for time-series data tag indexes, allowing users to influence factors such as whether tag indexes are enabled or disabled, and the access order, thereby optimizing the performance of the time-series data processing system in filtering time-series data based on tag indexes.

[0043] The index list includes an enabled index list, an ignored index list, a preferred index list, and a delayed index list. The construction method of the enabled index list is described below.

[0044] like Figure 2 As shown, Embodiment 1 of the present invention provides a method for constructing an enabled index list, which specifically includes the following steps: Step 201: Parse the enable index clause in the index hint.

[0045] In one embodiment, index hints allow users to control the use of tag indexes at the granularity of tag names. That is, users can use index hints to precisely specify whether to use the tag index corresponding to a specific tag name when querying time-series data, thereby achieving fine-grained control over the use of tag indexes.

[0046] Furthermore, to assist the query optimizer in selecting different forms of tag indexes and composite indexes corresponding to the same tag, index hints allow users to control the use of tag indexes at the granular level of index objects. That is, users can use index hints to select and specify which specific index object names correspond to the tag indexes to use based on the specific situation. Through this fine-grained control, the query optimizer can more accurately adjust the query execution plan to ensure optimal query performance.

[0047] Step 202: Retrieve the tag names corresponding to all tags in the Enable Index clause in sequence.

[0048] Step 203: For each tag, obtain the tag index corresponding to each tag according to the tag name, and add the index object name of the tag index to the enabled index list.

[0049] In one embodiment, when a user specifies precisely which tag index to use when querying time-series data using index hints, the query optimizer finds the corresponding tag index (generated by sorting the tag values ​​contained in a tag) based on the tag name and adds the index object name of the tag index to the enabled index list. Subsequently, when the database uses the tag index to perform time-series data retrieval, it is essentially performing specific operations on the specific data table corresponding to the tag index. For example, the query statement can use the INDEX HINT(USE tag_col1, tag_col2) clause to enable the tag indexes for tags tag_col1 and tag_col2; and use the INDEX HINT(IGNORE tag_col1, tag_col2) clause to ignore the tag indexes for tags tag_col1 and tag_col2. Here, tag_col1 and tag_col2 are the tag names corresponding to the tags.

[0050] Furthermore, when a user selects and specifies which specific tag indexes to use based on the index hints, they are essentially directly specifying the index object names corresponding to the tag indexes in the index clause. Specifically: the enable index clause in the index hints is parsed; the index object names corresponding to all tag indexes in the enable index clause are obtained sequentially; and all index object names are added to the enabled index list. For example, in a query statement, the INDEX HINT(USE tag_index1, tag_index2) clause supports enabling tag indexes with index object names tag_index1 and tag_index2, and the INDEX HINT(IGNORE tag_index1, tag_index2) clause supports ignoring tag indexes with index object names tag_index1 and tag_index2. Here, tag_index1 and tag_index2 are the index object names.

[0051] The method for constructing the list of ignored indices is the same as the method for constructing the list of enabled indices, so it will not be repeated here.

[0052] In one embodiment, the method of constructing the priority index list is the same as the method of constructing the deferred index list. Here, the method of constructing the priority index list is described as follows: parse the priority index clause in the index hint; obtain the tag names corresponding to all tags in the priority index clause in order; for each tag, obtain the tag index corresponding to each tag according to the tag name, and add the index object name of the tag index to the priority index list. For example, in a query statement, the INDEX HINT(FIRST tag_col1, tag_col2) clause supports prioritizing the use of the tag indexes named tag_col1 and tag_col2, while the INDEX HINT(LAST tag_col1, tag_col2) clause supports deferring the use of the tag indexes named tag_col1 and tag_col2. Similarly, in a query statement, the INDEX HINT(FIRST tag_index1, tag_index2) clause supports prioritizing the use of the tag indexes named tag_index1 and tag_index2, while the INDEX HINT(LAST tag_index1, tag_index2) clause supports deferring the use of the tag indexes named tag_index1 and tag_index2.

[0053] like Figure 3 As shown, Embodiment 1 of the present invention provides a specific implementation method for step 102, which specifically includes the following steps: Step 301: Get all index object names in the index list.

[0054] In one embodiment, all index object names in the enabled index list, ignored index list, preferred index list, and deferred index list are obtained. In this case, the tagged index corresponding to the index object name represents the tagged index suggested by the user. However, since the ignoring index clause essentially means that the user does not recommend using the tagged index, and users may make mistakes when writing Structured Query Language (SQL), the query optimizer needs to check all index object names first to identify the index object names suggested by the user in the index hints.

[0055] Step 302: Combine the enabled index list and the ignored index list to filter all index object names, and add the index object names that meet the criteria to the intermediate results list.

[0056] In one embodiment, when determining whether each index object name needs to be added to the intermediate results list, it is necessary to traverse the enabled index list and the ignored index list, and add the index object names that meet the conditions to the intermediate results list. At this time, the intermediate results list contains the tag indexes suggested by the user.

[0057] Specifically, for each index object name, by traversing the enabled index list, it is determined whether the index object name exists in both the enabled index list and the ignored index list. For example, if both the enabled index list and the ignored index list contain tag_index4, it may be a user error, and the query optimizer will not consider using the tag index corresponding to tag_index4 for retrieval. If not, it is determined whether the index object name exists alone in the enabled index list. If so, the index object name is added to the intermediate result list. For example, when determining tag_index2, if the ignored index list contains tag_index1 and the enabled index list contains tag_index2, then tag_index2 exists alone in the enabled index list. Tag_index2 is added to the intermediate result list, and the tag index corresponding to tag_index1 is not considered for retrieval. This process continues to determine each tag index until all tag indexes have been determined.

[0058] Step 303: Combine the priority index list and the delayed index list, sort each index object name in the intermediate result list to obtain the usage order of the tag index corresponding to the index object name.

[0059] In one embodiment, the preferred index clause contains the index object name of the tag index that the user suggests should be used first, and the deferred index clause contains the index object name of the tag index that the user suggests should be used later. By combining the preferred index list and the deferred index list, each index object name in the intermediate result list can be sorted according to the user's prompts to obtain the usage order of the tag indexes corresponding to the index object names.

[0060] Specifically, for each index object name in the intermediate result list, if the index object name is in the priority index list, it is added to the priority list; if the index object name is in the deferred index list, it is added to the deferred list; otherwise, it is added to the normal list. The priority list, the normal list, and the deferred list are concatenated in order to obtain the usage order of the tag indexes corresponding to the index object name. For example, if the intermediate result list contains tag_index1, tag_index2, and tag_index3, the priority index list contains tag_index3, and the deferred index list contains tag_index2, after sorting, the priority order of the tag indexes is tag_index3, tag_index1, and tag_index2. That is, the tag index corresponding to tag_index3 is used first to retrieve the initial query result, and then the tag indexes corresponding to tag_index1 and tag_index2 are used sequentially to retrieve the intermediate query result.

[0061] In this embodiment, the more static and user-familiar distribution of tag data in time-series data is fully utilized to support user suggestions for tag indexes used in querying time-series data. This increases the optimization space for time-series data queries based on tag indexes, thereby improving the query performance of the time-series data processing system. Furthermore, based on the application scenarios of high cardinality and high real-time requirements of time-series data, a design is proposed that supports user suggestions for the order in which tag indexes are used, which can further reduce the access overhead to the time-series database, thereby improving the query performance of the time-series data processing system. Moreover, the system supports user suggestions for tag index usage based on both tag name and index object name, allowing users to flexibly choose between different types of indexes and composite indexes on the same column, while retaining the simplicity of suggestions based on tag name.

[0062] Example 2: The following is an example illustrating a query optimization method based on time-series data tag indexes provided in this application. For ease of understanding, Table 1, as shown in Table 1, is used as an example to illustrate the query process.

[0063] Table 1 Partial Definitions of Data Table 1

[0064] This embodiment implements the query suggestion function of tag index in the form of SQL-like syntax. In Table 1, a tag index is generated after sorting the tag values ​​in tag1, and the index object corresponding to the tag index is named tag1_idx; a tag index is generated after sorting the tag values ​​in tag2, and the index object corresponding to the tag index is named tag2_idx; a tag index is generated after sorting the tag values ​​in tag1, tag2 and timestamp ts, and the index object corresponding to the tag index is named series_idx.

[0065] In this embodiment, the query statement SELECT tag1,tag2,value1,ts FROM table1 WHERE tag1 ='abc' AND tag2<5 AND ts>= '2025-01-01' AND ts<'2025-01-07' INDEX HINT(USEtag1,tag2 IGNORE series_idx FIRST tag2 LAST tag1) is used as an example for illustration.

[0066] After the user inputs the query, the query optimizer performs syntax and semantic checks and finds that the query contains index hints. It then constructs four lists of index object names. Specifically, since `tag1` and `tag2` are the tag names corresponding to the tags, the query optimizer finds the tag indexes corresponding to `tag1` and `tag2` respectively. The index object names of these tag indexes are `tag1_idx` and `tag2_idx`, respectively. At this point, the used indexes list `used_indexes` contains `tag1_idx` and `tag2_idx`. Since `series_idx` is an index object name, it is directly added to the ignored indexes list. At this point, the ignored indexes list `ignored_indexes` contains `series_idx`. Similarly, the first indexes list `first_indexes` contains `tag2`, and the last indexes list `last_indexes` contains `tag1`.

[0067] Subsequently, the query optimizer analyzes the query and finds that it filters conditionally for `tag1`, `tag2`, and `ts`, and that the `Table1` table contains corresponding tag indexes. At this point, the query optimizer can obtain the user-suggested tag indexes based on index suggestions. Specifically, for the index object name `series_idx` to be selected, it checks the enabled index list `used_indexes` and the ignored index list `ignored_indexes`. Finding that `series_idx` is in the ignored index list, it is not selected. For the index object names `tag1_index` and `tag2_index` to be selected, it checks the enabled index list `used_indexes` and the ignored index list `ignored_indexes`. Finding that both are in the enabled index list `used_indexes`, both are selected and added to the intermediate result list `{tag1_index, tag2_index}` in sequence. Next, the query optimizer iterates through the enabled index list `used_indexes` and finds no unselected index object names. Thus, the tag index selection based on the user-suggested index object names is complete. Finally, the query optimizer iterates through the priority index list `first_indexes` and the deferred index list `last_indexes`, prioritizing `tag1_index` and `tag2_index` in the intermediate result list. The sorted result is `{tag2_index, tag1_index}`, completing the query optimization related to the tag indexes. After receiving the user-suggested order of tag index usage, the query optimizer first performs time-series data retrieval based on the tag index corresponding to `tag2_index` to obtain the initial query results. Then, it performs time-series data retrieval based on the tag index corresponding to `tag1_index` to obtain intermediate query results. The initial query results and intermediate query results are then intersected, and time-condition filtering is performed to obtain the final query results that meet all conditions.

[0068] Example 3: Based on the query optimization method based on time-series data tag index provided in the foregoing embodiments, the present invention also provides an apparatus for query optimization based on time-series data tag index that can be used to implement the above method, such as... Figure 4 The diagram shown is a schematic representation of the device architecture according to an embodiment of the present invention. The device for query optimization based on time-series data tag indexing in this embodiment includes one or more processors 21 and a memory 22. Figure 4 Take a processor 21 as an example.

[0069] Processor 21 and memory 22 can be connected via a bus or other means. Figure 4 Taking the example of a connection between China and Israel via a bus.

[0070] The memory 22, as a non-volatile computer-readable storage medium for query optimization based on time-series data tag indexes, can be used to store non-volatile software programs and non-volatile computer-executable programs, such as the query optimization method based on time-series data tag indexes in the foregoing embodiments. The processor 21 executes various functional applications and data processing of the query optimization device based on time-series data tag indexes by running the non-volatile software programs, instructions, and modules stored in the memory 22, thereby implementing the query optimization method based on time-series data tag indexes in the foregoing embodiments.

[0071] Memory 22 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, memory 22 may include memory remotely located relative to processor 21, which can be connected to processor 21 via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0072] The program instructions / modules are stored in memory 22. When executed by one or more processors 21, they perform the query optimization method based on time-series data tag index in the aforementioned embodiments, for example, executing the above-described... Figures 1-3 The steps shown.

[0073] This invention also provides a non-volatile computer storage medium storing computer-executable instructions that are executed by one or more processors, for example... Figure 4 One of the processors 21 can enable the above-described one or more processors to execute the query optimization method based on time-series data tag index in the foregoing embodiments, for example, to execute the above-described... Figures 1-3 The steps shown.

[0074] It is worth noting that the information interaction and execution process between the modules and units in the above-mentioned device and system are based on the same concept as the processing method embodiment of the present invention. For details, please refer to the description in the method embodiment of the present invention, and will not be repeated here.

[0075] Those skilled in the art will understand that all or part of the steps in the various methods of the embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, which may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0076] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A query optimization method based on time-series data label index, characterized in that, include: Obtain the query statement written by the user based on the distribution characteristics of the tag data, parse the query statement to obtain index hints, and construct multiple index lists based on the index hints; Obtain the usage order of the tag indexes corresponding to the index object names in the index list; The query is executed using the highest priority tag index, yielding the initial query results; Execute queries using the other tag indexes in sequence to obtain intermediate query results for each tag index; The initial query results and all intermediate query results are processed to obtain time series data.

2. The query optimization method based on time-series data tag index according to claim 1, characterized in that, The index list includes an enabled index list, an ignored index list, a priority index list, and a delayed index list. Obtaining the usage order of the tag indexes corresponding to the index object names in the index list includes: Retrieve the names of all index objects in the index list; By combining the enabled index list and the ignored index list, all index object names are filtered, and the index object names that meet the criteria are added to the intermediate results list. By combining the priority index list and the deferred index list, the name of each index object in the intermediate result list is sorted to obtain the usage order of the tag index corresponding to the index object name.

3. The query optimization method based on time-series data tag index according to claim 2, characterized in that, The process involves combining the enabled index list and the ignored index list to filter all index object names, adding those that meet the criteria to the intermediate result list, including: For each index object name, determine whether the index object name exists in both the enabled index list and the ignored index list; If not, determine whether the index object name exists alone in the enabled index list; If so, add the index object name to the intermediate results list.

4. The query optimization method based on time-series data tag index according to claim 2, characterized in that, The process of combining the priority index list and the delayed index list to sort each index object name in the intermediate result list, thereby obtaining the usage order of the tag indexes corresponding to the index object names, includes: For each index object name in the intermediate results list, if the index object name is in the preferred index list, then add the index object name to the preferred list in order; If the index object name is in the deferred index list, then add the index object name to the deferred list in order; Otherwise, add the indexed object names to the regular list in order; By concatenating the priority list, the ordinary list, and the deferred list in sequence, the usage order of the tag index corresponding to the index object name is obtained.

5. The query optimization method based on time-series data tag index according to claim 2, characterized in that, Methods for constructing an enabled index list include: Parse the enable index clause in the index suggestion; Retrieve the tag names corresponding to all tags in the enabled index clause in sequence; For each tag, obtain the tag index corresponding to each tag based on the tag name, and add the index object name of the tag index to the enabled index list.

6. The query optimization method based on time-series data tag index according to claim 2, characterized in that, Methods for constructing an enabled index list also include: Parse the enable index clause in the index suggestion; Retrieve the index object names corresponding to all tag indexes in the enabled index clause in sequence; Add all indexed object names to the enabled index list.

7. The query optimization method based on time-series data tag index according to claim 2, characterized in that, Methods for constructing a priority index list include: Parse the preferred index clause in the index hint; Retrieve the tag names corresponding to all tags in the priority index clause in order; For each tag, obtain the tag index corresponding to each tag based on the tag name, and add the index object name of the tag index to the priority index list.

8. The query optimization method based on time-series data tag index according to any one of claims 1-7, characterized in that, The process of performing calculations on the initial query results and all intermediate query results to obtain time-series data includes: Parse the join operators of the filtering conditions in the query statement; When the join operator is AND, the initial query result and all intermediate query results are intersected to obtain time series data; When the join operator is OR, the initial query result and all intermediate query results are combined to obtain time series data.

9. A query optimization device based on time-series data tag index, characterized in that, The device includes: At least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor for performing the query optimization method based on time-series data tag index as described in any one of claims 1-8.

10. A non-volatile computer storage medium, characterized in that, The computer storage medium stores computer-executable instructions, which are executed by one or more processors to perform the query optimization method based on time-series data tag index as described in any one of claims 1-8.

Images