Implementation method for filtering push-down of storage engine of time sequence database

Abstract

The invention discloses an implementation method for filtering and pushing down a storage engine of a time sequence database, and relates to the technical field of time sequence databases, which comprises the following steps of: designing the storage engine of the time sequence database into five levels including a time sequence table data main directory, Entity Groups, SubEntity Groups, time partitions and data subfolders, each Entity Group comprises Tag table data, Metric data and a Data data table, and each Entity Group comprises the Tag table data, the Metric data and the Data data table; designing data structures, filtering push-down elements and filtering push-down processes of the Tag table and the Data table; based on the above design, filtering push-down is implemented in each level of a time sequence database storage engine, a query execution plan is optimized, and query is executed to obtain a summarized result. According to the method, effective filtering and pushing-down can be realized, and the data reading and transmission quantity is reduced, so that the query performance is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of time series databases, and in particular to a method for implementing filtering and push-down of a time series database storage engine. Background Art

[0002] The time series database storage engine is specially designed for time series data to efficiently store and query such data.

[0003] Filter pushdown is a technique used to optimize query processing. Its core idea is to apply filter conditions in the early stages of query execution, thereby reducing the amount of data that needs to be processed. Filter pushdown mainly includes two aspects: predicate pushdown and index pushdown. Predicate pushdown is the process of pushing some conditions in the WHERE clause to the storage engine layer, which helps to filter data early and reduce data transmission and calculation. Index pushdown (ICP) is to use indexes to speed up queries in databases that support indexes. When a query statement contains multiple filter conditions, these conditions are combined to generate a filter predicate list, which is passed to the storage engine for physical filtering. Summary of the invention

[0004] In view of the needs and shortcomings of current technology development, the present invention provides a method for implementing filter push-down of a time series database storage engine to improve the speed of data query and reduce resource load.

[0005] The present invention provides a method for implementing filtering and push-down of a time series database storage engine, and the technical solution adopted to solve the above technical problems is as follows:

[0006] A method for implementing filter pushdown of a time series database storage engine specifically includes the following contents:

[0007] S1. The time series database storage engine is designed to include five levels: the main directory of the time series table data, EntityGroup, SubEntityGroup, time partition, and data subfolder. The main directory of the time series table data is the top-level organization form of data, which contains multiple EntityGroups. EntityGroup is responsible for organizing and managing the data in the time series table. Each EntityGroup is a relatively independent physical unit, including Tag table data, Metrics data, and Data data table.

[0008] S2. Design the data structure of the Tag table and the Data table;

[0009] S3. Design the filter push-down elements and filter push-down process of the Tag table and Data table;

[0010] S4. Based on the design of steps S1 to S3, by implementing filter push-down at each level of the time series database storage engine, the query execution plan is optimized and the query is executed to obtain a summary result.

[0011] Optionally, execute step S1, each EntityGroup includes Tag table data, Metrics data and Data data table. Tag table data is used to describe and classify time series data. It consists of a key and a value, which is used to specify the data subcategory under the metric. Metrics data is measurement data, which refers to the quantitative indicators of the monitored system. The Data data table is the actual data storage unit in the time series database, which is responsible for storing the specific information of each data point. Each data point consists of Metric, Timestamp, Value and multiple Tags, forming a specific time series record.

[0012] SubEntityGroup is responsible for balancing the space usage of metadata and entity query performance. Each SubEntityGroup contains a set number of entities.

[0013] Time partitions are used to organize and manage data by time range. Each time partition contains a meta-mapped shard table() to further refine data management.

[0014] The data subfolder is the lowest level of the time series database storage engine. It is a compressible unit and supports the undo / redo operations of Createtable / insert / delete / alter operations.

[0015] Further optionally, the data structure of the tag table involved includes a logical layer structure and a physical layer structure, wherein:

[0016] (A1) The logical structure of the Tag table includes:

[0017] SubEntity ID, a unique identifier for identifying a sub-entity group;

[0018] Entity ID, a unique identifier for identifying an entity;

[0019] Primary Tags, which means main tags, are used to quickly access and query data;

[0020] Non-primary Tags, meaning non-primary tags, are used to supplement the descriptive or dimensional information other than Primary Tags, providing an additional screening dimension;

[0021] Quickly access and query data;

[0022] Common Tags: additional tag information for further data description and filtering;

[0023] (A2) The physical layer structure of the tag table includes:

[0024] Tag.header file, which stores deletion tags and null value bitmaps, and is used to handle data deletion and null values;

[0025] Tag.pt file, which stores primary Tags, subEntity ID and entity ID;

[0026] var file: stores variable-length data by column, with one file for each column.

[0027] Further optionally, the filter push-down elements of the Tag table involved include: support for data types, support for predicate types, and packaged filter return results.

[0028] Further optionally, the steps of filtering and pushing down the Tag table involved include:

[0029] (a1) Traversing all Tag tables: In a time series database, each EntityGroup is usually associated with one or more Tag tables, which are used to store relevant metadata and tag information. The Tag tables in each EntityGroup are checked and processed one by one;

[0030] (a2) Register filter conditions for each Tag table: When executing a query, set the filter requirements for tags in the query conditions in advance, and then register the filter conditions in each Tag table to ensure that only entities that meet the filter conditions are queried and returned;

[0031] (a3) Parsing query conditions: The query execution plan first parses the filter conditions involving the Tag table;

[0032] (a4) Locate Tag data: According to the logical and physical layer structures of the Tag table, locate the relevant Tag.pt file and var file; the Tag.pt file contains primary Tags information, while the var file contains non-primary Tags information;

[0033] (a5) Apply filtering conditions: Apply the filtering conditions of primary Tags in the Tag.pt file, and apply the filtering conditions of non-primary Tags in the var file, and retain the data rows that meet the conditions;

[0034] (a6) Traverse all rows: In each Tag table, traverse all rows, find the rows that meet the filter conditions, and package the rows that do not meet the filter conditions and send them to the next processing stage;

[0035] (a7) Pack each column into a batch: Pack the same columns in multiple rows of data into batches;

[0036] (a8) Data retrieval: Based on the filtered data, retrieve the corresponding Entity ID and data file location information;

[0037] (a9) Result return: The filtered result set is returned to the upper-level application or further data processing flow. The returned data set only contains entities that meet the query conditions and their related data.

[0038] Optionally, the data structure of the Data table involved includes a logical layer structure and a physical layer structure, wherein:

[0039] (B1) The logical layer structure of the Data table includes:

[0040] EntityHeader, contains datafile id, used to identify the data file;

[0041] EntityItem, which contains entity_id, BlockItem ID, and total row count. It implements the function of mapping Entity to BlockItem. Each EntityItem represents an entity, which is a collection of time series data.

[0042] BlockItem, responsible for recording the data writing location, contains 1000 rows of columndata, which are stored in the .data file;

[0043] DataBlock is the lowest level structure of the data storage engine, which contains nullbitmap and columndata. Nullbitmap is used to mark which data is empty.

[0044] (B2) The physical layer structure of the Data table includes a .meta file and a .data file, wherein: the .meta file contains a MetaFileHeader and multiple EntityItems, and the .data file actually stores data and is organized according to the structure of BlockItem and DataBlock.

[0045] Further optionally, the filter push-down elements involved in the Data table include: data type support, predicate type support, data value of BlockItem located in the Data table, and packaged filter return results.

[0046] Further optionally, the filtering and pushing down process steps of the Data table involved include:

[0047] (b1) Query parsing: First, the query conditions are parsed, including time range, entity ID, and other relevant dimensions;

[0048] (b2) Data location: Map and locate the corresponding BlockItem and DataBlock according to the information in the EntityItem;

[0049] (b3) Traverse the iterator of each subgroup in turn: access each subgroup through the iterator and return the data;

[0050] (b4) Traverse each BlockItem: further traverse in the located BlockItem to prepare to apply the filter conditions to the data rows;

[0051] (b5) Traverse each row: traverse each row of data in each BlockItem, find the rows that meet the filter conditions, and pack all rows that do not meet the filter conditions;

[0052] (b6) Apply filter conditions: Use nullbitmap and columndata to apply time and preset filter conditions to extract data that meets the conditions, so as to filter out data that meets the query conditions from the located data blocks;

[0053] (b7) Data assembly: Assemble the filtered data blocks DataBlock into a result set and prepare to return it to the requester;

[0054] (b8) Result return: Return the final query result data set to the client or upper-level application.

[0055] Optionally, the step S4 involved specifically includes:

[0056] (c1) Apply filter conditions: First, determine the filter conditions that need to be pushed down. These filter conditions usually come from query requests and can be directly applied to the Tag table and Data table for data screening;

[0057] (c2) Building indexes: Building corresponding index structures at each level of the time series database storage engine;

[0058] (c3) Filtering by using indexes: When executing a query, first use the top-level index to filter out the main directories of the time series table data that do not meet the conditions, and then continue to use the indexes in the next-level EntityGroup and SubEntityGroup for further filtering;

[0059] (c4) Filtering at the data subfolder level: Finally, at the data subfolder level, the final filtering conditions are applied according to the time partition and the specific Data table structure to read the specific data files that meet the query conditions;

[0060] (c5) Optimize query plan: optimize query execution plan based on statistical information and query characteristics;

[0061] (c6) Execute the query and summarize the results: After applying the filtering conditions at each layer, the data that meets the conditions is passed to the upper layer and finally summarized and returned to the user.

[0062] Compared with the prior art, the method for implementing filtering and push-down of a time series database storage engine of the present invention has the following beneficial effects:

[0063] 1. The present invention can realize effective filtering and push-down, reduce the amount of data reading and transmission, and thus improve query performance;

[0064] 2. The present invention filters data early in the query execution process, so that the database system can avoid unnecessary calculations, thereby speeding up the query results; by pushing the filtering conditions to the data source, the amount of data transmitted between the storage and processing layers is minimized, especially for large data sets. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Attached Figure 1 It is a flow chart of the method of embodiment 1 of the present invention. DETAILED DESCRIPTION

[0066] In order to make the technical solution, the technical problem solved and the technical effect of the present invention more clearly understood, the technical solution of the present invention is clearly and completely described below in conjunction with specific embodiments.

[0067] The English words involved in the following embodiments are explained as follows:

[0068] Write-Ahead Logging (WAL), which means "pre-write log" in Chinese, is a common logging mechanism in database management systems (DBMS) to ensure data persistence and transaction atomicity. In WAL mode, all data modification operations are first written to the log file and then applied to the data file in the database. The main purpose of this mechanism is to provide a reliable recovery mechanism so that the system can be restored to a consistent state in the event of a crash or failure.

[0069] The Chinese meaning of EntityGroup is "entity group". EntityGroup is usually used to organize and manage multiple related entities to ensure that these entities can efficiently perform data interaction and update operations.

[0070] SubEntityGroup means "sub-entity group" in Chinese. Sub-entity group is a data structure used to further subdivide and manage entities in an entity group. By organizing entities into sub-entity groups, data processing and query operations can be performed more efficiently. In a time series database, sub-entity groups help balance the space usage of metadata and entity query performance, avoiding space waste and query performance consumption.

[0071] The Chinese meaning of meta-mapped shard table is "metadata-mapped shard table". Meta-mapped shard table effectively organizes time series data into different shard tables through metadata mapping, thereby improving data query efficiency and management convenience.

[0072] Embodiment 1:

[0073] Combined with Figure 1 This embodiment proposes a method for implementing filter pushdown in a time series database storage engine, which specifically includes the following contents:

[0074] S1. Design the time series database storage engine to include five levels: the main directory of time series table data, EntityGroup, SubEntityGroup, time partition, and data subfolder, where:

[0075] The main directory of the time series table data is the top-level organization form of data, including Write-Ahead Logging and multiple EntityGroups;

[0076] EntityGroup is responsible for organizing and managing the data in the time series table. Each EntityGroup is a relatively independent physical unit, including Tag table data, Metrics data and Data data table. Tag table data is used to describe and classify time series data. It consists of a key and a value to specify the data subcategory under the metric. Metrics data refers to the quantitative indicators of the monitored system. The Data data table is the actual data storage unit in the time series database, responsible for storing the specific information of each data point. Each data point consists of Metric, Timestamp, Value and multiple Tags to form a specific time series record.

[0077] SubEntityGroup is responsible for balancing the space usage of metadata and entity query performance. Each SubEntityGroup contains a set number of entities.

[0078] Time partitions are used to organize and manage data by time range. Each time partition contains a meta-mapped shard table() to further refine data management.

[0079] The data subfolder is the lowest level of the time series database storage engine. It is a compressible unit and supports the undo / redo operations of Createtable / insert / delete / alter operations.

[0080] S2. Design the data structure of the Tag table and the Data table.

[0081] S2.1. The data structure of the Tag table includes a logical layer structure and a physical layer structure, where:

[0082] S2.1.1. The logical layer structure of the Tag table includes:

[0083] SubEntity ID, a unique identifier for identifying a sub-entity group;

[0084] Entity ID, a unique identifier for identifying an entity;

[0085] Primary Tags, which means main tags, are used to quickly access and query data;

[0086] Non-primary Tags, meaning non-primary tags, are used to supplement the descriptive or dimensional information other than Primary Tags, providing an additional screening dimension;

[0087] Quickly access and query data;

[0088] Common Tags: additional tag information for further data description and filtering;

[0089] S2.1.2 The physical layer structure of the Tag table includes:

[0090] Tag.header file, which stores deletion tags and null value bitmaps, and is used to handle data deletion and null values;

[0091] Tag.pt file, which stores primary Tags, subEntity ID and entity ID;

[0092] var file: stores variable-length data by column, with one file for each column.

[0093] S2.2. The data structure of the Data table includes a logical layer structure and a physical layer structure, where:

[0094] S2.2.1. The logical layer structure of the Data table includes:

[0095] EntityHeader, contains datafile id, used to identify the data file;

[0096] EntityItem, which contains entity_id, BlockItem ID, and total row count. It implements the function of mapping Entity to BlockItem. Each EntityItem represents an entity, which is a collection of time series data.

[0097] BlockItem, responsible for recording the data writing location, contains 1000 rows of columndata, which are stored in the .data file;

[0098] DataBlock is the lowest level structure of the data storage engine, which contains nullbitmap and columndata. Nullbitmap is used to mark which data is empty.

[0099] S2.2.2. The physical layer structure of the Data table includes a .meta file and a .data file, wherein: the .meta file contains a MetaFileHeader and multiple EntityItems, and the .data file actually stores data and is organized according to the structure of BlockItem and DataBlock.

[0100] S3. Design the filter push-down elements and filter push-down process of the Tag table and Data table.

[0101] S3.1.1. The filter push-down elements of the Tag table include: data type support, predicate type support, and packaged filter return results.

[0102] S3.1.2, the steps of the Tag table filtering and push-down process include:

[0103] (a1) Traversing all Tag tables: In a time series database, each EntityGroup is usually associated with one or more Tag tables, which are used to store relevant metadata and tag information. The Tag tables in each EntityGroup are checked and processed one by one;

[0104] (a2) Register filter conditions for each Tag table: When executing a query, set the filter requirements for tags in the query conditions in advance, and then register the filter conditions in each Tag table to ensure that only entities that meet the filter conditions are queried and returned;

[0105] (a3) Parsing query conditions: The query execution plan first parses the filter conditions involving the Tag table;

[0106] (a4) Locate Tag data: According to the logical and physical layer structures of the Tag table, locate the relevant Tag.pt file and var file; the Tag.pt file contains primary Tags information, while the var file contains non-primary Tags information;

[0107] (a5) Apply filtering conditions: Apply the filtering conditions of primary Tags in the Tag.pt file, and apply the filtering conditions of non-primary Tags in the var file, and retain the data rows that meet the conditions;

[0108] (a6) Traverse all rows: In each Tag table, traverse all rows, find the rows that meet the filter conditions, and package the rows that do not meet the filter conditions and send them to the next processing stage;

[0109] (a7) Pack each column into a batch: Pack the same columns in multiple rows of data into batches (Batch);

[0110] (a8) Data retrieval: Based on the filtered data, retrieve the corresponding Entity ID and data file location information;

[0111] (a9) Result return: The filtered result set is returned to the upper-level application or further data processing flow. The returned data set only contains entities that meet the query conditions and their related data.

[0112] S3.2.1. The filter push-down elements of the Data table include: data type support, predicate type support, data values ​​of BlockItem located in the Data table, and packaged filter return results.

[0113] S3.2.2, the filtering and push-down process steps of the Data table include:

[0114] (b1) Query parsing: First, the query conditions are parsed, including time range, entity ID, and other relevant dimensions;

[0115] (b2) Data location: Map and locate the corresponding BlockItem and DataBlock according to the information in the EntityItem;

[0116] (b3) Traverse the iterator of each subgroup in turn: access each subgroup through the iterator and return the data;

[0117] (b4) Traverse each BlockItem: further traverse in the located BlockItem to prepare to apply the filter conditions to the data rows;

[0118] (b5) Traverse each row: traverse each row of data in each BlockItem, find the rows that meet the filter conditions, and pack all rows that do not meet the filter conditions;

[0119] (b6) Apply filter conditions: Use nullbitmap and columndata to apply time and preset filter conditions to extract data that meets the conditions, so as to filter out data that meets the query conditions from the located data blocks;

[0120] (b7) Data assembly: Assemble the filtered data blocks DataBlock into a result set and prepare to return it to the requester;

[0121] (b8) Result return: Return the final query result data set to the client or upper-level application.

[0122] S4. Based on the design of steps S1 to S3, by implementing filter push-down at each level of the time series database storage engine, the query execution plan is optimized and the query is executed to obtain the summary result, which specifically includes:

[0123] (c1) Apply filter conditions: First, determine the filter conditions that need to be pushed down. These filter conditions usually come from query requests and can be directly applied to the Tag table and Data table for data screening;

[0124] (c2) Building indexes: Building corresponding index structures at each level of the time series database storage engine;

[0125] (c3) Filtering by using indexes: When executing a query, first use the top-level index to filter out the main directories of the time series table data that do not meet the conditions, and then continue to use the indexes in the next-level EntityGroup and SubEntityGroup for further filtering;

[0126] (c4) Filtering at the data subfolder level: Finally, at the data subfolder level, the final filtering conditions are applied according to the time partition and the specific Data table structure to read the specific data files that meet the query conditions;

[0127] (c5) Optimize query plan: optimize query execution plan based on statistical information and query characteristics;

[0128] (c6) Execute the query and summarize the results: After applying the filtering conditions at each layer, the data that meets the conditions is passed to the upper layer and finally summarized and returned to the user.

[0129] In summary, the method for implementing filter push-down of a time series database storage engine of the present invention can realize effective filter push-down, reduce the amount of data reading and transmission, and thus improve query performance.

[0130] The above specific examples are used to explain the principles and implementation methods of the present invention in detail. These examples are only used to help understand the core technical content of the present invention. Based on the above specific embodiments of the present invention, any improvements and modifications made by technicians in this technical field without departing from the principles of the present invention should fall within the scope of patent protection of the present invention.

Claims

1. A method for implementing filter pushdown in a time series database storage engine, characterized in that: The specific contents include the following: S1. The time series database storage engine is designed to include five levels: the main directory of the time series table data, EntityGroup, SubEntityGroup, time partition, and data subfolder. The main directory of the time series table data is the top-level organization form of data, which contains multiple EntityGroups. EntityGroup is responsible for organizing and managing the data in the time series table. Each EntityGroup is a relatively independent physical unit, including Tag table data, Metrics data, and Data data table. S2. Design the data structure of the Tag table and the Data table; S3. Design the filter push-down elements and filter push-down process of the Tag table and Data table; S4. Based on the design of steps S1 to S3, by implementing filter push-down at each level of the time series database storage engine, the query execution plan is optimized and the query is executed to obtain a summary result.

2. According to claim 1, a method for implementing filtering and push-down of a time series database storage engine is characterized in that: Execute step S1. Each EntityGroup includes Tag table data, Metrics data, and Data data table. Tag table data is used to describe and classify time series data. It consists of a key and a value, which are used to specify the data subcategory under the metric. Metrics data is measurement data, which refers to the quantitative indicators of the monitored system. The Data data table is the actual data storage unit in the time series database, which is responsible for storing the specific information of each data point. Each data point consists of Metric, Timestamp, Value, and multiple Tags, forming a specific time series record. SubEntityGroup is responsible for balancing the space usage of metadata and entity query performance. Each SubEntityGroup contains a set number of entities. Time partitions are used to organize and manage data by time range. Each time partition contains a meta-mappedshard table() to further refine data management. The data subfolder is the lowest layer of the time series database storage engine. It is a compressible unit and supports the undo / redo operations of Create table / insert / delete / alter operations.

3. According to claim 2, a method for implementing filtering and push-down of a time series database storage engine is characterized in that: The data structure of the Tag table includes a logical layer structure and a physical layer structure, wherein: (A1) The logical structure of the Tag table includes: SubEntity ID, a unique identifier for identifying a sub-entity group; Entity ID, a unique identifier for identifying an entity; Primary Tags, which are main tags used to quickly access and query data; Non-primary Tags refer to non-primary tags, which are used to supplement the descriptive or dimensional information other than Primary Tags and provide additional filtering dimensions; Quickly access and query data; Common Tags: refers to additional tag information used for further data description and filtering; (A2) The physical layer structure of the tag table includes: Tag.header file, which stores deletion tags and null value bitmaps, and is used to handle data deletion and null values; Tag.pt file, which stores primary Tags, subEntity ID and entity ID; var file: stores variable-length data by column, with one file for each column.

4. According to claim 3, a method for implementing filtering and push-down of a time series database storage engine is characterized in that: The filter push-down elements of the Tag table include: data type support, predicate type support, and packaged filter return results.

5. According to claim 4, a method for implementing filtering and push-down of a time series database storage engine is characterized in that: The steps of filtering and pushing down the Tag table include: (a1) Traversing all Tag tables: In a time series database, each EntityGroup is associated with one or more Tag tables, which are used to store relevant metadata and tag information. The Tag tables in each EntityGroup are checked and processed one by one; (a2) Register filter conditions for each Tag table: When executing a query, set the filter requirements for tags in the query conditions in advance, and then register the filter conditions in each Tag table to ensure that only entities that meet the filter conditions are queried and returned; (a3) Parsing query conditions: The query execution plan first parses the filter conditions involving the Tag table; (a4) Locate Tag data: According to the logical and physical layer structures of the Tag table, locate the relevant Tag.pt file and var file. The Tag.pt file contains primary Tags information, and the var file contains non-primary Tags information. (a5) Apply filtering conditions: Apply the filtering conditions of primary Tags in the Tag.pt file, and apply the filtering conditions of non-primary Tags in the var file, and retain the data rows that meet the conditions; (a6) Traverse all rows: In each Tag table, traverse all rows, find the rows that meet the filter conditions, and package the rows that do not meet the filter conditions and send them to the next processing stage; (a7) Pack each column into a batch: Pack the same columns in multiple rows of data into batches; (a8) Data retrieval: Based on the filtered data, retrieve the corresponding Entity ID and data file location information; (a9) Result return: The filtered result set is returned to the upper-level application or further data processing flow. The returned data set only contains entities that meet the query conditions and their related data.

6. The method for implementing filter pushdown of a time series database storage engine according to claim 2, characterized in that: The data structure of the Data table includes a logical layer structure and a physical layer structure, wherein: (B1) The logical layer structure of the Data table includes: EntityHeader, contains datafile id, used to identify the data file; EntityItem, which contains entity_id, BlockItem ID, and total row count. It implements the function of mapping Entity to BlockItem. Each EntityItem represents an entity, which is a collection of time series data. BlockItem, responsible for recording the data writing location, contains 1000 rows of columndata, which are stored in the .data file; DataBlock is the lowest level structure of the data storage engine, which contains nullbitmap and columndata. Nullbitmap is used to mark which data is empty. (B2) The physical layer structure of the Data table includes a .meta file and a .data file, wherein: the .meta file contains a MetaFileHeader and multiple EntityItems, and the .data file actually stores data and is organized according to the structure of BlockItem and DataBlock.

7. The method for implementing filtering and push-down of a time series database storage engine according to claim 6, characterized in that: The filter push-down elements of the Data table include: data type support, predicate type support, data values ​​of BlockItem located in the Data table, and packaged filter returned results.

8. The method for implementing filter push-down of a time series database storage engine according to claim 7, characterized in that: The filtering and pushing down process steps of the Data table include: (b1) Query parsing: First, the query conditions are parsed, including time range, entity ID, and other relevant dimensions; (b2) Data location: Map and locate the corresponding BlockItem and DataBlock according to the information in the EntityItem; (b3) Traverse the iterator of each subgroup in turn: access each subgroup through the iterator and return the data; (b4) Traverse each BlockItem: further traverse in the located BlockItem to prepare to apply the filter conditions to the data rows; (b5) Traverse each row: traverse each row of data in each BlockItem, find the rows that meet the filter conditions, and pack all rows that do not meet the filter conditions; (b6) Apply filter conditions: Use nullbitmap and columndata to apply time and preset filter conditions to extract data that meets the conditions, so as to filter out data that meets the query conditions from the located data blocks; (b7) Data assembly: Assemble the filtered data blocks DataBlock into a result set and prepare to return it to the requester; (b8) Result return: Return the final query result data set to the client or upper-level application.

9. The method for implementing filter pushdown of a time series database storage engine according to claim 2, characterized in that: The step S4 specifically includes: (c1) Apply filter conditions: First, determine the filter conditions that need to be pushed down. These filter conditions are derived from the query request and are directly applied to the Tag table and Data table for data screening; (c2) Building indexes: Building corresponding index structures at each level of the time series database storage engine; (c3) Filtering by using indexes: When executing a query, first use the top-level index to filter out the main directories of the time series table data that do not meet the conditions, and then continue to use the indexes in the next-level EntityGroup and SubEntityGroup for further filtering; (c4) Filtering at the data subfolder level: Finally, at the data subfolder level, the final filtering conditions are applied according to the time partition and the specific Data table structure to read the specific data files that meet the query conditions; (c5) Optimize query plan: optimize query execution plan based on statistical information and query characteristics; (c6) Execute the query and summarize the results: After applying the filtering conditions at each layer, the data that meets the conditions is passed to the upper layer and finally summarized and returned to the user.

Images