Index system and construction method for efficient query of time series data

Abstract

The application relates to the technical field of databases, in particular to an index system for efficient query of time-series data and a construction method thereof. The structure comprises a data linked list for storing data elements, a linked list composed of data nodes, a tail node for maintaining the last data node of the linked list, a data node for a structure body for storing data elements, an index linked list, a multi-level index structure established on the data nodes, an index node for maintaining a head node for pointing to the first index node of the highest layer index, an index node for establishing an index for the data node in the data linked list, and an index tail node, an IndexTail structure maintained by each layer index, for maintaining the last index node of the current index layer, statistical information of the data after the last index node, and a time threshold for creating the next index node. The technical scheme can reduce the query complexity and reduce the hardware resource consumption.

Description

Technical Field

[0001] This invention relates to the field of database technology, and specifically to an indexing system and construction method for efficient querying of time-series data. Background Technology

[0002] With the rapid development of applications such as the Internet of Things (IoT), the Industrial Internet, financial monitoring, and smart manufacturing, a large amount of data generated in chronological order has been produced, known as time-series data. Time-series data has unique generation characteristics, typically being generated continuously at a frequency of seconds or even higher. For example, in the Industrial Internet, various sensors collect real-time operating parameters of equipment, such as temperature, pressure, and rotational speed; this data is recorded in chronological order, forming a time-series data stream. In the field of financial monitoring, data such as stock prices and trading volumes are updated every second, also falling under the category of time-series data.

[0003] The application requirements for time-series data also have distinct characteristics. Due to its continuous generation and massive data volume, write operations are extremely frequent, necessitating efficient write mechanisms to ensure timely data storage. Simultaneously, data retrieval typically involves specific time range requirements, such as querying the operational status of equipment within a certain period or analyzing stock price trends over a specific timeframe. Furthermore, in many scenarios, queries focus more on data aggregation and statistics, such as calculating averages, sums, maximums, and minimums over a certain period, to extract more valuable information for decision analysis.

[0004] However, existing time-series data storage and indexing technologies have many shortcomings: Log-Structured Merge Tree (LSM) is a common data storage structure used by time-series databases such as InfluxDB. Its data writing method involves quickly writing data as a log, and then periodically merging and sorting these logs into time-series blocks. This design offers significant advantages in write performance, meeting the high-frequency write requirements of time-series data. However, it has significant shortcomings in query performance. When performing range queries across time periods, because the data may be distributed across multiple different files or time-series blocks, multiple files need to be read and complex merging operations need to be performed, especially when aggregation statistics are involved, resulting in substantial overhead. For example, querying device operation data for one month and calculating the average may require reading data from multiple files and performing extensive calculations and merging, leading to low query efficiency and excessively long response times.

[0005] Databases like TimescaleDB are based on traditional relational databases, using time as the index field. B+ tree indexes are widely used in traditional relational databases and offer good query performance, while hash indexes can quickly locate data with specific keys. However, when processing time-series data, this structure is inefficient in scenarios involving high-frequency writes and large-span aggregation queries. During high-frequency writes, B+ trees require frequent node splitting and merging operations to maintain tree balance, consuming significant system resources and impacting write performance. Furthermore, in large-span aggregation queries, such as querying the sum of data for a year, the wide data distribution necessitates traversing a large number of index nodes and data records, resulting in slow query speeds and failing to meet the demands of applications with high real-time requirements.

[0006] To reduce storage space and speed up queries, some systems employ sparse indexing schemes with predefined time granularity. This approach reduces the number of indexes by creating them at regular time intervals, thereby lowering storage overhead and improving query efficiency. However, sparse indexes lack flexibility and struggle to support queries at arbitrary time granularities. For example, if the predefined time granularity is hours, when a user needs to query data at the minute level, the sparse index cannot be used for quick location; instead, a large number of data records need to be scanned, leading to a significant drop in query efficiency. While sharding can distribute data across different nodes and improve parallel processing capabilities, it also faces the challenge of high data integration and computational overhead when handling cross-shard aggregation queries.

[0007] Traditional skip lists, as a data structure with good time complexity, are widely used in in-memory databases such as Redis. Skip lists achieve fast lookup, insertion, and deletion operations through a multi-level linked list structure, with a time complexity approaching that of a balanced tree. However, the design of traditional skip lists is primarily geared towards key-value pair lookups and is not optimized for the time dimension and aggregation scenarios of time-series data. Time-series data queries often require range queries and aggregation statistics, which traditional skip lists cannot directly provide, lacking native hierarchical aggregation and statistical capabilities. For example, when querying the maximum value of data within a certain time period, a traditional skip list needs to traverse all data nodes within that range and compare them one by one to obtain the result, resulting in low efficiency. Summary of the Invention

[0008] The purpose of this invention is to propose an indexing system and construction method for efficient querying of time-series data. This technical solution can reduce query complexity and reduce hardware resource consumption.

[0009] To achieve the above objectives, in a first aspect, the present invention proposes an indexing system for efficient querying of time-series data, comprising: A data linked list is used to store data elements. It is a linked list composed of data nodes and maintains a tail node, which points to the last data node in the linked list. A data node is a structure that stores data elements, containing a timestamp, a data value, and a pointer to the next data node. An indexed linked list is a multi-level index structure built on top of data nodes. It consists of index nodes and maintains a head node, which points to the first index node of the highest-level index. An index node is an index created for data nodes in a data linked list. It contains pointers to data nodes, pointers to the next level index node, pointers to the next index node at the same level, and statistical information about data elements between nodes. Statistical information, including the number of data elements between nodes, the sum, the maximum value, and the minimum value; The index tail node is an IndexTail structure maintained by each index level. It is used to maintain the last index node of the current index level, the statistical information of the data after the last index node, and the time threshold for creating the next index node.

[0010] Beneficial effects of the basic solution: This solution constructs a multi-level indexed linked list based on a data linked list. Index nodes are associated with corresponding data nodes and support cross-level and same-level skip access. During querying, it is not necessary to traverse all data nodes. Data intervals can be located directly through index nodes according to the target time range. It can quickly filter indexes and data outside the target time range, greatly reducing invalid access. At the same time, combined with the time granularity aggregation information built into the index nodes, the O(n) query complexity of traditional linear traversal is optimized to a hierarchical O(log n) level, which significantly improves the efficiency of high-frequency time range queries and fixed-point queries of time series data.

[0011] Each level of index node maintains statistical information such as the quantity, sum, maximum value, and minimum value of the corresponding data range, and the index tail node maintains the statistical information of the latest unarchived data in real time. For common aggregation statistical queries of time series data (such as extreme values, summation, and data volume statistics within a specified time range), the statistical results can be directly extracted from the index node without traversing the original data nodes for secondary calculation, which greatly improves the response speed of statistical queries.

[0012] The data linked list uses the tail node to achieve continuous tail insertion of raw data, which meets the core requirement of continuous writing of time-series data in increments over time. The write operation only needs to update the tail node, and the time complexity is O(1). At the same time, each index maintains the last index node, unarchived data statistics and the time threshold for creating a new index node through the IndexTail structure. The new index node can be created dynamically and automatically according to the time threshold, realizing the synchronous update of the index structure and data writing without manual intervention, which ensures both writing efficiency and the validity of the index structure.

[0013] The index linked list adopts a decoupled multi-level design, with index nodes at each level only linked across and within the same level through pointers. The number of index levels and the time thresholds for each level can be flexibly adjusted according to actual business needs (e.g., a coarse-grained top-level index corresponds to a monthly / grade time range, while a fine-grained bottom-level index corresponds to a minute / hour time range), adapting to time-series data query needs with different time granularities. At the same time, the multi-level structure can be flexibly extended to massive time-series data scenarios. When the data volume increases, only index nodes or index levels need to be dynamically added, without reconstructing the overall data and index structure, significantly improving system maintainability and scalability.

[0014] The index nodes in this solution contain only pointers and lightweight statistical information, without complex index metadata. The index linked list and the data linked list are independent of each other but are precisely linked through pointers. Adding or deleting data nodes will not lead to a large-scale reconstruction of the index structure. Only the associated pointers and statistical information need to be updated, which greatly reduces the maintenance cost of the index structure. At the same time, the lightweight index design has low hardware requirements for the deployment environment and can be flexibly deployed in lightweight devices (such as edge computing nodes) and large server clusters, adapting to the time-series data query needs of different application scenarios such as edge and cloud.

[0015] The data linked list stores the timestamps and data values ​​of the original time-series data in its entirety, ensuring the accuracy of the original data. The index nodes point directly to the data nodes through pointers, and the index tail node maintains the statistical information of the unarchived data after the last index node in real time, avoiding index coverage and missing data during the data writing process. This ensures that at any point in time, the query operation can achieve complete and accurate location of the data within the target time range through the combination of index nodes and index tail nodes, balancing query efficiency and data integrity.

[0016] This technical solution specifically addresses the technical problems of traditional time-series data indexing schemes, such as low query efficiency, high aggregation and statistical overhead, conflict between writing and index maintenance, and poor scalability for massive data. In fields where time-series data is generated and queried frequently, such as industrial IoT, new energy monitoring, financial transactions, and equipment status monitoring, it can significantly improve the system's query response speed, reduce resource consumption, and adapt to flexible business expansion needs.

[0017] As a feasible preferred solution, the timestamp of the data node is accurate to the second, the data value is the measurement value at the corresponding time, and a time-series linked list is formed by connecting the data node to the next data node through a pointer.

[0018] As a feasible preferred solution, the statistical information maintained in the index node is specifically as follows: span_count records the number of elements from the previous node to the current node; span_sum is the sum of the corresponding element values; span_max and span_min are the maximum and minimum values ​​of the corresponding elements, respectively.

[0019] As a feasible preferred solution, the index tail node includes: a last_index field, pointing to the last index node of the current layer; an agg_stats field, containing aggregated statistics of span_count, span_sum, span_max, and span_min; and a next_threshold field, recording the time threshold for creating the next index node.

[0020] Secondly, this invention also proposes a method for constructing an index system for efficient querying of time-series data, applied to the aforementioned index system for efficient querying of time-series data, comprising: The system initializes and creates the basic hierarchical structure of the skip list, which may include year, month, day, hour, minute index layers and underlying data linked list layers, forming an empty multi-level time index framework, providing the basic structure for subsequent insertion of data in time order and generation of multi-level aggregated indexes; The data write operation receives time-series data, judges and updates the index tail node. If the data timestamp exceeds the current IndexTail's next_threshold, a new index node is generated and the data is appended to the data list. Query operations include querying statistical values ​​at a specific index granularity, querying statistical values ​​within a range of index granularity, and querying statistical values ​​within any second-level time range; Index hierarchy adjustment: The index hierarchy is dynamically configured based on the actual query conditions and storage resources.

[0021] The beneficial effects of the basic solution are: the construction method clarifies the unified initialization rules for data linked lists, index linked lists, and IndexTail structures at each level, eliminating the need to customize initialization logic for different business scenarios and significantly reducing access costs; at the same time, the standardized initialization process can be directly encapsulated into a general interface to adapt to the deployment requirements of different software and hardware environments, improving the system's portability and deployment efficiency.

[0022] The write operation uses a closed-loop logic of timestamp threshold judgment → triggering the generation of new index nodes → appending data to the linked list to achieve automatic linkage between data writing and index updating: it ensures efficient tail insertion writing of time-series data with time increment (maintaining O(1) write complexity), and avoids problems such as delayed index node creation and missing statistical information caused by manual intervention; at the same time, the IndexTail structure maintains the statistical information of unarchived data in real time, ensuring that the query operation at any time point during the writing process can obtain complete statistical data, which solves the problem of incomplete query data caused by the asynchronous data writing and index updating in traditional indexing schemes.

[0023] The construction method supports core scenarios such as querying statistical values ​​with specified index granularity, querying statistical values ​​with index granularity range, and querying arbitrary time ranges at the second level, adapting to all scenarios' time-series data query needs: For coarse-grained statistical needs (such as daily / hourly extreme values ​​and summation), the pre-defined aggregated statistics in the index nodes can be directly invoked, achieving millisecond-level response. For fine-grained precise queries (such as raw data / statistics within a second-level time range), a recursive algorithm can prioritize the use of statistical values ​​in each layer of index nodes. Boundary intervals not covered by each layer of index nodes can be statistically analyzed by traversing the underlying data nodes, balancing accuracy and efficiency. Compared to the limitations of traditional solutions that either perform a full traversal or only support fixed-granularity queries, this method achieves full coverage and optimal efficiency for both coarse and fine-grained queries, adapting to the query needs of different scenarios such as industrial monitoring, financial risk control, and IoT data collection.

[0024] All query operations are based on the skip-location logic of index nodes, which prioritizes filtering non-target data through the index and only accesses the data list when necessary, greatly reducing the CPU's traversal operations on invalid data; in scenarios with massive time-series data, it can significantly reduce the resource consumption of query operations and improve the system's concurrent query capabilities.

[0025] The construction method supports dynamically adjusting the index level based on actual query conditions (such as high-frequency query granularity) and storage resources (such as memory / disk capacity): if the business mainly uses coarse-grained queries and storage resources are limited, the index level can be reduced to lower index storage overhead; if the business requires high-frequency fine-grained queries and resources are sufficient, the index level can be increased to further improve fine-grained query efficiency. This breaks the bottleneck of the traditional fixed-level index where resource overhead and query efficiency cannot be balanced, and achieves dynamic matching between index structure and business scenarios and hardware resources, maximizing the use of limited resources to improve the efficiency of core query scenarios.

[0026] As a feasible and preferred approach, each layer initializes a head node. The statistics in the index head node are initialized to default values: span_count is set to 0, span_sum is set to 0, span_min is set to the maximum value, and span_max is set to the minimum value. The timestamp and data value in the data head node are set to null. The head nodes of each index layer are connected by downward pointers, forming a vertical path from the higher-level index to the lower-level index. Each index layer head node points to the data head node through a pointer. The horizontal pointers of each layer all point to null. head points to the highest-level index head node, and tail points to the data head node. An IndexTail structure is created for each index layer, where last_index points to the index head node of each layer, span_count in agg_stats is set to 0, span_sum is set to 0, span_min is set to the maximum value, span_max is set to the minimum value, and next_threshold is set to the current time rounded up to the next complete time granularity unit.

[0027] As a feasible and preferred approach, the specific process of data writing includes: Receive time-series data and obtain the timestamp t and the value v; For each level of the index, IndexTail is checked: if t is less than or equal to next_threshold, v is incrementally updated to agg_stats in IndexTail, including incrementing span_count by 1, adding v to span_sum, and updating span_min and span_max; if t is greater than next_threshold, a new index node is generated, a new index node newIndexNode is created, the agg_stats of IndexTail is copied to the aggStats of newIndexNode, the data node pointed to by newIndexNode is set to the last data node of the current underlying data linked list, newIndexNode is attached to the index linked list, last_index of IndexTail is updated to newIndexNode, aggStats is initialized according to the data to be inserted (t, v), and next_threshold is recalculated. The data (t, v) is appended to the linked list, and the tail node is updated.

[0028] As a feasible and preferred solution, the query operation specifically includes the following: Methods for querying statistics at a specific index granularity include: Based on the query time and the characteristics of multi-level indexes, the system traverses from the top-level index head node and locates the index node corresponding to the target granularity by comparing timestamps. It directly returns the sum, min, max, and count statistics maintained by the node; The query time complexity is O(log n), and no access to the underlying data is required; Methods for querying statistical values ​​within a specific index granularity range include: Starting from the top-level index head node, traverse downwards level by level until the target index granularity level is reached; The query range of each layer consists of three parts: the left part is from the query start point to the smallest index node in the current layer within the query range, the middle part is from the smallest index node to the largest index node in the current layer within the query range, and the right part is from the largest index node in the current layer within the query range to the query end point. The left and right parts of the query range use a recursive algorithm to calculate the aggregate value from the next level of index, while the middle part accumulates the statistical values ​​of the current level index node within the query range to calculate the aggregate value. The aggregated values ​​of the three ranges are summed, where span_count and span_sum are added separately, span_min is the minimum value of span_min of the three ranges, and span_max is the maximum value of span_max of the three ranges. Using a recursive algorithm, higher-level indexes can be selected when the query span is large, and lower-level indexes can be selected when the span is small, making full use of the statistical information of index nodes and improving query efficiency; Methods for querying statistical values ​​within any time range of seconds include: Use multi-level indexes to retrieve statistical values ​​for the middle part of a time range; Traverse the underlying linked list to obtain statistical data for the boundary portion; Merge the aggregated statistics of the index with the statistics of the underlying data linked list, where the count and sum values ​​are added separately, the min value is the minimum of the two, and the max value is the maximum of the two; Most queries are transformed into index aggregations, while the boundary parts are read in detail from the underlying data linked lists.

[0029] As a feasible and preferred approach, specific strategies for index-level adjustment include: When resources are sufficient, multi-level indexes of minutes, hours, days, months, and years are established, starting from the underlying data linked list and building indexes at each level in sequence according to the set time granularity; When resources are scarce, create only a single-level or two-level index based on common query conditions to prioritize meeting the main query needs. The index hierarchy supports dynamic expansion and contraction. Attached Figure Description

[0030] Figure 1A diagram illustrating the data stored for each data node.

[0031] Figure 2 This is a diagram illustrating the statistical data of the index node.

[0032] Figure 3 A diagram illustrating the data stored for the index tail node.

[0033] Figure 4 This is a schematic diagram of the time-series data writing process.

[0034] Figure 5 This is a schematic diagram of a time-series data storage structure that stores aggregated data from different dimensions. Detailed Implementation

[0035] To make the technical solution and advantages of this application clearer, the technical solution of the present invention will be further described in detail below with reference to the accompanying drawings. It is understood that the specific embodiments described herein are only some embodiments of the present invention, and are only used to explain this application, not to limit it. It should be noted that the technical features or combinations of technical features described in the following embodiments should not be considered isolated; they can be combined with each other to achieve better technical effects. The same reference numerals appearing in the accompanying drawings of the following embodiments represent the same features or components, and can be applied to different embodiments.

[0036] Furthermore, unless otherwise defined, the technical or scientific terms used in this invention description shall have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains.

[0037] The present invention will now be described in further detail with reference to the accompanying drawings.

[0038] This disclosure provides an indexing system for efficient querying of time-series data, including: A data linked list is used to store data elements. It is a linked list of data nodes and maintains a tail node, which points to the last data node in the linked list.

[0039] A data node is a structure that stores data elements, containing a timestamp, a data value, and a pointer to the next data node. For example, refer to... Figure 1 A data node records the measurement value at a specific moment. The timestamp is accurate to the second, and the data value is the specific numerical value of the measurement. A linked list is formed by connecting the data node to the next data node through a pointer.

[0040] An indexed linked list is a multi-level index structure built on top of data nodes. It consists of index nodes and maintains a head node, which points to the first index node of the highest-level index.

[0041] An index node is an index created for data nodes in a data linked list. It contains pointers to the data node, pointers to the next-level index node, and pointers to the next index node at the same level. It also maintains statistical information such as the number, sum, maximum, and minimum values ​​of data elements between nodes. For example, an index node points to a data node in the data linked list and is simultaneously connected to the next-level and same-level index nodes through pointers. The statistical information it maintains can quickly reflect the overall characteristics of the data within the time period corresponding to that index node.

[0042] Statistical information, the statistical information of the data maintained by the inode, including the number of data elements between nodes (span_count), the sum (span_sum), the maximum value (span_max), and the minimum value (span_min). For example, refer to... Figure 2 `span_count` records the number of elements from the previous node (excluding) to the current node (including), `span_sum` is the sum of these element values, and `span_max` and `span_min` are the maximum and minimum values ​​among these elements, respectively.

[0043] The index tail node is maintained at each index level. It stores the last index node of the current index level, statistical information about the data following the last index node, and the time threshold for creating the next index node. For example, refer to... Figure 3 IndexTail can record information about the last index node in the current layer, and if new data is written, a new index node needs to be created when the time exceeds next_threshold.

[0044] Reference Figure 4 and Figure 5 It provides a time-series data writing process and a time-series data storage structure that stores aggregated data of different dimensions, which can efficiently support aggregate queries and range queries of different dimensions.

[0045] This disclosure provides a method for constructing efficient queries for time-series data, including: Step S100, system initialization, including: The basic hierarchical structure of the skip list is created, which may include year, month, day, hour, minute index layers and an underlying data linked list layer. Each layer initializes a head node. The statistics in the index head node are initialized to default values: span_count is set to 0, span_sum is set to 0, span_min is set to the maximum value, and span_max is set to the minimum value. The timestamp and data value of the data head node are set to null. The head nodes of each index layer are connected by downward pointers, forming a vertical path from the higher-level index to the lower-level index; each index layer head node points to the data head node. The horizontal pointers of each layer all point to null. head points to the highest-level index head node, and tail points to the data head node. An IndexTail structure is created for each index layer, where last_index points to the index head node of each layer, span_count in agg_stats is set to 0, span_sum is set to 0, span_min is set to the maximum value, span_max is set to the minimum value, and next_threshold is set to the current time rounded up to the next complete time granularity unit. For example, setting the next_threshold for minute-level indexes can be set to round up the current time to the next full minute.

[0046] Step S200, data writing operation. Time-series data writing has two characteristics: ordered writing time (data is written in ascending order of timestamps, naturally having an order) and immutable data (historical data is not modified, and there are no update operations). The specific steps are as follows: The system receives data in seconds. Each data point contains a timestamp 't' and a numerical value 'v'. For example, in an industrial monitoring scenario, if the sensor sends temperature data once per second, the system receives data such as (2025-01-01 12:00:01, 25.5).

[0047] Determine and update the index tail node. For each level of the index, IndexTail, determine whether the data timestamp t is less than or equal to the next_threshold of IndexTail.

[0048] If the value is less than the current value, the incremented value v is updated in IndexTail's agg_stats. Specifically, span_count is incremented by 1, span_sum is incremented by the value of v, and if v is less than the current span_min, span_min is updated to v; if v is greater than the current span_max, span_max is updated to v. For example, if the current span_count in IndexTail's agg_stats is 10, span_sum is 200, span_min is 15, span_max is 25, and the new data v is 18, then after the update, span_count will be 11, span_sum will be 218, span_min will remain 15, and span_max will remain 25.

[0049] If the value is greater than next_threshold: trigger the generation of a new index node. Create a new index node newIndexNode, pointing to the last data node of the current underlying linked list. Copy the statistics fields from IndexTail's agg_stats to newIndexNode's aggStats. Mount this node to the next pointer position of the index node pointed to by IndexTail's last_index, completing the linked list connection. Update IndexTail's last_index to newIndexNode, reset IndexTail's aggStats according to v (span_count set to 1, span_sum set to v, span_min set to v, span_max set to v), and recalculate IndexTail's next_threshold (e.g., rounding up the current node's time to the next full minute, hour, etc.). For example, if the current IndexTail's next_threshold is 2025-01-01 12:01:00, and the new data timestamp t is 2025-01-01 12:01:01, then the recalculated next_threshold is 2025-01-01 12:02:00.

[0050] Appending data to a linked list involves appending the data (t, v) to the underlying linked list. This means connecting the new data node to the tail node of the linked list and updating the tail node to point to the new data node. For example, if the original tail node of the linked list pointed to the last data node, after the new data node is connected to that node, the tail node is updated to point to the new data node.

[0051] Step S300, query operation, including: Based on the defined data structure, three types of queries are supported: Type 1: Querying statistics at a specific index granularity; for example, querying statistics for a specific day (such as May 10, 2025), the steps are as follows: To locate the index node, traverse from the top-level index head node based on the query time and the characteristics of multi-level indexes. For example, the top-level index might be at the yearly granularity, then down to the monthly granularity, and then to the daily granularity. Quickly locate the index node corresponding to the target granularity (2025-05-10) by comparing timestamps.

[0052] Returns statistical information, directly returning the sum, min, max, count, and other statistical information maintained by this node. For example, if this node maintains span_sum of 1000, span_min of 10, span_max of 100, and span_count of 100, then these values ​​are returned as the statistical values ​​for that day.

[0053] Performance Description: With a time complexity of O(log n), it requires no access to underlying data, exhibiting high performance and low resource consumption. This is because rapid location via multi-level indexing avoids scanning the entire linked list.

[0054] Type 2: Querying statistical values ​​within a specific index granularity range, for example, querying statistical values ​​from May 10, 2025 to August 7, 2025. The steps are as follows: Locate the starting index node: Start traversing from the top-level index header node to locate the index node containing the starting point (May 10, 2025). For example, find the index node corresponding to the date in the top-level index by comparing timestamps.

[0055] Iterate through the layers and accumulate the aggregated values: Traverse all corresponding index nodes along the "day" level index linked list from May 10, 2025 to May 31, 2025, and accumulate the aggregated values. For example, for each index node, accumulate and update the values ​​of span_sum, span_min, span_max, etc. in its aggStats according to certain rules (such as directly adding span_count, directly adding span_sum, taking the minimum value among all nodes for span_min, and taking the maximum value among all nodes for span_max).

[0056] Traverse the corresponding index nodes along the "month" level index list from 2025-06 to 2025-07, and accumulate the aggregated value. Similarly, accumulate and update the statistical information according to the above rules.

[0057] Traverse all corresponding index nodes along the "sky" level index chain from 2025-08-01 to 2025-08-07, and accumulate the aggregate value.

[0058] Returns final statistics: Returns the accumulated statistics from all the layers as the final result. For example, the final returned sum value is the sum of span_sum from all layers, the min value is the minimum span_min from each layer, and the max value is the maximum span_max from each layer.

[0059] Performance Description: The query time complexity is O(k) (where k is the number of index nodes involved), typically much less than a second-level data traversal. Aggregation is performed only on the index nodes, avoiding the scanning of a large amount of raw data. For larger spans, "monthly" or even "yearly" levels can be selected to further improve query efficiency. This is because aggregation statistics are performed on the index nodes, eliminating the need to access a large number of underlying data nodes.

[0060] Type 3: Querying statistical values ​​within any time range of seconds. For example, to query the statistical values ​​for 2025-05-10 10:22:30 - 2025-08-08 10:24:15, the steps are as follows: Retrieve most statistical values ​​using multi-level indexes: Referring to the method in query type two, use multi-level indexes to aggregate data to obtain statistical values ​​for 2025-05-10 10:23 to 2025-08-08 10:23. For example, first locate the corresponding time range index node, and then perform accumulation and aggregation as in query type two.

[0061] The underlying linked list is traversed to obtain boundary segment statistics. The statistics for the second-level ranges 2025-05-10 10:22:30 - 2025-05-10 10:22:59 and 2025-08-08 10:24:00 - 2025-08-08 10:24:15 are obtained by traversing the underlying linked list. Starting from the corresponding time position in the linked list, each data node is visited sequentially, and the sum, min, max, and count values ​​for that time period are calculated.

[0062] Merging statistical data, specifically data ranging from seconds to minutes, yields the final aggregation result. For example, statistical values ​​obtained through indexes and those obtained through the underlying data linked lists can be merged according to appropriate rules, such as adding count values, adding sum values, taking the minimum of the two for the min value, and taking the maximum of the two for the max value.

[0063] Performance Description: By transforming most queries into index aggregations and only performing fine-grained readings of the underlying data linked lists for a small number of boundary segments, the overall efficiency is far superior to a full data scan, ensuring both query accuracy and performance. This is because index aggregations reduce the amount of data that needs to be processed, accessing the underlying data only in necessary boundary segments.

[0064] Step S400: Adjust the index hierarchy based on the actual query conditions and storage resources: When resources are sufficient, multi-level indexes can be created at the minute, hour, day, month, and year levels. For example, in a financial monitoring system with high requirements for time-series data queries and abundant storage resources, multi-level indexes can be created to meet query needs at different time granularities. Starting from the underlying data linked list, indexes at each level are built sequentially according to the set time granularity. For example, first, a first-level index is built with a minute granularity, then a second-level index is built based on the minute index with an hour granularity, and so on, until the highest-level index with a year granularity is built.

[0065] When resources are limited, create only a single-level or two-level index based on common query conditions. For example, in an industrial internet scenario with limited storage resources but primarily focused on daily and monthly granular queries, only two-level indexes (daily and monthly granularities) can be created. First, build the daily granular index, and then build the monthly granular index based on the daily granular index to meet the main query requirements while saving storage resources.

[0066] The above content is merely an embodiment of the present invention. Commonly known structures and characteristics of the solutions are not described in detail here. Those skilled in the art are aware of all common technical knowledge in the field prior to the application date or priority date, are aware of all existing technologies in that field, and have the ability to apply conventional experimental methods prior to that date. Those skilled in the art can improve and implement this solution based on the guidance provided in this application and their own capabilities. Some typical known structures or methods should not be obstacles for those skilled in the art to implement this application. It should be noted that those skilled in the art can make several modifications and improvements without departing from the structure of the present invention. These should also be considered within the scope of protection of the present invention, and will not affect the effectiveness of the implementation of the present invention or the practicality of the patent. The scope of protection claimed in this application should be determined by the content of its claims, and the specific embodiments described in the specification can be used to interpret the content of the claims.

Claims

1. An indexing system for efficient querying of time-series data, characterized in that, include: A data linked list is used to store data elements. It is a linked list composed of data nodes and maintains a tail node, which points to the last data node in the linked list. A data node is a structure that stores data elements, containing a timestamp, a data value, and a pointer to the next data node. An indexed linked list is a multi-level index structure built on top of data nodes. It consists of index nodes and maintains a head node, which points to the first index node of the highest-level index. An index node is an index created for data nodes in a data linked list. It contains pointers to data nodes, pointers to the next level index node, pointers to the next index node at the same level, and statistical information about data elements between nodes. Statistical information, including the number of data elements between nodes, the sum, the maximum value, and the minimum value; The index tail node is an IndexTail structure maintained by each index level. It is used to maintain the last index node of the current index level, the statistical information of the data after the last index node, and the time threshold for creating the next index node.

2. The indexing system and construction system for efficient querying of time-series data according to claim 1, characterized in that, The timestamps of the data nodes are accurate to the second, and the data values ​​are the measured values ​​at the corresponding times. They are connected to the next data node through pointers to form a time-series linked list.

3. The indexing system and construction system for efficient querying of time-series data according to claim 1, characterized in that, The statistical information maintained in the index node is as follows: span_count records the number of elements from the previous node to the current node; span_sum is the sum of the corresponding element values; span_max and span_min are the maximum and minimum values ​​of the corresponding elements, respectively.

4. The indexing system and construction system for efficient querying of time-series data according to claim 1, characterized in that, The index tail node includes: the last_index field, which points to the last index node in the current layer; the agg_stats field, which contains aggregated statistics of span_count, span_sum, span_max, and span_min; and the next_threshold field, which records the time threshold for creating the next index node.

5. A method for constructing an index system for efficient querying of time-series data, characterized in that, An indexing system for efficient time-series data querying as described in any one of claims 1-4, comprising: The system initializes and creates the basic hierarchical structure of the skip list, which may include year, month, day, hour, minute index layers and underlying data linked list layers, forming an empty multi-level time index framework, providing the basic structure for subsequent insertion of data in time order and generation of multi-level aggregated indexes; The data write operation receives time-series data, judges and updates the index tail node. If the data timestamp exceeds the current IndexTail's next_threshold, a new index node is generated and the data is appended to the data list. Query operations include querying statistical values ​​at a specific index granularity, querying statistical values ​​within a range of index granularity, and querying statistical values ​​within any second-level time range; Index hierarchy adjustment: The index hierarchy is dynamically configured based on the actual query conditions and storage resources.

6. The method for constructing an index system for efficient querying of time-series data according to claim 5, characterized in that, The specific process of system initialization includes: Each level initializes a head node. The statistics in the index head node are initialized to default values: span_count is set to 0, span_sum is set to 0, span_min is set to the maximum value, and span_max is set to the minimum value. The timestamp and data value in the data head node are set to null. The head nodes of each index level are connected by downward pointers, forming a vertical path from the higher-level index to the lower-level index. Each index level head node points to the data head node through a pointer. The horizontal pointers of each level all point to null. head points to the highest-level index head node, and tail points to the data head node. An IndexTail structure is created for each index level, where last_index points to the index head node of each level, span_count in agg_stats is set to 0, span_sum is set to 0, span_min is set to the maximum value, span_max is set to the minimum value, and next_threshold is set to the current time rounded up to the next complete time granularity unit.

7. The method for constructing an index system for efficient querying of time-series data according to claim 5, characterized in that, The specific process of data writing includes: Receive time-series data and obtain the timestamp t and the value v; For each level of the index, IndexTail is checked: if t is less than or equal to next_threshold, v is incrementally updated to agg_stats in IndexTail, including incrementing span_count by 1, adding v to span_sum, and updating span_min and span_max; if t is greater than next_threshold, a new index node is generated, a new index node newIndexNode is created, the agg_stats of IndexTail is copied to the aggStats of newIndexNode, the data node pointed to by newIndexNode is set to the last data node of the current underlying data linked list, newIndexNode is attached to the index linked list, last_index of IndexTail is updated to newIndexNode, aggStats is initialized according to the data to be inserted (t, v), and next_threshold is recalculated. The data (t, v) is appended to the linked list, and the tail node is updated.

8. The method for constructing an index system for efficient querying of time-series data according to claim 5, characterized in that, The query operation specifically includes the following: Methods for querying statistics at a specific index granularity include: Based on the query time and the characteristics of multi-level indexes, the system traverses from the top-level index head node and locates the index node corresponding to the target granularity by comparing timestamps. It directly returns the sum, min, max, and count statistics maintained by the node; The query time complexity is O(log n), and no access to the underlying data is required; Methods for querying statistical values ​​within a specific index granularity range include: Starting from the top-level index head node, traverse downwards level by level until the target index granularity level is reached; The query range of each layer consists of three parts: the left part is from the query start point to the smallest index node in the current layer within the query range, the middle part is from the smallest index node to the largest index node in the current layer within the query range, and the right part is from the largest index node in the current layer within the query range to the query end point. The left and right parts of the query range use a recursive algorithm to calculate the aggregate value from the next level of index, while the middle part accumulates the statistical values ​​of the current level index node within the query range to calculate the aggregate value. The aggregated values ​​of the three ranges are summed, where span_count and span_sum are added separately, span_min is the minimum value of span_min of the three ranges, and span_max is the maximum value of span_max of the three ranges. Using a recursive algorithm, higher-level indexes can be selected when the query span is large, and lower-level indexes can be selected when the span is small, making full use of the statistical information of index nodes and improving query efficiency; Methods for querying statistical values ​​within any time range of seconds include: Use multi-level indexes to retrieve statistical values ​​for the middle part of a time range; Traverse the underlying linked list to obtain statistical data for the boundary portion; Merge the aggregated statistics of the index with the statistics of the underlying data linked list, where the count and sum values ​​are added separately, the min value is the minimum of the two, and the max value is the maximum of the two; Most queries are transformed into index aggregations, while the boundary parts are read in detail from the underlying data linked lists.

9. A method for constructing an index system for efficient querying of time-series data according to claim 6, characterized in that, Specific strategies for index level adjustment include: When resources are sufficient, multi-level indexes of minutes, hours, days, months, and years are established, starting from the underlying data linked list and building indexes at each level in sequence according to the set time granularity; When resources are scarce, create only a single-level or two-level index based on common query conditions to prioritize meeting the main query needs. The index hierarchy supports dynamic expansion and contraction.

Images