A method and system for writing to a time series database
By introducing dual-track numbering and joint hashing deduplication mechanisms into time-series databases, constructing local order dependency graphs and performing topological sorting, the problems of replay deadlock and duplicate point identification caused by out-of-order data are solved, improving data consistency and recovery efficiency.
Patent Information
- Application Number
- CN202611280045.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-22
- Publication Date
- 2026-11-10
AI Technical Summary
Existing time-series databases suffer from deadlocks and unidentifiable duplicate points during parallel playback due to loops in the dependency of out-of-order data sequence numbers, affecting data consistency and recovery efficiency.
A dual-track numbering mechanism is used to generate write-ahead log entries containing local timeline sequence numbers for time-series data points. A local order dependency graph is constructed and topological sorting is performed. Combined with a joint hash deduplication mechanism of timeline identifier and logical timestamp, an acyclic scheduling queue and a set of deduplicated data points are generated.
It effectively avoids dependency loops and replay deadlocks caused by the interleaved physical write order of out-of-order data, accurately identifies duplicate data points interleaved across files, and improves data consistency and efficiency during the crash recovery phase of time-series databases.
Smart Images

Figure CN123068403A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of Internet of Things and Industrial Internet technology, and in particular relates to a method and system for writing time-series databases. Background Technology
[0002] With the rapid development of IoT and Industrial Internet technologies, the time-series data generated by massive numbers of devices exhibits characteristics of high-concurrency writes and frequent clock skew. To ensure data persistence and system high availability, time-series databases commonly employ write-ahead logging (LAB) technology, which appends data points to a log file on disk before writing them to the in-memory table. When the system crashes and restarts, the database needs to replay the LAB to restore the in-memory table state before the crash. To shorten system recovery time, current time-series databases have introduced a parallel replay scheduling mechanism during the crash recovery phase, using multi-threaded concurrent processing of log records to accelerate the reconstruction of the in-memory table.
[0003] In traditional techniques, time-series databases typically use a dependency graph based on global sequence numbers for scheduling when performing parallel replay of write-ahead logs. Specifically, during the write phase, the system assigns a globally incrementing sequence number to each log record. During the crash recovery phase, the scheduler extracts the global sequence numbers from the log records as the basis for dependency judgment, constructs a dependency graph according to the physical order of the global sequence numbers, and schedules multiple worker threads to perform parallel replay based on this dependency graph. Furthermore, for duplicate data points generated in the same timeline due to network retransmissions, traditional methods usually rely on the magnitude of the global sequence number to determine the data's order, thus implementing a "last write wins" overwriting and deduplication logic.
[0004] However, current parallel playback scheduling methods suffer from playback deadlocks and unidentifiable duplicates due to dependency loops caused by out-of-order data sequence numbers. In practical applications, data points often arrive with out-of-order timestamps due to network latency or device clock skew, resulting in a discrepancy between the physical write order (i.e., global sequence number order) and logical timestamp order of data points on the same timeline in the write-ahead log. Existing methods rely on global sequence numbers to construct a dependency graph. When processing out-of-order data interleaved across files, data points on different timelines can form dependency loops due to the interleaved waiting of global sequence numbers, leading to topology sorting failures and scheduling deadlocks. Furthermore, the deduplication logic based on global sequence numbers struggles to accurately determine the last written version of the same logical time point in a parallel playback environment, further resulting in unidentifiable duplicates and inconsistencies in the data reconstructed from the memory table. Summary of the Invention
[0005] Therefore, it is necessary to provide a writing method and system for a time-series database that can avoid deadlock during out-of-order data replay and accurately identify duplicate data points, addressing the aforementioned technical problems.
[0006] Firstly, this application provides a method for writing to a time-series database, including:
[0007] S1. Obtain time series data points and assign dual-track numbers to the time series data points to generate write-ahead log entries. The write-ahead log entries include the local sequence number of the timeline.
[0008] S2. Using the local sequence number of the timeline as the sorting key, and based on the timeline identifier, reorganize the dependencies of the write-ahead log entries to generate a local order dependency graph by constructing nodes and connecting directed edges.
[0009] S3. Perform topological sorting on the local order dependency graph to generate an acyclic scheduling queue;
[0010] S4. Based on the replay order allocated by the acyclic scheduling queue, perform joint hashing to deduplicate the pre-written log entries, generate a set of deduplicated data points, and write the set of deduplicated data points into the memory table to generate the reconstructed memory table.
[0011] In one embodiment, S2 includes:
[0012] S21. Based on the timeline identifier, perform global sorting on the write-ahead log entries to obtain an aggregated set of log entries in the same timeline dimension;
[0013] S22. Based on the preset time span, divide the log entry aggregation set into multiple timestamp ordered windows;
[0014] S23. Based on the order of each timestamp ordered window on the corresponding local sequence number of the timeline, construct directed edges between each timestamp ordered window to obtain a set of directed edges;
[0015] S24. Based on the set of directed edges, merge the direct and indirect dependencies between nodes to generate a local order dependency graph.
[0016] In one embodiment, S4 includes:
[0017] S41. Based on the replay range allocated by the acyclic scheduling queue, extract the log entries corresponding to the replay range of the pre-written log entries to obtain the set of log entries to be judged.
[0018] S42. Perform a joint hash calculation on the timeline identifier and logical timestamp of each log entry in the set of log entries to be judged to obtain the joint hash key corresponding to each log entry.
[0019] S43. Using the combined hash key of each log entry as an index, perform a search and match in the pre-built idempotent hash table to obtain the duplicate point matching results;
[0020] S44. Based on the duplicate point matching results, perform deduplication filtering on the duplicate log entries in the log entry set to be judged, and generate a deduplicated data point set.
[0021] In one embodiment, S44 includes:
[0022] S441. Based on the duplicate point matching results, write the log entries in the set of log entries to be judged whose joint hash keys do not match the idempotent hash table into the idempotent hash table to obtain the first set of filtered entries.
[0023] S442. Based on the log entries in the set of log entries to be judged whose joint hash keys hit the idempotent hash table, compare the local timeline sequence number corresponding to the log entry with the local timeline sequence number already stored in the idempotent hash table to obtain the sequence number comparison result.
[0024] S443. Based on the sequence number comparison results, selectively retain the log entries that match the combined hash key, retain the log entries with larger local sequence numbers on the timeline and discard the log entries with smaller local sequence numbers on the timeline, to obtain the second set of filtered entries.
[0025] S444: Merge the first filter item set and the second filter item set to generate a deduplicated data point set.
[0026] In one embodiment, S42 includes:
[0027] S421. Extract the timeline identifier and logical timestamp from each log entry in the set of log entries to be judged to obtain the parameter pair to be calculated;
[0028] S422. Perform a bitwise XOR operation on the timeline identifier and logical timestamp in each pair of parameters to be calculated to obtain the XOR intermediate value.
[0029] S423. Perform hash calculation on the XOR intermediate value to obtain the combined hash key corresponding to each log entry. The expression for the combined hash key is as follows:
[0030] ;
[0031] in, For the combined hash key, For hash calculation function, For timeline identification, For logical timestamps, This is the bitwise XOR operator.
[0032] In one embodiment, S43 includes:
[0033] S431. Using the combined hash key of each log entry as the query key, perform a hash lookup in the idempotent hash table to obtain the lookup status identifier;
[0034] S432. When the search status indicator indicates that the query key does not exist, a no-match result is generated;
[0035] S433. When the search status indicator indicates that the query key exists, extract the existing timeline local sequence number corresponding to the query key from the idempotent hash table to obtain the existing sequence number.
[0036] S434. Compare the existing sequence number with the local timeline sequence number of the current log entry to generate a duplicate point matching result. The duplicate point matching result includes whether there is a match and information about the existing sequence number.
[0037] In one embodiment, S3 includes:
[0038] S31. Calculate the in-degree value of each node in the local order dependency graph to obtain the node in-degree set;
[0039] S32. Select nodes with an in-degree value of zero from the node in-degree set and add them to the scheduling queue to obtain the initial scheduling queue;
[0040] S33. Remove nodes from the initial scheduling queue from the local order dependency graph and update the in-degree values of the adjacent nodes of the removed nodes to obtain the updated set of node in-degrees.
[0041] S34. Repeat S32-S33 until all nodes in the local order dependency graph are added to the scheduling queue, generating an acyclic scheduling queue.
[0042] Secondly, this application also provides a writing system for a time-series database, comprising:
[0043] The dual-track numbering module is used to acquire time-series data points and assign dual-track numbers to them to generate write-ahead log entries. The write-ahead log entries include local timeline sequence numbers.
[0044] The dependency graph construction module is used to use the local sequence number of the timeline as the sorting key and, based on the timeline identifier, to reorganize the dependency relationships of the write-ahead log entries, thereby generating a local order dependency graph by constructing nodes and connecting directed edges.
[0045] The topology sorting module is used to perform topology sorting on the local order dependency graph and generate an acyclic scheduling queue.
[0046] The joint deduplication writing module is used to perform joint hash deduplication on the pre-written log entries according to the replay order allocated by the acyclic scheduling queue, generate a set of deduplicated data points, and write the set of deduplicated data points into the memory table to generate the reconstructed memory table.
[0047] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the writing method of the time-series database as described in the first aspect.
[0048] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the writing method for the time-series database as described in the first aspect.
[0049] The aforementioned writing method and system for time-series databases introduces a dual-track numbering mechanism to generate write-ahead log entries containing local timeline sequence numbers for the acquired time-series data points. The local timeline sequence numbers are used as sorting keys, combined with timeline identifiers, to reorganize the dependencies of the write-ahead log entries. This generates a local order dependency graph by constructing nodes and connecting directed edges, thus cutting off dependency waiting across timeline data. The local order dependency graph is then topologically sorted to generate an acyclic scheduling queue. Based on the replay order allocated by the acyclic scheduling queue, the write-ahead log entries are deduplicated using joint hashing. The deduplicated data point set is written to a memory table to generate a reconstructed memory table. This fundamentally avoids dependency loops and replay deadlocks caused by the interleaved physical write order of out-of-order data, accurately identifies duplicate data points interleaved across files, and ensures the efficiency and data consistency of memory table reconstruction during the time-series database crash recovery phase. Attached Figure Description
[0050] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0051] Figure 1 A flowchart illustrating a method for writing to a time-series database provided by the present invention;
[0052] Figure 2 This is a schematic diagram of the structure of a time-series database writing system provided by the present invention. Detailed Implementation
[0053] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.
[0054] In one embodiment, such as Figure 1As shown, a method for writing to a time-series database is provided. This embodiment illustrates the application of this method to a data writing terminal. It is understood that this method can also be applied to a data writing server, and further to a system including both a data writing terminal and a data writing server, and is implemented through the interaction between the data writing terminal and the data writing server. In this embodiment, the method includes the following steps:
[0055] S1. Obtain time series data points and assign dual-track numbers to the time series data points to generate write-ahead log entries. The write-ahead log entries include the local sequence number of the timeline.
[0056] Optionally, the data writing terminal can continuously receive real-time uploaded business data from IoT devices and industrial internet devices, filter and extract structured data carrying device identifiers, acquisition times, and numerical indicators, and define them as time-series data points. The data writing terminal can perform dual-track numbering processing on the acquired time-series data points. Dual-track numbering is a time-series database-specific numbering mechanism, including two independent numbering dimensions: global sequence number and timeline local sequence number. The global sequence number is a uniformly incrementing number across the entire database, used to mark the order in which write-ahead log entries are physically written to disk. The timeline local sequence number is an independently incrementing number for a single timeline dimension, generated sequentially only for all time-series data points within the same timeline. After completing the dual-track numbering operation, the data writing terminal can encapsulate and integrate the original data content, global sequence number, timeline local sequence number, timeline identifier, and logical timestamp of the time-series data point to generate standardized write-ahead log entries. The write-ahead log entry is the smallest storage unit of the time-series database write-ahead log system and serves as the basic data carrier for subsequent log replay, dependency building, and data deduplication.
[0057] S2. Using the local sequence number of the timeline as the sorting key, and based on the timeline identifier, reorganize the dependencies of the pre-written log entries to generate a local order dependency graph by constructing nodes and connecting directed edges.
[0058] Optionally, the data writing terminal can use the local timeline sequence number as the unique sorting key, and based on the timeline identifier carried by the write-ahead log entries as the classification criterion, to reorganize the dependencies of all write-ahead log entries globally, generating a local order dependency graph. The data writing terminal can first classify and aggregate massive write-ahead log entries according to the timeline identifier, grouping write-ahead log entries belonging to the same device and the same monitoring indicator into independent sets, achieving physical isolation of log data from different timelines. Based on this, the data writing terminal can construct a local order dependency graph according to the graph construction rules of nodes corresponding to individual write-ahead log entries and directed edges corresponding to data dependencies. The data writing terminal can determine the sequential logical relationship between different write-ahead log entries based on the increasing logic of the local timeline sequence number within the same timeline, connect directed edges between log entry nodes with sequential dependencies to represent the temporal dependency constraints between log entries, integrate all node and directed edge resources, and generate a complete local order dependency graph. The local order dependency graph is a dependency topology driven by single-timeline temporal logic, which differs from the global dependency graph constructed by traditional global sequence numbers.
[0059] S3. Perform topological sorting on the local order dependency graph to generate an acyclic scheduling queue.
[0060] Optionally, the data writing terminal can perform topological sorting on the constructed local order dependency graph to solve the dependency loop problem existing in traditional global dependency graphs and output an ordered task queue that can be executed in parallel. The data writing terminal can first traverse all nodes and directed edges within the local order dependency graph, and count the in-degree value of each node. The in-degree value of a node is the total number of directed edges pointing to the current node, representing the number of dependent tasks that need to be completed before the current log entry node can perform a replay operation. The data writing terminal can continuously filter independent nodes with an in-degree value of zero, add such nodes without dependency constraints to the scheduling queue in turn, and remove nodes that have been added to the queue from the local order dependency graph. At the same time, the in-degree values of all adjacent nodes of the node are updated to eliminate the influence of dependency constraints of completed nodes. The data writing terminal can repeatedly execute the operations of node filtering, queue addition, node removal, and in-degree update until all nodes within the local order dependency graph have completed the enqueue processing, generating an acyclic scheduling queue that can be ordered and scheduled in parallel, avoiding the problems of topological sorting failure and scheduling deadlock.
[0061] S4. Based on the replay order allocated by the acyclic scheduling queue, perform joint hashing to deduplicate the pre-written log entries, generate a set of deduplicated data points, and write the set of deduplicated data points into the memory table to generate the reconstructed memory table.
[0062] Optionally, the data writing terminal can follow the log replay order allocated by the acyclic scheduling queue to perform data cleaning and writing operations on pre-written log entries, generating a reconstructed in-memory table. The data writing terminal can first accurately extract all pre-written log entries within the corresponding interval according to the replay data range defined by the acyclic scheduling queue, using them as the raw log data to be processed. The data writing terminal can perform joint hash deduplication processing on all extracted pre-written log entries. This deduplication mechanism uses timeline identifiers and logical timestamps as the core judgment criteria, replacing the traditional single sequence number judgment method, and can accurately identify duplicate time-series data points arriving across files or out of order. The data writing terminal can generate a unique joint hash key through a unified hash calculation rule, and complete the matching, comparison, and filtering operations of duplicate log entries based on a pre-built idempotent hash table, eliminating all duplicate data entries caused by network retransmissions and clock skew, generating a unique and non-redundant deduplicated data point set. The data writing terminal can then batch write the processed deduplicated data point set into the in-memory table of the time-series database, completing the in-memory table data reconstruction in crash recovery scenarios, generating a reconstructed in-memory table with consistent data and complete state.
[0063] In the aforementioned method for writing time-series databases, dual-track numbering is achieved by adding local timeline sequence numbers to time-series data points. Log dependencies are reconstructed based on a single timeline dimension, and a local order dependency graph is constructed. This effectively eliminates the dependency loop defects caused by the traditional global sequence number scheduling mechanism and solves the deadlock problem in the parallel log playback process. At the same time, through a joint hash deduplication mechanism of timeline identifiers and logical timestamps, various out-of-order and cross-file duplicate time-series data points are accurately identified, improving the accuracy of data deduplication during the crash recovery phase. This method is adapted to the complex characteristics of high concurrency, out-of-order arrival, and clock skew in IoT and Industrial Internet scenarios, improving the stability and data consistency of time-series database crash recovery and ensuring the efficient and reliable operation of writing and playing back massive amounts of time-series data.
[0064] In one embodiment, S2 may include:
[0065] S21. Based on the timeline identifier, perform a global sort on the write-ahead log entries to obtain an aggregate set of log entries in the same timeline dimension.
[0066] Optionally, the data writing terminal can read the timeline identifiers carried within all pre-written log entries, use the timeline identifiers as the primary key for classification and sorting, and perform a global sort to obtain an aggregated set of log entries at the same timeline dimension. The data writing terminal can traverse all pre-written log entries, compare the timeline identifier character information contained in each pre-written log entry, group pre-written log entries with the same timeline identifier into the same group, and distinguish and store pre-written log entries with different timeline identifiers. After completing the grouping and aggregation, the data writing terminal can systematically collect all pre-written log entries belonging to the same timeline identifier within each group, and output an aggregated set of log entries at the same timeline dimension. The aggregated set of log entries at the same timeline dimension only contains pre-written log entries belonging to the same monitoring object and the same indicator channel, achieving data isolation between different timelines and avoiding disordered mixing of data across timelines.
[0067] S22. Based on the preset time span, divide the log entry aggregation set into multiple timestamp ordered windows.
[0068] Optionally, the data writing terminal can read an aggregated set of log entries along the same timeline, perform data segmentation using a preset time span as the interval division standard, and obtain multiple timestamp ordered windows. The data writing terminal can extract the logical timestamp carried by each pre-written log entry within the aggregated set of log entries along the same timeline, classify them according to the interval of the logical timestamp, and integrate pre-written log entries whose logical timestamps fall within the same preset time span interval into an independent unit. Each integrated independent unit is defined as a timestamp ordered window, and a timestamp ordered window can contain multiple pre-written log entries whose logical timestamps are within a continuous time period. The data writing terminal can continuously perform interval division and entry aggregation, outputting multiple independent timestamp ordered windows.
[0069] S23. Based on the order of each timestamp ordered window on the corresponding local sequence number of the timeline, construct directed edges between each timestamp ordered window to obtain a set of directed edges.
[0070] Optionally, the data writing terminal can obtain all timestamp-ordered windows and extract the local timeline sequence numbers corresponding to all write-ahead log entries within each timestamp-ordered window. The data writing terminal can compare the order of the local timeline sequence numbers corresponding to different timestamp-ordered windows to determine the temporal dependency constraints between them. When the local timeline sequence number corresponding to one timestamp-ordered window is less than that corresponding to another timestamp-ordered window, the data writing terminal can create a directed edge between the nodes corresponding to the two timestamp-ordered windows, representing the sequential constraint of replay execution. The data writing terminal can traverse all combinations of timestamp-ordered windows with dependency constraints and construct directed edges sequentially. All created directed edges are then aggregated to obtain a set of directed edges.
[0071] S24. Based on the set of directed edges, merge the direct and indirect dependencies between nodes to generate a local order dependency graph.
[0072] Optionally, the data writing terminal can read the complete set of directed edges, merge dependencies based on the window node association information recorded in the directed edge set, and generate a local order dependency graph. The data writing terminal can identify direct and indirect dependencies between nodes. Direct dependencies represent natively created directed edges between two nodes, while indirect dependencies represent temporal constraints formed between two nodes only after passing through an intermediate node. The data writing terminal can integrate and simplify repetitive and redundant dependency associations, retaining valid dependencies with binding force and eliminating redundant direct dependencies that can be covered by indirect dependencies. The data writing terminal can then uniformly organize and encapsulate the merged nodes and valid directed edges to generate a locally order dependency graph with a simplified structure and clear constraints.
[0073] In the above embodiments, pre-written log entries are grouped and collected based on timeline identifiers. Then, timestamp-ordered windows are divided with a preset time span. Directed edges between windows are built based on local timeline sequence numbers, and various dependencies are integrated to form a local order dependency graph. This changes the traditional mode of relying on global sequence numbers to establish dependency constraints, reduces the possibility of circular dependencies formed by cross-timeline out-of-order data, increases the probability of the topology itself being acyclic, helps to improve scheduling blocking during parallel log playback, and improves the operational stability of the log scheduling process during the time-series database crash recovery phase.
[0074] In one embodiment, S4 may include:
[0075] S41. Based on the replay range allocated by the acyclic scheduling queue, extract the log entries corresponding to the replay range of the pre-written log entries to obtain the set of log entries to be judged.
[0076] Optionally, the data writing terminal can read the replay range parameter configured within the acyclic scheduling queue. This replay range is an independent log replay processing interval defined by the acyclic scheduling queue based on the topology sorting result, used to limit the range of pre-written log entries covered by a single deduplication process. The data writing terminal can use the replay range as a data filtering boundary to traverse and match all pre-written log entries in global storage. The data writing terminal can verify the log storage location and sorting number information corresponding to each pre-written log entry one by one, filtering out all pre-written log entries belonging to the current replay range and removing pre-written log entries outside the replay range. The data writing terminal can integrate and aggregate all the filtered and matched pre-written log entries to generate a complete and range-matched set of log entries to be judged.
[0077] S42. Perform a joint hash calculation on the timeline identifier and logical timestamp of each log entry in the set of log entries to be judged to obtain the joint hash key corresponding to each log entry.
[0078] Optionally, the data writing terminal can perform an independent joint hash calculation on each write-ahead log entry within the set of log entries to be judged, generating a unique corresponding joint hash key. The data writing terminal can first extract the timeline identifier and logical timestamp from fixed fields within a single write-ahead log entry to form a parameter pair to be calculated. The data writing terminal can perform a bitwise XOR operation on the timeline identifier and logical timestamp contained in the parameter pair to obtain a fixed-length XOR intermediate value, and then perform a standard hash mapping operation on the XOR intermediate value to output the joint hash key corresponding to the log entry.
[0079] S43. Using the combined hash key of each log entry as an index, perform a search and match in the pre-built idempotent hash table to obtain the duplicate point matching results.
[0080] Optionally, the idempotent hash table is a pre-initialized key-value storage structure for the time-series database, specifically used to store the hash information and corresponding sequence number information of verified log entries, supporting fast hash matching queries. The data writing terminal can use the combined hash key corresponding to each pre-written log entry as an independent query key, calling the pre-built idempotent hash table to perform hash lookup matching. The data writing terminal can generate a corresponding lookup status identifier based on the hash lookup return result, which is divided into two types: key-value not found and key-value found. Based on different lookup status identifiers, the data writing terminal can distinguish and generate corresponding matching content, integrate the lookup results of all pre-written log entries, and generate a complete duplicate point matching result containing the key-value match status and the stored sequence number information.
[0081] S44. Based on the duplicate point matching results, perform deduplication filtering on the duplicate log entries in the log entry set to be judged, and generate a deduplicated data point set.
[0082] Optionally, the data writing terminal can read the duplicate point matching results. Based on the different types of duplicate point matching results, it performs classification filtering and selective retention on all write-ahead log entries within the set of log entries to be judged. The data writing terminal can retain write-ahead log entries whose combined hash keys do not match the idempotent hash table. For write-ahead log entries whose combined hash keys match the idempotent hash table, it performs a comparison between the current timeline local sequence number and the existing timeline local sequence numbers in the idempotent hash table to obtain the sequence number comparison result. Based on the sequence number comparison result, the data writing terminal can retain write-ahead log entries with larger timeline local sequence number values and discard write-ahead log entries with smaller timeline local sequence number values. The data writing terminal can integrate all retained write-ahead log entries to complete the duplicate log entry filtering and generate a set of deduplicated data points without redundancy.
[0083] In the above embodiments, log data to be processed is filtered by defining a dedicated playback range, and a unique identifier key value is generated based on the joint hash operation of timeline identifier and logical timestamp. The data duplication matching and optimal filtering are completed by combining an idempotent hash table. This changes the traditional method of judging duplicate data by relying solely on sequence number. It can adapt to time-series data scenarios with out-of-order and cross-file storage, improve the adaptability of time-series data duplication identification, and alleviate the problem of inaccurate duplication identification in parallel playback scenarios. It also makes the reconstructed data content during the time-series database crash recovery process have higher consistency and adapts to the log playback processing needs of massive high-concurrency time-series data in IoT scenarios.
[0084] In one embodiment, S44 may include:
[0085] S441. Based on the duplicate point matching results, write the log entries in the set of log entries to be judged whose joint hash keys do not match the idempotent hash table into the idempotent hash table to obtain the first set of filtered entries.
[0086] Optionally, the data writing terminal can read the complete duplicate point matching results, and use the key value hit status of the duplicate point matching results as the judgment criterion to classify and filter all pre-written log entries within the set of log entries to be judged, thereby obtaining the first set of filtered entries.
[0087] Specifically, the data writing terminal can individually filter out all write-ahead log entries where the combined hash key does not match the idempotent hash table. These write-ahead log entries represent entirely new time-series data points not recorded in the current time-series database system, eliminating the possibility of data duplication. The data writing terminal can then completely write the combined hash key, local timeline sequence number, timeline identifier, and logical timestamp corresponding to the filtered write-ahead log entries into the idempotent hash table, completing the new data entry for the hash table. The data writing terminal can then uniformly collect and aggregate all write-ahead log entries that have completed hash table writing, generating a first set of filtered entries that is complete and free of duplicate data.
[0088] S442. Based on the log entries in the set of log entries to be judged whose joint hash keys match the idempotent hash table, compare the local timeline sequence number corresponding to the log entry with the local timeline sequence number already stored in the idempotent hash table to obtain the sequence number comparison result.
[0089] Optionally, the data writing terminal can, based on the duplicate point matching results, extract all write-ahead log entries in the set of log entries to be judged whose combined hash keys match the idempotent hash table. These write-ahead log entries are data to be verified that pose a risk of data duplication. The data writing terminal can extract a unique timeline local sequence number from the currently matched write-ahead log entry, and simultaneously extract the existing timeline local sequence number that matches the current combined hash key from the idempotent hash table. The data writing terminal can perform a numerical comparison operation between the timeline local sequence number corresponding to the current write-ahead log entry and the existing timeline local sequence number in the idempotent hash table to quantify the generation order of the two sequence numbers. The data writing terminal can record all the status and result information of this numerical comparison, generating a complete sequence number comparison result that includes the sequence number size relationship.
[0090] S443. Based on the sequence number comparison results, selectively retain log entries that match the combined hash key, retaining log entries with larger local sequence numbers on the timeline and discarding log entries with smaller local sequence numbers on the timeline, thus obtaining the second set of filtered entries.
[0091] Optionally, the data writing terminal can retrieve the complete sequence number comparison results, perform differential filtering and retention on all write-ahead log entries whose combined hash keys match the idempotent hash table, and obtain a second set of filtered entries.
[0092] Specifically, the local timeline sequence number has an increasing attribute within a single timeline; a larger local timeline sequence number corresponds to time-series data that was generated later and has a higher update priority. The data writing terminal can use this attribute rule to selectively filter write-ahead log entries that match duplicates, retaining those with larger local timeline sequence numbers and discarding those with smaller numbers. The data writing terminal can then aggregate and collect all valid write-ahead log entries that have undergone filtering, completing the selective filtering of duplicate data and generating a valid second set of filtered entries.
[0093] S444: Merge the first filter item set and the second filter item set to generate a deduplicated data point set.
[0094] Optionally, the data writing terminal can obtain the independently generated first filter entry set and second filter entry set, and perform lossless merging and integration on the two sets of entry sets to generate a deduplicated data point set.
[0095] Specifically, the first filter set contains newly entered, non-duplicate write-ahead log entries, while the second filter set contains valid write-ahead log entries selected and retained after filtering for duplicate data. There is no data redundancy or conflict between the two sets. The data writing terminal can remove the boundary markers between the sets, merging all write-ahead log entries in the first and second filter sets into a single, unified set. This standardizes the data storage format and arrangement within the set, completing the set merging operation and generating a deduplicated data point set.
[0096] In the above embodiments, the deduplication and filtering of time-series data is completed through a hierarchical process: storing log entries that do not match the hash table, comparing and selecting the best log entries that match the hash table by sequence number, and merging and integrating the two types of valid entry sets. The data update version is determined based on the single time-series dimension features of the local sequence number of the timeline. This can adapt to the deduplication requirements of out-of-order data and cross-file storage data in parallel playback scenarios, alleviate the matching failure problem of traditional global sequence number determination methods, and ensure that the reconstructed data content in the time-series database crash recovery stage has stable consistency. This is suitable for the write recovery scenario of high-concurrency time-series data in the industrial Internet.
[0097] In one embodiment, S42 may include:
[0098] S421. Extract the timeline identifier and logical timestamp from each log entry in the set of log entries to be judged to obtain the parameter pairs to be calculated.
[0099] Optionally, the data writing terminal treats the set of log entries to be judged as an independent data source. It performs independent field extraction on each log entry within the set. The data writing terminal can locate pre-defined fixed storage fields within a single log entry and read the timeline identifier field and logical timestamp field content stored in the log entry. Both types of fields are inherent attribute parameters fixed and encapsulated during the pre-written log entry generation stage, possessing uniqueness and identifiability. The data writing terminal can pair and bind a set of timeline identifiers and logical timestamps obtained in a single extraction to generate a standardized binary parameter combination structure. The data writing terminal can perform field extraction and parameter pairing for each log entry in the set to be judged, batch generating multiple pairs of parameters with uniform structure and independent content to be calculated.
[0100] S422. Perform a bitwise XOR operation on the timeline identifier and logical timestamp in each parameter pair to be calculated to obtain the XOR intermediate value.
[0101] Optionally, bitwise XOR operation is a low-level binary bitwise operation method used in computer software terminals to achieve fusion encoding of two types of parameters with different dimensions. The data writing terminal can perform an independent bitwise XOR operation on each pair of parameters to be calculated. The data writing terminal can uniformly convert the timeline identifier and logical timestamp contained in the parameter pair into a binary data format of equal length, ensuring that the number of bits used in the two types of data matches and the operation format is unified. The data writing terminal can perform XOR comparison operations bit by bit on the two sets of converted binary data, following the operation rule that if the values of the same binary bits are different, the valid bits are output; if the values are the same, the null bits are output. After completing the operation processing of all binary bits, the data writing terminal can generate binary encoded data with a fixed format and a unique correspondence. This binary encoded data is the intermediate XOR value obtained from a single operation.
[0102] S423. Perform a hash calculation on the XOR intermediate value to obtain the joint hash key corresponding to each log entry.
[0103] Optionally, the data writing terminal can perform a standardized hash mapping operation on each XOR intermediate value, generating a unique identifier key value through the irreversible encoding characteristic of hash operations, thus obtaining the combined hash key corresponding to each log entry. The expression for the combined hash key is as follows:
[0104] ;
[0105] in, The combined hash key is a unique encoded identifier for the time sequence attribute of a single log entry; This is a hash calculation function, a general-purpose irreversible hash operation function; This serves as a timeline identifier, used to distinguish different data acquisition channels; This is a logical timestamp used to mark the logical acquisition time of time-series data; This is the bitwise XOR operator. The above operation generates independent key values by merging the timeline identifier and the time identifier. After the data is written to the terminal, the hash operation of all XOR intermediate values is completed, and the combined hash key corresponding to all log entries is output in batches.
[0106] In the above embodiments, a standardized parameter structure is obtained through field extraction, multi-dimensional parameter fusion is achieved based on bitwise XOR operation, and an independent joint hash key is generated through hash operation to complete the unified encoding and identification of time series data. By combining the channel differentiation attribute of timeline identifier and the time series differentiation attribute of logical timestamp, time series data with the same acquisition channel and the same logical time can generate consistent encoding, while time series data with different attributes can generate differentiated encoding. This can improve the standardization of time series data duplication identification, adapt to the matching and identification scenario of out-of-order time series data, and improve the overall stability of data deduplication processing in the time series database log playback stage.
[0107] In one embodiment, S43 may include:
[0108] S431. Using the combined hash key of each log entry as the query key, perform a hash lookup in the idempotent hash table to obtain the lookup status identifier.
[0109] Optionally, the data writing terminal can set the combined hash key corresponding to a single log entry as a fixed query key, and perform hash retrieval on all log entries to be processed in a traversal manner to obtain the search status identifier.
[0110] Specifically, idempotent hash tables possess the storage characteristic of fast key-value addressing and matching, enabling rapid traversal of all stored hash key data based on the input query key. The data writing terminal can call the system's built-in hash retrieval interface, inputting the configured query key into the idempotent hash table's retrieval logic. The idempotent hash table's built-in hash index structure then completes the key-value matching traversal. The data writing terminal can receive the retrieval status information returned by the idempotent hash table's retrieval logic, standardize and encapsulate this information, and generate a lookup status identifier that intuitively indicates the existence of key-value pairs, thus completing the hash lookup for a single log entry.
[0111] S432. When the search status indicator indicates that the query key does not exist, generate a no-match result.
[0112] Optionally, the data writing terminal can perform status identification and judgment on the lookup status identifier, and separately identify the identifier type where the query key does not exist. A query key not existing means that the combined hash key corresponding to the current log entry has not appeared in the historical storage data of the idempotent hash table, indicating that the time-series data corresponding to the current log entry has not retained the same logical data in the system. The data writing terminal can execute matching result generation logic for log entries corresponding to this type of lookup status identifier, encapsulate the status information of key-value mismatch and no historical storage data according to the system's preset matching result format specifications, and independently generate standardized miss matching results to indicate that the current log entry does not have the risk of duplicate data.
[0113] S433. When the lookup status indicator indicates that the query key exists, extract the existing timeline local sequence number corresponding to the query key from the idempotent hash table to obtain the existing sequence number.
[0114] Optionally, the data writing terminal can differentiate the lookup status identifiers and filter out identifier types corresponding to the query key. The existence of the query key indicates that the combined hash key corresponding to the current log entry has been stored within the idempotent hash table, and the system has historical data records with logical attributes consistent with the current log entry. The data writing terminal can locate the data storage unit within the idempotent hash table that is bound to the current query key, and read the sequence number data stored within that data storage unit. This sequence number data is a local timeline sequence number synchronously stored when the historical log entry was written to the hash table. The data writing terminal can format the read sequence number data and output standardized stored sequence numbers.
[0115] S434. Compare the existing sequence number with the local timeline sequence number of the current log entry to generate a duplicate point matching result. The duplicate point matching result includes whether there is a match and information about the existing sequence number.
[0116] Optionally, the data writing terminal can retrieve the already stored sequence number and read the local timeline sequence number carried by the current log entry itself, perform numerical comparison calculations on the two independent sequence number data, and generate duplicate point matching results.
[0117] Specifically, the data writing terminal can record two core types of information: the numerical relationship between two sets of sequence numbers and the key-value match status. It then integrates the data according to the system's preset result encapsulation format. The data writing terminal can encapsulate the key-value match status information and the stored sequence number information within the hash table into a structured data body, generating a complete and standardized duplicate point matching result. This duplicate point matching result can be directly used as the basis for deduplication filtering of log entries.
[0118] In the above embodiments, hash table retrieval is completed based on the joint hash key and standardized search status is output. Differentiated state branches are used to generate miss results and extract existing sequence numbers respectively. The structured duplicate point matching results are constructed by combining sequence number comparison. Based on the hash index structure, the standardized identification of time series data duplication features is realized, which unifies the duplication judgment standard of out-of-order time series data, weakens the judgment interference caused by global sequence number disorder and interleaving, can adapt to the running scenario of parallel log playback, and improves the standardization and stability of data matching and identification in the time series database crash recovery stage.
[0119] In one embodiment, S3 may include:
[0120] S31. Calculate the in-degree value of each node in the local order dependency graph to obtain the node in-degree set.
[0121] Optionally, the data writing terminal treats the local order dependency graph as an independent computational object, performs in-degree value statistical calculations on all nodes contained within the local order dependency graph, and obtains the node in-degree set.
[0122] Specifically, the local order dependency graph consists of several nodes and directed edges. Nodes correspond to log processing units for time-series data, and directed edges correspond to time-series dependency constraints between nodes. The in-degree value of a node is defined as the total number of all valid incoming directed edges received by the current node, representing the total number of pre-dependent tasks that the current node needs to complete before executing scheduling. The data writing terminal can traverse all directed edges associated with each node in the local order dependency graph, count the number of incoming directed edges corresponding to each node, and complete the quantification calculation of the in-degree value of all nodes. The data writing terminal can bind and store each node with its corresponding calculated in-degree value, uniformly organize it into a structured data set, and generate a node in-degree set containing the in-degree parameters of all nodes.
[0123] S32. Select nodes with an in-degree value of zero from the node in-degree set and add them to the scheduling queue to obtain the initial scheduling queue.
[0124] Optionally, the data writing terminal can retrieve the complete set of node in-degrees, use zero in-degree as a filtering condition, and traverse and filter all node parameters recorded in the set of node in-degrees to obtain the initial scheduling queue.
[0125] Specifically, nodes with an in-degree of zero represent nodes that do not have any preceding log processing units and are not subject to the timing constraints of other nodes, thus possessing the basic conditions for priority scheduling and execution. The data writing terminal can filter out all nodes in the node in-degree set that meet the condition of an in-degree value of zero, and store all the filtered nodes into the scheduling queue in sequence according to the system's preset queue inclusion rules. After completing the enqueue inclusion operation of all zero-in-degree nodes, the data writing terminal can generate an ordered queue structure with initial schedulable capabilities and no dependency constraints, resulting in a structurally complete initial scheduling queue.
[0126] S33. Remove nodes from the initial scheduling queue from the local order dependency graph and update the in-degree values of the adjacent nodes of the removed nodes to obtain the updated set of node in-degrees.
[0127] Optionally, the data writing terminal can read all the node information contained in the initial scheduling queue, locate the corresponding node storage location in the local order dependency graph, remove all nodes contained in the initial scheduling queue, and obtain the updated node in-degree set.
[0128] Schematic illustration: After completing the node removal process, the data writing terminal can identify all associated neighboring nodes of the removed node in the local order dependency graph. Adjacent nodes are subsequent nodes that have formed an association with the removed node through outgoing directed edges. The data writing terminal can decrease the in-degree value of each neighboring node to offset the dependency constraints caused by the removed node and eliminate invalid topological constraints. The data writing terminal can then aggregate all node data that have completed in-degree value updates, reorganize the correspondence between nodes and in-degree values, and obtain the updated set of node in-degrees after parameter refresh.
[0129] S34. Repeat S32-S33 until all nodes in the local order dependency graph are added to the scheduling queue, generating an acyclic scheduling queue.
[0130] Optionally, the data writing terminal can iteratively execute the entire processing logic of node filtering and enqueueing, node removal, and in-degree value updating, continuously calling the zero-in-degree node filtering process and the adjacent node parameter update process. In each iteration, the data writing terminal can continuously filter eligible zero-in-degree nodes from the updated node in-degree set to complete enqueueing, synchronously updating the topology and node in-degree parameters of the local order dependency graph, gradually completing the traversal and enqueueing of the remaining nodes within the local order dependency graph. The data writing terminal can continuously judge the node traversal status of the local order dependency graph, terminating the iterative loop after all nodes in the local order dependency graph have completed the operation of being added to the scheduling queue. The data writing terminal can integrate all nodes enqueued throughout the process, generating an ordered queue with a complete topological order, thereby generating a loop-free scheduling queue that can be parallelized.
[0131] In the above embodiments, by statistically analyzing the in-degree parameters of each node in the local order dependency graph, selecting nodes with zero in-degree to construct an initial scheduling queue, and combining the iterative method of node removal and dynamic update of in-degree to complete the topological sorting of all nodes, and completing the log replay scheduling sorting based on the topological structure characteristics of the local order dependency graph, the dependency loop problem that is easy to be generated by the traditional global sequence number scheduling method can be avoided, and the occurrence of topological sorting failure and scheduling blocking can be reduced. Thus, it can adapt to the log replay scenario of out-of-order time-series data, making the multi-threaded parallel scheduling process in the time-series database crash recovery stage more stable and improving the overall execution efficiency of the log replay task.
[0132] The aforementioned writing method and system for time-series databases addresses the technical problems of scheduling deadlocks caused by global sequence number interleaving due to out-of-order data and clock skew during parallel playback of write-ahead logs in existing time-series databases, as well as the failure of identification of duplicate data based on global sequence numbers and inconsistencies in data reconstruction of memory tables. It designs a complete technical system encompassing dual-track numbering, local order dependency graph construction, topology acyclic scheduling, and joint hash deduplication. By adding local timeline sequence numbers to time-series data points, it achieves data ordering constraints at the single timeline dimension. Based on timeline aggregation, time window segmentation, and window dependency reconstruction, it constructs local sequence numbers that are free from global sequence number constraints. The system uses a dependency graph and layer-by-layer iterative topology sorting to generate acyclic, parallel schedulable queues. It also employs a deduplication mechanism that combines timeline identifiers and logical timestamps with hash matching and selective overlay of local timeline sequence numbers to clean log data and rebuild in-memory tables. This effectively avoids dependency loops and replay deadlocks caused by traditional global sequence number scheduling, accurately identifies duplicate time-series data generated by cross-file and out-of-order transmission, and significantly improves the scheduling stability of pre-written log parallel replay and the data consistency of in-memory table reconstruction during the time-series database crash recovery phase. It is suitable for the complex business characteristics of high-concurrency writing, clock skew, and out-of-order arrival of time-series data in IoT and Industrial Internet scenarios.
[0133] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.
[0134] Based on the same inventive concept, this application also provides a system for implementing the above-described method for writing time-series databases. The solution provided by this system is similar to the implementation described in the above method; therefore, the specific limitations of one or more time-series database writing system embodiments provided below can be found in the limitations of the time-series database writing method described above, and will not be repeated here.
[0135] In one exemplary embodiment, such as Figure 2 As shown, a time-series database writing system 10 is provided, comprising:
[0136] The dual-track numbering module 11 can be used to acquire time series data points, assign dual-track numbers to the time series data points, and generate write-ahead log entries. The write-ahead log entries include local sequence numbers of the timeline.
[0137] The dependency graph construction module 12 can be used to use the local sequence number of the timeline as the sorting key value and reorganize the dependency relationship of the write-ahead log entries according to the timeline identifier to generate a local order dependency graph by constructing nodes and connecting directed edges.
[0138] Topology sorting module 13 can be used to perform topology sorting on local order dependency graphs to generate acyclic scheduling queues;
[0139] The joint deduplication writing module 14 can be used to perform joint hash deduplication on the pre-written log entries according to the replay order allocated by the acyclic scheduling queue, generate a set of deduplicated data points, and write the set of deduplicated data points into the memory table to generate the reconstructed memory table.
[0140] In one embodiment, a computer device is provided, including a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the steps of a time-series database writing method as described above.
[0141] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of a time-series database writing method as described above.
[0142] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The components described as separate parts may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this disclosure according to actual needs. Those skilled in the art can understand and implement this without creative effort.
[0143] The above-described embodiments are merely illustrative of several implementation methods of the embodiments of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of the patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the embodiments of this application, and these modifications and improvements all fall within the protection scope of the embodiments of this application.
Claims
1. A method for writing to a time-series database, characterized in that, The method includes: S1. Obtain time series data points and assign dual-track numbers to the time series data points to generate write-ahead log entries, wherein the write-ahead log entries include timeline local sequence numbers; S2. Using the local sequence number of the timeline as the sorting key, and based on the timeline identifier, reorganize the dependency relationship of the pre-written log entries to generate a local order dependency graph by constructing nodes and connecting directed edges. S3. Perform topological sorting on the local order dependency graph to generate an acyclic scheduling queue; S4. According to the playback order allocated by the acyclic scheduling queue, perform joint hash deduplication on the pre-written log entries to generate a deduplicated data point set, and write the deduplicated data point set into the memory table to generate the reconstructed memory table.
2. The method according to claim 1, characterized in that, The S2 includes: S21. Based on the timeline identifier, the pre-written log entries are globally sorted to obtain an aggregated set of log entries in the same timeline dimension; S22. Based on a preset time span, the log entry aggregation set is divided to obtain multiple timestamp ordered windows; S23. Based on the order of each timestamp ordered window on the corresponding local sequence number of the timeline, construct directed edges between each timestamp ordered window to obtain a set of directed edges; S24. Based on the set of directed edges, merge the direct and indirect dependencies between nodes to generate the local order dependency graph.
3. The method according to claim 1, characterized in that, The S4 includes: S41. Based on the replay range allocated by the acyclic scheduling queue, extract the log entries corresponding to the replay range of the pre-written log entries to obtain a set of log entries to be judged. S42. Perform a joint hash calculation on the timeline identifier and logical timestamp of each log entry in the set of log entries to be determined to obtain the joint hash key corresponding to each log entry. S43. Using the combined hash key of each log entry as an index, perform a search and match in the pre-built idempotent hash table to obtain the duplicate point matching results; S44. Based on the duplicate point matching results, perform deduplication filtering on the duplicate log entries in the set of log entries to be judged to generate the deduplicated data point set.
4. The method according to claim 3, characterized in that, S44 includes: S441. Based on the duplicate point matching result, write the log entries in the log entry set to be judged whose joint hash key does not match the idempotent hash table into the idempotent hash table to obtain the first filter entry set; S442. Based on the log entries in the set of log entries to be determined whose joint hash keys match the idempotent hash table, compare the local timeline sequence number corresponding to the log entry with the local timeline sequence number already stored in the idempotent hash table to obtain the sequence number comparison result. S443. Based on the sequence number comparison result, the log entries that match the joint hash key are selectively retained. Log entries with larger local sequence numbers on the timeline are retained, and log entries with smaller local sequence numbers on the timeline are discarded, to obtain a second set of filtered entries. S444. Merge the first set of filtered entries and the second set of filtered entries to generate the set of deduplicated data points.
5. The method according to claim 3, characterized in that, S42 includes: S421. Extract the timeline identifier and the logical timestamp from each log entry in the set of log entries to be determined to obtain the parameter pair to be calculated; S422. Perform a bitwise XOR operation on the timeline identifier and the logical timestamp in each pair of parameters to be calculated to obtain the XOR intermediate value. S423. Perform a hash calculation on the XOR intermediate value to obtain the combined hash key corresponding to each log entry. The expression of the combined hash key is as follows: ; in, For the combined hash key, For hash calculation function, For timeline identification, For the logical timestamp, This is the bitwise XOR operator.
6. The method according to claim 3, characterized in that, S43 includes: S431. Using the combined hash key of each log entry as the query key, perform a hash lookup in the idempotent hash table to obtain the lookup status identifier; S432. When the search status indicator indicates that the query key does not exist, a no-match result is generated; S433. When the search status indicator indicates that the query key exists, extract the stored timeline local sequence number corresponding to the query key from the idempotent hash table to obtain the stored sequence number. S434. Compare the stored sequence number with the timeline local sequence number of the current log entry to generate the duplicate point matching result. The duplicate point matching result includes whether there is a match and the stored sequence number information.
7. The method according to claim 1, characterized in that, The S3 includes: S31. Calculate the in-degree value of each node in the local order dependency graph to obtain the node in-degree set; S32. Select nodes with an in-degree value of zero from the node in-degree set and add them to the scheduling queue to obtain the initial scheduling queue; S33. Remove nodes from the initial scheduling queue from the local order dependency graph and update the in-degree values of the adjacent nodes of the removed nodes to obtain the updated node in-degree set. S34. Repeat S32-S33 until all nodes in the local order dependency graph are added to the scheduling queue, generating an acyclic scheduling queue.
8. A writing system for a time-series database, characterized in that, The system includes: The dual-track numbering module is used to acquire time-series data points and assign dual-track numbers to the time-series data points to generate write-ahead log entries, wherein the write-ahead log entries include timeline local sequence numbers. The dependency graph construction module is used to use the local sequence number of the timeline as the sorting key value, and according to the timeline identifier, to reorganize the dependency relationship of the pre-written log entries, so as to generate a local order dependency graph by constructing nodes and connecting directed edges. The topology sorting module is used to perform topology sorting on the local order dependency graph to generate an acyclic scheduling queue. The joint deduplication writing module is used to perform joint hash deduplication on the pre-written log entries according to the replay order allocated by the acyclic scheduling queue, generate a set of deduplicated data points, and write the set of deduplicated data points into a memory table to generate a reconstructed memory table.
9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 7.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 7.
