Data processing method, device and equipment based on lock-free time sequence database and storage medium

Abstract

The invention discloses a data processing method, device and equipment based on a lock-free time sequence database and a storage medium, and belongs to the technical field of computers. The method comprises the steps that the request type of an access request is received and judged; for the data write-in request, obtaining target write-in data and a write-in timestamp, determining a target time interval index based on the write-in timestamp, and writing the target write-in data into the lock-free time sequence database by using CAS atomic operation; for the data query request, obtaining target query time, determining a target time interval index based on the target query time, and performing data query operation in the lock-free time sequence database by using CAS atomic operation to obtain a data query result; and for a data updating request, acquiring target updating time, determining a target time interval index based on the target updating time, performing data compression processing in the lock-free time sequence database by utilizing CAS atomic operation, and updating the lock-free time sequence database. The problem of low data processing efficiency can be solved.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a data processing method, apparatus, device, and storage medium based on a lock-free time-series database. Background Technology

[0002] In the field of time-series databases, indexing technology is the core of ensuring efficient reading, writing, and querying of massive amounts of time-series data. Traditional time-series databases organize data in the form of "tags + timestamps." Although they support partitioned storage by time dimension, their index design focuses on tag matching. Data within a partition is distributed by tag hashing. Concurrency processing uses software-level locking mechanisms such as partition locks and batch locks. Writing requires locking the entire partition, and data updates are simulated by "deleting old data + inserting new data." In distributed scenarios, synchronization is guaranteed by distributed locks.

[0003] Therefore, traditional time-series databases need to traverse the data within a partition when querying, and cannot quickly locate the target data block by leveraging the continuity of time; at the same time, they rely on software-level locking mechanisms, lack hardware-level atomic operation support, and suffer from thread blocking and context switching overhead during writing, resulting in intermediate states in data updates, which affects the efficiency of data processing. Summary of the Invention

[0004] This invention provides a data processing method, apparatus, device, and storage medium based on a lock-free time-series database, which can solve the problem of low data processing efficiency in the prior art.

[0005] To address the aforementioned technical problems, this invention provides a data processing method based on a lock-free time-series database, comprising:

[0006] Receive an access request from a lock-free time-series database and determine the request type of the access request;

[0007] When the request type is a data write request, the target write data and the corresponding write timestamp are obtained in the access request, and the target time interval index is determined based on the write timestamp. Then, the target write data is written to the lock-free time-series database based on the target time interval index using CAS atomic operation.

[0008] When the request type is a data query request, the target query time is obtained from the access request, and the target time interval index is determined based on the target query time. Then, the CAS atomic operation is used to perform a data query operation in the lock-free time-series database based on the target time interval index to obtain the data query result.

[0009] When the request type is a data update request, the target update time is obtained from the access request, and the target time interval index is determined based on the target update time. Then, CAS atomic operations are used to perform data compression processing on the lock-free time-series database based on the target time interval index to update the lock-free time-series database.

[0010] As a preferred embodiment, the step of determining the target time interval index based on the write timestamp, and then using CAS atomic operations to write the target write data to the lock-free time-series database based on the target time interval index, includes:

[0011] The target time interval index is determined based on the written timestamp;

[0012] Determine whether the target time interval index exists in the lock-free time-series database;

[0013] When the target time interval index exists in the lock-free time series database, a hot data block is allocated for the target write data, the target write data is written to the hot data block, and the block element information of the hot data block is updated.

[0014] When the target time interval index does not exist in the lock-free time series database, the target time interval index is created in the lock-free time series database using CAS atomic operations, a hot data block is allocated for the target write data, the target write data is written to the hot data block, and the block cell information of the hot data block is updated.

[0015] As a preferred embodiment, the step of creating the target time interval index in the lock-free time-series database using CAS atomic operations further includes:

[0016] If creating the target time interval index in the lock-free time series database fails, the temporary index in the lock-free time series database is released, and the target time interval index is recreated.

[0017] As a preferred embodiment, the step of determining a target time interval index based on the target query time, and then performing a data query operation in the lock-free time-series database based on the target time interval index to obtain the data query results includes:

[0018] The validity of the target query time is validated.

[0019] Once the target query time passes the validity check, the cached data corresponding to the target query time is queried in the lock-free time-series database to determine whether the cached data is valid.

[0020] When the cached data is valid, several target time interval indices are matched in the time interval index of the lock-free time-series database according to the target query time;

[0021] The data blocks corresponding to each target time interval index are determined as the query data blocks;

[0022] Generate data query results based on the data corresponding to all query data blocks.

[0023] As a preferred embodiment, generating data query results based on the data corresponding to all query data blocks includes:

[0024] Determine the data type of each queried data block;

[0025] When the data block type of the query data block is cold data, the query data block is decompressed to obtain compressed data;

[0026] When the data block type of the queried data block is hot data, the hot data corresponding to the queried data block is obtained;

[0027] Based on the compressed data and the cold data, generate data query results.

[0028] As a preferred embodiment, the step of determining a target time interval index based on the target update time, and then performing data compression processing on the lock-free time series database based on the target time interval index, and updating the lock-free time series database, includes:

[0029] Determine the target time interval index based on the target update time;

[0030] The data blocks corresponding to each target time interval index are identified as the data blocks to be updated.

[0031] Determine the data type of each data block to be updated;

[0032] When the data block type of the queried data block is hot data, determine whether the hot data corresponding to the queried data block is valid;

[0033] If the hot data corresponding to the query data block is valid, the hot data corresponding to the query data block is compressed to form compressed data, and the compressed data is stored in the cold data block to update the lock-free time-series database.

[0034] As a preferred embodiment, when the data block type of the queried data block is hot data, after determining whether the hot data corresponding to the queried data block is valid, the method further includes:

[0035] If the hot data corresponding to the query data block is invalid, then delete the invalid hot data and update the query data block.

[0036] Accordingly, the present invention provides a data processing device based on a lock-free time-series database, comprising: a request type determination module, a data writing module, a data query module, and a data update module;

[0037] The request type determination module is used to receive access requests from the lock-free time-series database and determine the request type of the access request.

[0038] The data writing module is used to obtain the target write data and the corresponding write timestamp from the access request when the request type is a data write request, and determine the target time interval index based on the write timestamp, and then use CAS atomic operation to write the target write data to the lock-free time-series database based on the target time interval index;

[0039] The data query module is used to obtain the target query time in the access request when the request type is a data query request, determine the target time interval index based on the target query time, and then use CAS atomic operation to perform data query operation in the lock-free time-series database based on the target time interval index to obtain the data query result.

[0040] The data update module is used to obtain the target update time from the access request when the request type is a data update request, determine the target time interval index based on the target update time, and then use CAS atomic operation to perform data compression processing on the lock-free time series database based on the target time interval index to update the lock-free time series database.

[0041] The present invention also provides a terminal device, comprising: a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, it implements the steps of the data processing method based on a lock-free timing database as described in the present invention.

[0042] The present invention also provides a computer-readable storage medium item, comprising: a stored computer program, which, when the computer program is running, controls the device where the computer-readable storage medium is located to perform the steps of the data processing method based on a lock-free time-series database of the present invention.

[0043] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0044] This invention provides a data processing method based on a lock-free time-series database. It receives and determines the request type of access requests. For data write requests, it obtains the target write data and write timestamp, determines the target time interval index based on the write timestamp, and uses CAS atomic operations to write the target write data to the lock-free time-series database. For data query requests, it obtains the target query time, determines the target time interval index based on the target query time, and uses CAS atomic operations to perform data query operations in the lock-free time-series database to obtain the data query result. For data update requests, it obtains the target update time, determines the target time interval index based on the target update time, and uses CAS atomic operations to perform data compression processing in the lock-free time-series database to update the lock-free time-series database. This invention utilizes the continuity of time range in time-series data to construct a time interval index in the lock-free time-series database. Therefore, when executing a database access request, the target data block can be quickly located through the time interval index to execute the corresponding data processing operation. During data processing, CAS atomic operations are used. Based on the hardware-level lock-free concurrency support of CAS atomic operations, no intermediate data states are required, effectively improving the efficiency of data processing. Attached Figure Description

[0045] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0046] Figure 1 A flowchart illustrating an embodiment of the data processing method based on a lock-free time-series database provided by the present invention;

[0047] Figure 2 This is a schematic diagram of an embodiment of the data processing device based on a lock-free time-series database provided by the present invention. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0049] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the application; the terms “comprising” and “having”, and any variations thereof, in the specification, claims, and foregoing description of the drawings are intended to cover non-exclusive inclusion.

[0050] In the description of the embodiments of this application, technical terms such as "first" and "second" are used only to distinguish different objects and should not be construed as indicating or implying relative importance or implicitly specifying the number, specific order, or primary and secondary relationship of the indicated technical features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly defined.

[0051] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0052] In the description of the embodiments in this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship.

[0053] In the description of the embodiments of this application, the term "multiple" refers to two or more (including two), similarly, "multiple sets" refers to two or more (including two sets), and "multiple pieces" refers to two or more (including two pieces).

[0054] In the description of the embodiments of this application, unless otherwise expressly specified and limited, technical terms such as "installation," "connection," "joining," and "fixing" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. For those skilled in the art, the specific meaning of the above terms in the embodiments of this application can be understood according to the specific circumstances.

[0055] See Figure 1To address the problem of low data processing efficiency in existing technologies, an embodiment of the present invention provides a data processing method based on a lock-free time-series database. This method includes steps 101 to 104, each of which is detailed below:

[0056] Step 101: Receive an access request from the lock-free time-series database and determine the request type of the access request.

[0057] In this embodiment of the invention, the lock-free time-series database is used to store power grid time-series data. The basic storage unit in the lock-free time-series database includes a timestamp (millisecond-level time-series timestamp, uniquely identifying a single data entry), monitored data values ​​(such as power grid current and voltage), and data status (0 = normal, non-0 = abnormal, 0xFF = deleted). To ensure the consistency of time sharding logic in the lock-free time-series database, the following basic granularity needs to be defined: the total number of milliseconds per day is 24×60×60×1000, the number of milliseconds per 10 minutes is (10×60×1000), and the total number of 10-minute intervals per day is 24h×6.

[0058] In this embodiment of the invention, the index of the lock-free time-series database is designed as a daily 10-minute interval index, used for the data block offset of all 10-minute intervals within the day. The lock-free time-series database can realize data processing operations such as data writing, data querying, and data updating. These data processing operations can be executed using CAS atomic operations. CAS (Compare-and-Swap) atomic operations are comparison and swap atomic operations, which are hardware-supported lock-free atomic instructions. Through the atomic execution of "compare-swap", the integrity of variable updates in multi-threaded / distributed scenarios is guaranteed (no intermediate state), and concurrency competition problems can be solved without traditional lock mechanisms. Its core logic is: input memory address (store target variable), expected value (the value that the thread believes the variable should currently have), new value (the target value to be updated). The hardware atomically completes the entire process of "reading the current value of the variable → comparing it with the expected value → updating to the new value if they are consistent, otherwise not updating". The operation result directly returns success or failure, without interruption or lock blocking throughout the entire process, which is the core technology for achieving efficient concurrency control.

[0059] Step 102: When the request type is a data write request, obtain the target write data and the corresponding write timestamp from the access request, determine the target time interval index based on the write timestamp, and then use CAS atomic operation to write the target write data to the lock-free time-series database based on the target time interval index.

[0060] As a preferred embodiment, a target time interval index is determined based on the write timestamp, and then the target write data is written to the lock-free time-series database using a CAS atomic operation based on the target time interval index, including:

[0061] The target time interval index is determined based on the written timestamp;

[0062] Determine whether the target time interval index exists in the lock-free time-series database;

[0063] When the target time interval index exists in the lock-free time series database, a hot data block is allocated for the target write data, the target write data is written to the hot data block, and the block element information of the hot data block is updated.

[0064] When the target time interval index does not exist in the lock-free time series database, the target time interval index is created in the lock-free time series database using CAS atomic operations, a hot data block is allocated for the target write data, the target write data is written to the hot data block, and the block cell information of the hot data block is updated.

[0065] As a preferred embodiment, the step of creating the target time interval index in the lock-free time-series database using CAS atomic operations further includes:

[0066] If creating the target time interval index in the lock-free time series database fails, the temporary index in the lock-free time series database is released, and the target time interval index is recreated.

[0067] In this embodiment of the invention, when a data write request to the lock-free time-series database is received, the target write data and its corresponding timestamp (i.e., the write timestamp) are first obtained from the access request. The target write data is then validated. If valid, the write timestamp is checked against the preset global maximum timestamp of the lock-free time-series database. If greater, the target write data cannot be written to the database, and a data write error signal is generated. If less than or equal to the preset global maximum timestamp, the target interval index of the write timestamp is further located. After determining the target interval index, it is checked whether the index exists in the lock-free time-series database. If it exists, subsequent data write operations are performed directly. If not, a new time interval index is created in the lock-free time-series database as the target interval index. During the creation of the target interval index, CAS atomic operations can be used for expansion, and the success of the expansion is checked. If unsuccessful, the memory of the temporary index is released, and the target time interval index is recreated. After successful creation of the target interval index, subsequent data write operations are performed.

[0068] In this embodiment of the invention, the data write operation can be completed using CAS atomic operations. First, a hot data block is allocated for the target data to be written, and the block element information and data of the hot data block are initialized. The interval index is updated using CAS atomic operations, and the next pointer is updated to point to the new hot data block. When the CAS atomic operation is determined to be successful, the reference count of the old data block is decremented by 1, and the global maximum timestamp of the lock-free time-series database is updated. If the CAS atomic operation is determined to be unsuccessful, the CAS atomic operation is retried within a preset number of retries. When the retry is successful, the latest old offset is obtained, and the next pointer is updated to point to the new data block. If all retries within the preset number of retries fail, the system sleeps for 1 millisecond. The next pointer needs to be updated before each retry to ensure the continuity of the linked list. When the block reference count of the old data block is 0, the data block is automatically cleaned up to improve the utilization of storage space in the lock-free time-series database.

[0069] Step 103: When the request type is a data query request, obtain the target query time from the access request, determine the target time interval index based on the target query time, and then use CAS atomic operation to perform data query operation in the lock-free time-series database based on the target time interval index to obtain the data query result.

[0070] As a preferred embodiment, the step of determining a target time interval index based on the target query time, and then performing a data query operation in the lock-free time-series database based on the target time interval index to obtain the data query result includes:

[0071] The validity of the target query time is validated.

[0072] Once the target query time passes the validity check, the cached data corresponding to the target query time is queried in the lock-free time-series database to determine whether the cached data is valid.

[0073] When the cached data is valid, several target time interval indices are matched in the time interval index of the lock-free time-series database according to the target query time;

[0074] The data blocks corresponding to each target time interval index are determined as the query data blocks;

[0075] Generate data query results based on the data corresponding to all query data blocks.

[0076] As a preferred embodiment, the step of generating data query results based on the data corresponding to all query data blocks includes:

[0077] Determine the data type of each queried data block;

[0078] When the data block type of the query data block is cold data, the query data block is decompressed to obtain compressed data;

[0079] When the data block type of the queried data block is hot data, the hot data corresponding to the queried data block is obtained;

[0080] Based on the compressed data and the cold data, generate data query results.

[0081] In this embodiment of the invention, when a data query request is received from a lock-free time-series database, the target query time is first obtained and its validity is checked. Specifically, the start and end times of the target query time are compared. If the start time is greater than the end time, the target query time is deemed invalid, and a data query anomaly signal is generated. If the target query time is deemed valid, the cached data in the lock-free time-series database is checked using a composite key to determine whether the cached data corresponding to the target query time is valid, i.e., whether cached data exists and whether it has expired. If cached data exists and has not expired, multiple corresponding target time interval indices are determined based on the target query time. The data blocks of each target time interval index are traversed and identified as query data blocks, and the reference count of these query data blocks is incremented by 1. The data type of the query data block is further determined. If it is determined to be hot data, the MOE algorithm is used to obtain the corresponding hot data; if it is determined to be cold data, the cold data is decompressed to obtain compressed data. The obtained hot data and compressed data are summarized to generate the data query result.

[0082] In this embodiment of the invention, hot data acquisition involves traversing the hot data block chain, filtering each hot data block, and merging the block chain data using the MoE algorithm to form an ordered result. For cold data, a decompression operation is performed to form compressed data, and the data block reference count is decremented by 1. The hot data and compressed data corresponding to all target time interval indices are sorted, and the cache of the lock-free time-series database is updated.

[0083] The data query operation of this invention achieves efficient range queries, reduces the overhead of repeated queries through a caching mechanism, ensures data security during queries through reference counting, and optimizes query performance by differentiating between hot and cold data processing methods. Specifically, the MoE algorithm employed ensures the orderly merging of hot data block chains, while cold data is efficiently queried through decompression and filtering.

[0084] Step 104: When the request type is a data update request, obtain the target update time from the access request, determine the target time interval index based on the target update time, and then use CAS atomic operation to perform data compression processing on the lock-free time series database based on the target time interval index to update the lock-free time series database.

[0085] As a preferred embodiment, the step of determining a target time interval index based on the target update time, and then performing data compression processing on the lock-free time series database based on the target time interval index to update the lock-free time series database includes:

[0086] Determine the target time interval index based on the target update time;

[0087] The data blocks corresponding to each target time interval index are identified as the data blocks to be updated.

[0088] Determine the data type of each data block to be updated;

[0089] When the data block type of the queried data block is hot data, determine whether the hot data corresponding to the queried data block is valid;

[0090] If the hot data corresponding to the query data block is valid, the hot data corresponding to the query data block is compressed to form compressed data, and the compressed data is stored in the cold data block to update the lock-free time-series database.

[0091] As a preferred embodiment, when the data block type of the queried data block is hot data, after determining whether the hot data corresponding to the queried data block is valid, the method further includes:

[0092] If the hot data corresponding to the query data block is invalid, then delete the invalid hot data and update the query data block.

[0093] In this embodiment of the invention, when a data update request for a lock-free time-series database is received, the target update time is first obtained, and the corresponding target time interval index is determined based on the target update time. The existence of the target time interval index is checked in the lock-free time-series database, and the data blocks corresponding to each target time interval index are identified as data blocks to be updated. The data type of each data block to be updated is checked to see if it is hot data. If it is determined to be hot data, the reference count of these data blocks is incremented by 1 to ensure that hot data blocks are not accidentally deleted during the data update process. The hot data block chain is traversed, filtering out data marked for deletion. Then, the validity of the hot data in the data blocks to be updated is further checked. If invalid, the invalid data in the data blocks to be updated is deleted, the reference count of the data block is decremented by 1, and the query data block is updated. If the hot data in the data blocks to be updated is valid, the hot data corresponding to the query data block is compressed using CAS atomic operations to form compressed data. A cold data block is created to store the compressed data, and the target time interval index is updated using CAS atomic operations, that is, the corresponding target time interval index is updated from a hot data block to a cold data block. The system checks whether the CAS atomic operation to update the target time interval index is successful. If successful, it reduces the reference count of the hot data block; if it fails, it reduces the reference count of the newly created cold data block and the reference count of the corresponding hot data block, thereby achieving lock-free time-series database updates.

[0094] The data update operation of this invention realizes a lock-free operation to compress hot data into cold data, ensures data security through reference counting, guarantees atomic updates through CAS, and handles various boundary cases such as empty data and compression failure.

[0095] Implementing the above embodiments has the following effects:

[0096] This invention provides a data processing method based on a lock-free time-series database. It receives and determines the request type of access requests. For data write requests, it obtains the target write data and write timestamp, determines the target time interval index based on the write timestamp, and uses CAS atomic operations to write the target write data to the lock-free time-series database. For data query requests, it obtains the target query time, determines the target time interval index based on the target query time, and uses CAS atomic operations to perform data query operations in the lock-free time-series database to obtain the data query result. For data update requests, it obtains the target update time, determines the target time interval index based on the target update time, and uses CAS atomic operations to perform data compression processing in the lock-free time-series database to update the lock-free time-series database. This invention utilizes the continuity of time range in time-series data to construct a time interval index in the lock-free time-series database. Therefore, when executing a database access request, the target data block can be quickly located through the time interval index to execute the corresponding data processing operation. During data processing, CAS atomic operations are used. Based on the hardware-level lock-free concurrency support of CAS atomic operations, no intermediate data states are required, effectively improving the efficiency of data processing.

[0097] like Figure 2 As shown, based on the above method embodiments, corresponding apparatus embodiments are provided;

[0098] One embodiment of the present invention provides a data processing device based on a lock-free time-series database, comprising: a request type determination module, a data writing module, a data query module, and a data update module;

[0099] The request type determination module is used to receive access requests from the lock-free time-series database and determine the request type of the access request.

[0100] The data writing module is used to obtain the target write data and the corresponding write timestamp from the access request when the request type is a data write request, and determine the target time interval index based on the write timestamp, and then use CAS atomic operation to write the target write data to the lock-free time-series database based on the target time interval index;

[0101] The data query module is used to obtain the target query time in the access request when the request type is a data query request, determine the target time interval index based on the target query time, and then use CAS atomic operation to perform data query operation in the lock-free time-series database based on the target time interval index to obtain the data query result.

[0102] The data update module is used to obtain the target update time from the access request when the request type is a data update request, determine the target time interval index based on the target update time, and then use CAS atomic operation to perform data compression processing on the lock-free time series database based on the target time interval index to update the lock-free time series database.

[0103] It is understood that the above-described device embodiments correspond to the method embodiments of the present invention, and can implement the data processing method based on a lock-free timing database provided by any of the above-described method embodiments of the present invention.

[0104] It should be noted that the device embodiments described above are merely illustrative, and some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, in the accompanying drawings of the device embodiments provided by this invention, the connection relationships between modules indicate that they have communication connections, which can specifically be implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement this without any creative effort.

[0105] Based on the above embodiments of the data processing method based on a lock-free time-series database, another embodiment of the present invention provides a terminal device, which includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the data processing method based on a lock-free time-series database according to any embodiment of the present invention.

[0106] For example, in this embodiment, the computer program can be divided into one or more modules, which are stored in the memory and executed by the processor to complete the present invention. The one or more modules may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program in the terminal device.

[0107] The terminal device may be a desktop computer, laptop, handheld computer, or cloud server, etc. The terminal device may include, but is not limited to, a processor and a memory.

[0108] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor. The processor is the control center of the terminal device, connecting all parts of the terminal device via various interfaces and lines.

[0109] Based on the above-described method embodiments, another embodiment of the present invention provides a computer-readable storage medium including a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to execute the data processing method based on a lock-free time-series database as described in any of the above-described method embodiments of the present invention.

[0110] The modules / units integrated in the device / terminal equipment, if implemented as software functional units and sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc.

[0111] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. In particular, it should be noted that any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention for those skilled in the art.

Claims

1. A data processing method based on a lock-free time-series database, characterized in that, include: Receive an access request from a lock-free time-series database and determine the request type of the access request; When the request type is a data write request, the target write data and the corresponding write timestamp are obtained in the access request, and the target time interval index is determined based on the write timestamp. Then, the target write data is written to the lock-free time-series database based on the target time interval index using CAS atomic operation. When the request type is a data query request, the target query time is obtained from the access request, and the target time interval index is determined based on the target query time. Then, the CAS atomic operation is used to perform a data query operation in the lock-free time-series database based on the target time interval index to obtain the data query result. When the request type is a data update request, the target update time is obtained from the access request, and the target time interval index is determined based on the target update time. Then, CAS atomic operations are used to perform data compression processing on the lock-free time-series database based on the target time interval index to update the lock-free time-series database.

2. The data processing method based on a lock-free time-series database according to claim 1, characterized in that, The step of determining the target time interval index based on the write timestamp, and then using CAS atomic operations to write the target write data to the lock-free time-series database based on the target time interval index, includes: The target time interval index is determined based on the written timestamp; Determine whether the target time interval index exists in the lock-free time-series database; When the target time interval index exists in the lock-free time series database, a hot data block is allocated for the target write data, the target write data is written to the hot data block, and the block element information of the hot data block is updated. When the target time interval index does not exist in the lock-free time series database, the target time interval index is created in the lock-free time series database using CAS atomic operations, a hot data block is allocated for the target write data, the target write data is written to the hot data block, and the block cell information of the hot data block is updated.

3. The data processing method based on a lock-free time-series database according to claim 2, characterized in that, The step of creating the target time interval index in the lock-free time-series database using CAS atomic operations also includes: If creating the target time interval index in the lock-free time series database fails, the temporary index in the lock-free time series database is released, and the target time interval index is recreated.

4. The data processing method based on a lock-free time-series database according to claim 1, characterized in that, The process of determining a target time interval index based on the target query time, and then performing a data query operation in the lock-free time-series database based on the target time interval index to obtain the data query results includes: The validity of the target query time is validated. Once the target query time passes the validity check, the cached data corresponding to the target query time is queried in the lock-free time-series database to determine whether the cached data is valid. When the cached data is valid, several target time interval indices are matched in the time interval index of the lock-free time-series database according to the target query time; The data blocks corresponding to each target time interval index are determined as the query data blocks; Generate data query results based on the data corresponding to all query data blocks.

5. The data processing method based on a lock-free time-series database according to claim 4, characterized in that, The step of generating data query results based on the data corresponding to all query data blocks includes: Determine the data type of each queried data block; When the data block type of the query data block is cold data, the query data block is decompressed to obtain compressed data; When the data block type of the queried data block is hot data, the hot data corresponding to the queried data block is obtained; Based on the compressed data and the cold data, generate data query results.

6. The data processing method based on a lock-free time-series database according to claim 1, characterized in that, The step of determining a target time interval index based on the target update time, and then performing data compression processing on the lock-free time series database based on the target time interval index, and updating the lock-free time series database, includes: Determine the target time interval index based on the target update time; The data blocks corresponding to each target time interval index are identified as the data blocks to be updated. Determine the data type of each data block to be updated; When the data block type of the queried data block is hot data, determine whether the hot data corresponding to the queried data block is valid; If the hot data corresponding to the query data block is valid, the hot data corresponding to the query data block is compressed to form compressed data, and the compressed data is stored in the cold data block to update the lock-free time-series database.

7. The data processing method based on a lock-free time-series database according to claim 6, characterized in that, When the data block type of the queried data block is hot data, after determining whether the hot data corresponding to the queried data block is valid, the method further includes: If the hot data corresponding to the query data block is invalid, then delete the invalid hot data and update the query data block.

8. A data processing device based on a lock-free time-series database, characterized in that, include: The module includes a request type determination module, a data writing module, a data query module, and a data update module. The request type determination module is used to receive access requests from the lock-free time-series database and determine the request type of the access request. The data writing module is used to obtain the target write data and the corresponding write timestamp from the access request when the request type is a data write request, and determine the target time interval index based on the write timestamp, and then use CAS atomic operation to write the target write data to the lock-free time-series database based on the target time interval index; The data query module is used to obtain the target query time in the access request when the request type is a data query request, determine the target time interval index based on the target query time, and then use CAS atomic operation to perform data query operation in the lock-free time-series database based on the target time interval index to obtain the data query result. The data update module is used to obtain the target update time from the access request when the request type is a data update request, determine the target time interval index based on the target update time, and then use CAS atomic operation to perform data compression processing on the lock-free time series database based on the target time interval index to update the lock-free time series database.

9. A terminal device, characterized in that, The method includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, it implements the data processing method based on a lock-free timing database as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, include: A stored computer program, wherein, when the computer program is executed, it controls the device containing the computer-readable storage medium to perform the data processing method based on a lock-free time-series database as described in any one of claims 1-7.

Images