Time series database statistics preprocessing method, electronic device, and storage medium

Abstract

The application discloses a time sequence database statistical information preprocessing method, an electronic device and a storage medium, belongs to the database technical field, and aims to solve the technical problem of how to reduce invalid performance loss caused in the data transmission process, and ensure that the size of data reading and the system overhead of aggregation function calculation can be reduced during aggregation query. The technical scheme is that a preprocessing statistical file is added to the storage layer, a corresponding preprocessing statistical file is added to each table in the storage engine, the preprocessing statistical file saves the statistical information of each column data in the device in the form of a fixed time period, and is updated in real time during data insertion; when the data in the table is aggregated and calculated, the aggregation function and the query time period are pushed down to the storage engine of the storage layer, and the storage engine finds the statistical information of the corresponding time period from the preprocessing statistical file according to the time range of the query.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of database, in particular to a time series database statistical information preprocessing method, electronic equipment and storage medium. BACKGROUND

[0002] KaiwuDB is an open architecture, multi-dimensional collaboration, distributed, multi-mode database. KaiwuDB uses its flexible and scalable open architecture, which can be used for resource-limited embedded devices, and can also be used for resource-rich cloud platforms. The main use environment is the time series scenario, that is, the saved data often has time information, and the data is increasing in time under normal circumstances. KaiwuDB adds the concept of device in order to better support the time series scenario, which mainly saves the collection information of the collection points under a certain device in the time series scenario. The information collected in the device is uniformly saved to a table. And the tables corresponding to the same type of device are uniformly saved to a super table. The data of a device is saved to a sub-table.

[0003] The data saving method in KaiwuDB is based on files, that is, the data of a device is saved to a file. The specific saving method is as follows: first, save the metadata information of the device to the file header. Since the number of collection points in a device and the data saved by the collection points are basically fixed length, the data generated by a device at a time is fixed length data, so the data can be encoded and saved to the file line by line, and only the saved data row number information in the metadata needs to be updated. Since in the time series scenario, the information collected by the collection device is basically saved, there will be no operation of modifying a certain data, only the addition, deletion and query of data need to be considered.

[0004] The common structure of time series database is as follows Figure 1As shown, the database is roughly divided into four layers, first is the protocol layer, responsible for the communication between the client and the data server, usually based on TCP / IP socket, mysql and postgreSQL etc. belong to the more common implementation of the protocol layer, then the parsing layer, responsible for parsing the user input sql statement, responsible for some syntax support and check, split the SQL statement into different fields, for the subsequent execution layer, the execution layer is responsible for generating an execution plan according to the parsed information, divided into different operators, and the execution plan is divided into different subnodes (if it is a cluster mode), and executed according to the operator. The real data is saved in the storage layer, which is responsible for data storage and management, and different storage architectures can be used, such as relational, time series, etc. Database to achieve different data types and business. Data can be divided into data information and metadata information, metadata information is the data that describes data information, which provides semantic, structure, source and other information of data information, such as column name, type, length, etc. of a table. And data information is the specific value and text, for example, the specific content of each row in a table, which is a specific description of an object or time.

[0005] In the traditional time series database, for the query statement with aggregation function (such as SUM, MIN, MAX, etc.), the usual way is to first extract the data of this time period to the execution layer through the storage engine, and then the operator in the execution layer performs aggregation operation on the data, and finally returns the calculation result to the user. This process may consume a lot of resources in the actual execution process because of the data transmission from the storage engine to the execution engine, and when the number of columns of the table is large, many invalid column information (which means the column does not need to participate in the aggregation calculation and the user does not need to obtain) is scanned out during the query, increasing the burden of the database system and causing great loss.

[0006] Therefore, how to reduce the invalid performance loss caused in the data transmission process, ensure that the data reading size and the system overhead of the aggregation function calculation can be reduced during the aggregation query, and then speed up the query speed of the aggregation function statement is a technical problem to be solved at present. SUMMARY

[0007] The technical task of the present application is to provide a time series database statistical information preprocessing method, electronic equipment and storage medium, to solve the problem of how to reduce the invalid performance loss caused in the data transmission process, ensure that the data reading size and the system overhead of the aggregation function calculation can be reduced during the aggregation query, and then speed up the query speed of the aggregation function statement.

[0008] The technical task of the present application is achieved in the following manner, a time series database statistical information preprocessing method, the method adds a preprocessing statistical file in the storage layer, adds a corresponding preprocessing statistical file in the storage engine for each table (one table corresponds to one preprocessing file), the preprocessing statistical file saves the statistical information of each column data in the device in the form of a fixed time period, and is updated in real time when data is inserted; when the data in the table is aggregated and calculated, the aggregation function and the query time period are pushed down to the storage engine of the storage layer, the storage engine finds the statistical information of the corresponding time period from the preprocessing statistical file through the time range of the query, and extracts the data that cannot be completely matched with the time period, and processes uniformly in the execution layer, and returns the statistical result.

[0009] As preferred, the preprocessing statistical file is specifically as follows:

[0010] The time series data itself has time characteristics, the data is divided into buckets according to a set time period, the time interval of each bucket is fixed, and the current displayed time interval is set to 1s through a configuration parameter, that is, the bucket saves the statistical information of all columns in the table within 1s; wherein the statistical information of all columns includes the maximum value Max, the minimum value Min, the sum Sum, the number of data rows in the time period and 1 byte of the validity bit marking the validity of the data in the bucket; the length of the maximum value Max, the minimum value Min and the sum Sum of each column is fixed at 8 bytes, the size of the statistical information block corresponding to each column is fixed, the statistical block is positioned through the column number, and the statistical information is mapped from the preprocessing statistical file to the memory through the MMAP method for reading and writing.

[0011] More preferably, the preprocessing statistical file includes a data header and a bucket.

[0012] More preferably, the data header includes the start time stamp of the first bucket (8 bytes), the time period size (8 bytes, such as 1s), and the number of existing buckets in the file (8 bytes);

[0013] Wherein, the columns of the table are fixed, so the length of the bucket is also fixed, denoted as bucket_len, and then the position of any existing bucket or time t is located through the number of buckets bucket_count and the time stamp information of the first bucket timestamp1, the formula is as follows: i

[0014] start_pos(t i )=(t i -timestamp1) / 1*bucket_len+header_len;

[0015] The position of the latest bucket, the formula is as follows:​

[0016] new_bucket_pos = bucket_count * bucket_len + header_len.

[0017] More preferably, the bucket includes a valid bit, a number of data rows, and statistical information of each column;

[0018] A 1-byte valid bit is set in the header of each bucket, which is set to false before updating the bucket and is restored to true after updating;

[0019] When other query threads query, the valid bit is first judged, and the statistical information is read only when the valid bit is valid.

[0020] More preferably, the valid bit is valid, that is, the byte is 0x01, indicating that the statistical information in the bucket is valid and complete, and when performing an aggregate query, the bucket is first located according to the timestamp, and then if the data in the bucket is being rewritten by other threads, the statistical information is invalid and incomplete, and the statistical value in the bucket cannot be read at this time. Only when the bucket is not rewritten by other threads, that is, the information is valid and complete, can the statistical value be extracted according to the aggregate function and column number for statistical calculation.

[0021] More preferably, the aggregate query is as follows:

[0022] According to the time range of the query, the corresponding bucket in the preprocessed statistical file is found;

[0023] After extracting the statistical information in the bucket, the calculation is performed in the storage layer, and the data of other time periods that cannot match the bucket are returned to the storage layer together for further calculation.

[0024] More preferably, the bucket includes a statistical block corresponding to each column, and the number of statistical blocks corresponds to the number of columns. The statistical block saves the maximum value, the minimum value, and the sum of any column.

[0025] More preferably, when performing an aggregate query, the statistical information in the preprocessed statistical file is preferentially searched. The aggregate function reading process is as follows:

[0026] (1) Query the maximum value of any column in the time period from t i d1 seconds later;

[0027] (2) Determine the position of the preprocessed statistical file according to the table name and open it;

[0028] (3) Judge whether t i is greater than the start timestamp in the preprocessed statistical file:

[0029] If yes, then locate the position of the bucket according to (t i -startTime) / interval, and then execute step (4);

[0030] If no, then locate the position of the first bucket in the file, and then execute step (4);

[0031] (4) judge whether the number of buckets in the pre-processing statistics file is greater than (t i -startTime) / interval:

[0032] If yes, then execute step (5);

[0033] If no, then return empty;

[0034] (5) extract data between t i and the start timestamp of the current bucket;

[0035] (6) locate the first bucket greater than t i ;

[0036] (7) judge whether the end timestamp of the current bucket is greater than t i +d1:

[0037] If yes, then extract data between the start timestamp of the current bucket and t i +d1, and then execute step (10);

[0038] If no, then execute step (8);

[0039] (8) judge whether the current bucket is valid:

[0040] If yes, then read the maximum value of the column corresponding to step (1) in the bucket, and record it, and then execute step (9);

[0041] If no, then record the actual timestamp of the current bucket, and then execute step (9);

[0042] (9) judge whether the number of buckets is exhausted:

[0043] If yes, then execute step (10);

[0044] If no, then jump to the next bucket, and jump to step (7);

[0045] (10) perform maximum value operation on the statistics results, and return the read data (if any) together.

[0046] An electronic device, comprising: a memory and at least one processor;

[0047] The memory has stored a computer program thereon;

[0048] The at least one processor executes the computer program stored in the memory, so that the at least one processor executes the time series database statistical information preprocessing method as described above.

[0049] A computer readable storage medium has stored a computer program therein, and the computer program can be executed by a processor to implement the time series database statistical information preprocessing method as described above.

[0050] The time series database statistical information preprocessing method, the electronic device and the storage medium have the following advantages:

[0051] (I) The method for increasing the preprocessing file saves the statistical information and avoids repeated calculation.

[0052] (II) The present application reduces the transmission of data from the storage layer to the execution layer, reduces the system overhead, and reduces the performance loss caused by calculation.

[0053] (III) The present application saves the aggregation result in advance, and accelerates the query speed of the aggregation query.

[0054] (IV) The present application returns the aggregation result and the original data to the execution layer, which guarantees the correctness of the result while reducing the computational complexity, and has universality.

[0055] (V) The purpose of the present application is to reduce the invalid performance loss caused by data transmission, and to calculate the results of some aggregation functions in advance when data is inserted, save the aggregation results in a period of time, and search the results in the preprocessing statistical file first when querying, and then extract and calculate the data that cannot match the time period in the preprocessing statistical file.

[0056] (VI) The present application can not only reduce the consumption of the original data transmission between the execution layer and the storage layer, but also reduce the data processing and calculation process of the execution layer. Since the calculation result is saved, the size of data reading and the system overhead of aggregation function calculation can be greatly reduced during aggregation query, and the query speed of the aggregation function statement is accelerated. BRIEF DESCRIPTION OF DRAWINGS

[0057] The present application will be further described below in conjunction with the drawings.

[0058] FIG. 1 is a structural schematic diagram of a storage system; Figure 1 FIG. 2 is a schematic diagram of a preprocessing statistical file; and

[0059] FIG. 2 is a schematic diagram of a preprocessing statistical file; and Figure 2 FIG. 2 is a schematic diagram of a preprocessing statistical file; and

[0060] attached Figure 3 Read flow chart for aggregation function. DETAILED DESCRIPTION

[0061] The time series database statistical information preprocessing method, electronic equipment and storage medium will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0062] Embodiment 1:

[0063] The embodiment provides a time series database statistical information preprocessing method, which adds a preprocessing statistical file to the storage layer, as shown in the attached Figure 1 As shown in the attached Figure 2 Since the time series data itself has time characteristics, the preprocessing statistical file saves the statistical information of each column of data in the device in the form of a fixed time period, and is updated in real time when data is inserted; when the data in the table is aggregated, the aggregation function and the query time period are pushed down to the storage engine of the storage layer, the storage engine finds the statistical information of the corresponding time period from the preprocessing statistical file through the query time range, and extracts the data that cannot be matched with the time period in an integrated manner, and processes in the execution layer, and returns the statistical result.

[0064] In the embodiment, the preprocessing statistical file is as follows:

[0065] The time series data itself has time characteristics, and the data is divided into buckets according to the set time period, the time interval of each bucket is fixed, and the current displayed time interval is set to 1s through the configuration parameter, that is, the bucket saves the statistical information of all columns in the table within 1s; wherein the statistical information of all columns includes the maximum value Max, the minimum value Min, the sum Sum, the number of data rows in the time period and the 1-byte valid bit marking the validity of the data in the bucket; the length of the maximum value Max, the minimum value Min and the sum Sum of each column is fixed at 8 bytes, the size of the statistical block corresponding to each column is fixed, the statistical block is positioned through the column number, and the statistical information is mapped from the preprocessing statistical file to the memory through the MMAP method for reading and writing.

[0066] In the embodiment, the preprocessing statistical file includes a data header and a bucket.

[0067] In the embodiment, the data header includes the start timestamp of the first bucket (8 bytes), the time period size (8 bytes, such as 1s), and the number of existing buckets in the file (8 bytes);

[0068] Wherein, the column of the table is fixed, so the length of the bucket is fixed, denoted as bucket_len, and then the position of any bucket or time t is located through the bucket number information bucket_count and the timestamp information timestamp1 of the first bucket, and the formula is as follows: i

[0069] start_pos(t i )=(t i -timestamp1) / 1*bucket_len+header_len;

[0070] The position of the latest bucket, and the formula is as follows:

[0071] new_bucket_pos=bucket_count*bucket_len+header_len.

[0072] In the embodiment, the bucket includes valid bits, data rows and statistical information of each column.

[0073] A 1-byte valid bit is arranged in the header of each bucket, which is set to false before the bucket is updated, and is restored to true after the bucket is updated.

[0074] When other query threads query, the valid bit is first judged, and only when the valid bit is valid, the statistical information is read.

[0075] Wherein, the valid bit is valid, that is, when the valid bit is true, that is, the byte is 0x01, indicating that the statistical information in the bucket is valid and complete, when the aggregation query is performed, the bucket is first located according to the timestamp, and then if the data in the bucket is being rewritten by other threads, the statistical information is invalid and incomplete, at this time, the statistical value in the bucket cannot be read, only when the bucket is not rewritten by other threads, that is, the information is valid and complete, the statistical value can be extracted according to the aggregation function and the column number to perform statistical calculation.

[0076] In the embodiment, the aggregation query is as follows:

[0077] According to the time range of the query, the corresponding bucket in the preprocessed statistical file is found.

[0078] After the statistical information in the bucket is extracted, the calculation is performed in the storage layer, and the data of other time periods that cannot match the bucket are returned to the storage layer together, and further calculation is performed by the storage layer.

[0079] In the embodiment, the bucket includes statistical blocks corresponding to each column, the number of statistical blocks corresponds to the number of columns, and the statistical blocks save the maximum value, minimum value and total sum of any column as three aggregation results.​

[0080] In this embodiment, when aggregating a query, the statistical information in the preprocessed statistical file is searched first; for example, as shown in the attached Figure 3 The aggregation function reading process is as follows:

[0081] (1) The query is from t i to t i +d1, and the maximum value of any column in the time interval is searched;

[0082] (2) The position of the preprocessed statistical file is determined according to the table name and is opened;

[0083] (3) It is judged whether t i is greater than the start time stamp in the preprocessed statistical file:

[0084] ① If yes, the position of the bucket is located according to (t i -startTime) / interval, and the next step (4) is executed;

[0085] ② If no, the position of the first bucket in the file is located, and the next step (4) is executed;

[0086] (4) It is judged whether the number of buckets in the preprocessed statistical file is greater than (t i -startTime) / interval:

[0087] ① If there is valid data in the preprocessed file, step (5) is executed;

[0088] ② If there is no valid data in the preprocessed file, an empty value is returned;

[0089] (5) The data from t i to the start time stamp of the current bucket is extracted;

[0090] (6) The first bucket greater than the t i time point is located;

[0091] (7) It is judged whether the end time stamp of the current bucket is greater than t i +d1:

[0092] ① If yes, the data from the start time stamp of the current bucket to t i +d1 is extracted, and the next step (10) is executed;

[0093] ② If no, step (8) is executed;

[0094] (8) It is judged whether the current bucket is valid:

[0095] ① If yes, the maximum value of the column corresponding to step (1) in the bucket is read and recorded, and the next step (9) is executed;

[0096] If not, record the real time stamp of the current bucket, and go to step (9);

[0097] (9) Determine whether the number of buckets is exhausted:

[0098] If yes, go to step (10);

[0099] If not, jump to the next bucket, and go to step (7);

[0100] (10) Perform maximum operation on the statistical result, and return the result together with the read data (if any).

[0101] When a query with an aggregation function is needed, because the data in the bucket is updated in real time, if the time range of the query is not within the time range recorded in the bucket, it can be considered that there is no data. Because the start time and end time of the query may not match the start time of the bucket accurately, the buckets that can be used need to be filtered according to the query range. For example, the query time is from 1997.01.0108:00:05:345 to 1997.01.01 08:00:10:345, and the interval in the statistical file is 1s, and the time stamp of the start bucket is 1997.01.01 08:00:00:000. Only the time period from 1997.01.0108:00:06:000 to 1997.01.01 08:00:10:000 is complete, that is, only the statistical data in the 6th to 9th buckets can be used. After extracting the statistical data in these buckets, a simple aggregation operation is performed to obtain an aggregation result. For the time that cannot complete the matching bucket, that is, from 1997.01.01 08:00:05:345 to 1997.01.01 08:00:06:000, and from 1997.01.0108:00:10:000 to 1997.01.01 08:00:10:345, the statistical result of the corresponding time period needs to be extracted and returned to the upper layer for data analysis and calculation. The final result is obtained by combining the aggregation result obtained from the statistical file.

[0102] When the aggregation function is used, first, the range of the bucket that can be used is determined according to the time range of the query, since the time interval of the data saved in each bucket is fixed, so the range of the bucket contained in the time range of the query can be determined according to the start time of the data in the first bucket, for example, the time of the first data is t1, when the first data is inserted into the table, the statistical file is generated, the first bucket is generated, and the start time of the bucket is set to t1. Then, a bucket is updated every 1s, so when I want to query the maximum value of the second column between t1 and t1+4s, then first, the range of the bucket that can be used is determined according to the time, which is the first to the fourth bucket, and then in each bucket, the address of the maximum value of the second column is located according to the width information of the column saved by the system (which is saved in the system metadata), for the figure, assuming that column 1 and column 2 are both 4 bytes, the address of the maximum value of the second column should be the first address of the bucket + 4 bytes of valid bits + 4 bytes of count + 4 bytes of maximum value + 4 bytes of minimum value + 8 bytes of sum value, then the maximum value of the second column can be located, and then the 4 bytes after the address are read out as the value read out in the bucket, the maximum value of the value read out in the 4 buckets is obtained, and the final result is obtained.

[0103] When the query time cannot be matched with the start time of the bucket completely, for example, the maximum value of the second column between t1+0.3s and t1+4.3s is queried, then the first bucket (t1 to t1+1s) and the fifth bucket (t1+4s to t1+5s) cannot be used, only the data in the second to fourth buckets can be read, and then the data from t1+0.3s to t1+1s and the data from t1+4s to t1+4.3s need to be queried, the data of the original table is read out, and then the maximum value is obtained.

[0104] Embodiment 2

[0105] The embodiment also provides an electronic device, comprising a memory and a processor.

[0106] The memory stores computer execution instructions.

[0107] The processor executes the computer execution instructions stored in the memory, so that the processor executes the time series database statistical information preprocessing method in any embodiment of the application.

[0108] The processor can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), ready-to-program gate arrays (FPGAs) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The processor can be a microprocessor, or the processor can be any conventional processor.

[0109] The memory can be used to store computer programs and / or modules, and the processor can realize various functions of the electronic device by running or executing the computer programs and / or modules stored in the memory and calling data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application program required for a function, and the like; and the data storage area can store data created according to use of the terminal, and the like. In addition, the memory can further include a high-speed random access memory, and can further include a non-volatile memory such as a hard disk, a memory, a plug-in hard disk, a secure memory card (SMC), a secure digital (SD) card, a flash memory card, at least one disk storage period, a flash memory device, or other volatile solid memory device.

[0110] Embodiment 3:

[0111] The embodiment also provides a computer readable storage medium, in which a plurality of instructions are stored, the instructions are loaded by a processor to make the processor execute the time series database statistical information preprocessing method in any embodiment of the application. Specifically, a system or device provided with a storage medium can be provided, the storage medium stores software program code for realizing the functions of any one of the above embodiments, and the computer (or CPU or MPU) of the system or device reads and executes the program code stored in the storage medium.

[0112] In this case, the program code read from the storage medium can realize the functions of any one of the above embodiments, and therefore the program code and the storage medium storing the program code constitute a part of the application.

[0113] The storage medium for providing the program code includes a floppy disk, a hard disk, a magneto-optical disk, an optical disk (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RYM, DVD-RW, DVD+RW), a magnetic tape, a non-volatile memory card, and a ROM. Alternatively, the program code can be downloaded from a server computer via a communication network.

[0114] In addition, it should be clear that not only the program code read by the computer can be executed, but also part or all of the actual operations can be completed by the operating system and the like operating on the computer based on the instructions of the program code, so as to realize the functions of any one of the above embodiments.

[0115] Further, it is understood that the program code read by the storage medium can be written into the memory provided in the expansion board inserted into the computer or the memory provided in the expansion unit connected to the computer, and then the CPU or the like mounted on the expansion board or the expansion unit is caused to perform part or all of the actual operation based on the instruction of the program code, thereby realizing the function of any of the above-described embodiments.

[0116] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, and are not limited thereto; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can still be modified, or part or all of the technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for preprocessing statistical information in a time-series database, characterized in that, This method involves adding a preprocessed statistics file to the storage layer and a corresponding preprocessed statistics file to each table in the storage engine. This preprocessed statistics file stores the statistical information of each column of data in the device in a fixed-length time period and updates it in real time when data is inserted. When performing aggregation calculations on the data in the table, the aggregation function and the query time period are pushed down to the storage engine in the storage layer. The storage engine finds the statistical information of the corresponding time period from the preprocessed statistics file based on the query time range, extracts other data that cannot completely match the time period, processes them uniformly in the execution layer, and returns the statistical results. The preprocessed statistical files are as follows: Time-series data inherently possesses time characteristics. The data is divided into buckets according to set time periods, with a fixed time interval for each bucket. The current display time interval is set to 1 second via configuration parameters, meaning that each bucket stores the statistical information of all columns within the table within 1 second. The statistical information for all columns includes the maximum value (Max), minimum value (Min), sum (Sum), the number of data rows within the time period, and a 1-byte valid bit marking the validity of the data in the bucket. The length of the maximum value (Max), minimum value (Min), and sum (Sum) for each column is a fixed 8 bytes. The size of the statistical block corresponding to each column is fixed. The statistical block is located by column number, and the statistical information is mapped from the preprocessed statistical file to memory using the MMAP method for reading and writing. Each bucket contains a statistical block corresponding to each column. The number of statistical blocks corresponds to the number of columns. Each statistical block stores the maximum, minimum, and sum of any column.

2. The method for preprocessing statistical information in a time-series database according to claim 1, characterized in that, The preprocessed statistics file consists of two parts: a header and buckets.

3. The time-series database statistical information preprocessing method according to claim 2, characterized in that, The data header includes the start timestamp of the first bucket, the size of the time period, and the number of buckets already in the file; The columns in the table are fixed, so the length of the bucket is also fixed, denoted as bucket_len. Then, the number of buckets (bucket_count) and the timestamp (timestamp1) of the first bucket are used to locate any existing bucket or time t. i The position is given by the following formula: start_pos(t i ) = (t i -timestamp1) / 1 bucket_len + header_len; The latest bucket position is determined by the following formula: new_bucket_pos = bucket_count bucket_len + header_len。 4. The time-series database statistical information preprocessing method according to claim 2, characterized in that, The bucket contains the significant bits, the number of data rows, and statistical information for each column; Set a 1-byte valid bit at the beginning of each bucket. Before updating the bucket, set the valid bit to false and restore it to true after the update. When other query threads perform a query, they first determine the validity bit, and only read the statistical information if and only if the validity bit is valid; Specifically, the validity bit is defined as follows: when the validity bit is true, that is, the byte is 0x01, it indicates that the statistical information in the bucket is valid and complete. When performing an aggregation query, the bucket is first located based on the timestamp. If the data in the bucket is being modified by other threads, the statistical information is invalid and incomplete. At this time, the statistical value in the bucket cannot be read. Only when the bucket has not been modified by other threads, that is, the information is valid and complete, can the statistical value be retrieved and statistical calculations be performed based on the aggregation function and column number.

5. The method for preprocessing statistical information in a time-series database according to claim 4, characterized in that, The aggregate query is as follows: Find the corresponding bucket in the preprocessed statistics file based on the time range of the query; After extracting the statistical information from the bucket, it is calculated in the storage layer and returned to the storage layer along with data from other time periods that do not match the bucket, for further calculation.

6. The time-series database statistical information preprocessing method according to claim 4, characterized in that, When performing aggregate queries, the system prioritizes searching for statistical information in the preprocessed statistics file; the specific process of reading aggregate functions is as follows: (1) Query from t i The maximum value in any column within d1 seconds after the time; (2) Determine the location of the preprocessing statistics file based on the table name and open it; (3) Determine t i Is it greater than the start timestamp in the preprocessed statistics file? ①If so, then according to (t) i -startTime) / interval locates the bucket position, and the next step is to execute step (4); ②If not, locate the position of the first bucket in the file and proceed to step (4). (4) Determine whether the number of buckets in the preprocessed statistics file is greater than (t). i -startTime) / interval: ①If there is valid data in the preprocessed statistics file, then proceed to step (5); ② If there is no valid data in the preprocessed statistics file, return empty; (5) Extract from t i Data between the start and end timestamps of the current bucket; (6) Locate a value greater than t i The first bucket at a given point in time; (7) Determine if the current bucket's end timestamp is greater than t. i +d1: ① If so, then extract the timestamp from the start of the current bucket to t. i The data between +d1 will be processed in step (10). ②If not, proceed to step (8); (8) Determine if the current bucket is valid: ① If so, read the maximum value of the column corresponding to step (1) in the bucket and record it. Then, execute step (9) next. ②If not, record the current timestamp of the bucket and proceed to step (9). (9) Determine if the number of buckets has been exhausted: ①If so, proceed to step (10); ②If not, then jump to the next bucket and jump to step (7); (10) Perform maximum value calculation on the statistical results and return them together with the read data.

7. An electronic device, characterized in that, include: Memory and at least one processor; The memory contains computer programs; The at least one processor executes the computer program stored in the memory, causing the at least one processor to perform the time-series database statistical information preprocessing method as described in any one of claims 1 to 6.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that can be executed by a processor to implement the time-series database statistical information preprocessing method as described in any one of claims 1 to 6.

Images