Construction method and device of time sequence database system, electronic equipment and readable medium

Abstract

The embodiment of the invention provides a construction method and device of a time sequence database system, electronic equipment and a readable medium, and a micro-service cluster of the time sequence database system comprises a plurality of Routers, a plurality of Writers, a plurality of Queriers, a plurality of Absorbers and a plurality of Compactors; according to the method, the value of a performance index of a micro-service cluster dynamically monitored by a self-monitoring and self-diagnosis module is obtained, and the number of Absorber, Writer and Querier is increased or decreased according to the value of the performance index. According to the method, the number of the Absorber micro-service nodes, the Writer micro-service nodes and the Querier micro-service nodes is dynamically adjusted according to the value of the performance index of the micro-service cluster dynamically monitored by the self-monitoring and self-diagnosis module, the number of the micro-service nodes is automatically expanded and contracted so as to adapt to constantly changing loads and flows, no perception is given to a user, and the user experience is improved.

Description

Technical Field

[0001] The present invention relates to the field of database technology, and in particular to a method for constructing a time series database system, a device for constructing a time series database system, an electronic device, and a computer-readable medium. Background Art

[0002] In the fields of meteorology, electricity, geography, etc., there is a need to collect and store a series of large-scale monitoring data with time as the dimension. Such data is highly related to time and usually focuses on data in the recent period. In this context, time series databases came into being, and databases represented by InfluxDB and Prometheus have become the mainstream of time series databases.

[0003] Among them, Prometheus provides rich query and chart functions, but is inferior to InfluxDB in terms of scalability, data storage structure, and eventual consistency. Therefore, it mainly provides monitoring services for other applications.

[0004] InfluxDB provides an open source stand-alone version and a commercial cluster version. The stand-alone version is inferior to the cluster version in terms of disaster recovery and performance. The cluster version is directly provided by InfluxDB as a hosting service, which is difficult to customize and cannot guarantee data security. Both the stand-alone version and the cluster version rely on the traditional operation and maintenance model, requiring regular attention to the health status of the database and manual operation and maintenance. Summary of the invention

[0005] The embodiments of the present invention provide a method, device, electronic device and computer-readable storage medium for constructing a time series database system to solve the problem that existing solutions rely on traditional operation and maintenance models and require regular attention to the health status of the database and manual operation and maintenance.

[0006] The embodiment of the present invention discloses a method for constructing a time series database system, the method comprising:

[0007] Obtain the values ​​of performance indicators of the microservice cluster dynamically monitored by the self-monitoring and self-diagnosis module; the performance indicators include the total usage of the central processing unit (CPU) and memory, the number of bytes written and read per second in the data directory, the amount of data received and written to the storage base, and the request response time;

[0008] If the total CPU and memory usage values ​​continue to be higher than the preset first usage threshold, or the values ​​of the number of bytes written and read per second in the data directory continue to be higher than the preset first ratio of the disk IO upper limit, or the value of the amount of data received and written to the storage base continues to be higher than the preset third ratio of the network IO upper limit, then increase the number of Absorber microservice nodes; if the total CPU and memory usage values ​​continue to be lower than the preset second usage threshold, or the values ​​of the number of bytes written and read per second in the data directory continue to be lower than the preset second ratio of the disk IO upper limit, or the value of the amount of data received and written to the storage base continues to be lower than the preset fourth ratio of the network IO upper limit, then reduce the number of Absorber microservice nodes;

[0009] If the request response time value continues to exceed the preset time, increase the number of Writer microservice nodes and Querier microservice nodes.

[0010] The embodiment of the present invention further discloses a device for constructing a time series database system, the device comprising:

[0011] An acquisition unit is used to obtain the values ​​of performance indicators of the microservice cluster dynamically monitored by the self-monitoring and self-diagnosis module; wherein the performance indicators include the total utilization rate of the central processing unit CPU and memory, the number of bytes written and read per second in the data directory, the amount of data received and written to the storage base, and the request response time;

[0012] A construction unit is used to increase the number of Absorber microservice nodes when the total CPU and memory usage values ​​are continuously higher than a preset first usage threshold, or the number of bytes written and read per second in the data directory is continuously higher than a preset first proportion of the disk IO upper limit, or the amount of data received and written to the storage base is continuously higher than a preset third proportion of the network IO upper limit; reduce the number of Absorber microservice nodes if the total CPU and memory usage values ​​are continuously lower than a preset second usage threshold, or the number of bytes written and read per second in the data directory is continuously lower than a preset second proportion of the disk IO upper limit, or the amount of data received and written to the storage base is continuously lower than a preset fourth proportion of the network IO upper limit; and increase the number of Writer microservice nodes and Querier microservice nodes when the request response time value continuously exceeds a preset time.

[0013] The embodiment of the present invention further discloses an electronic device, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0014] The memory is used to store computer programs;

[0015] The processor is used to implement the method described in the embodiment of the present invention when executing the program stored in the memory.

[0016] The embodiment of the present invention further discloses one or more computer-readable media on which instructions are stored. When executed by one or more processors, the processors are enabled to execute the method described in the embodiment of the present invention.

[0017] The embodiments of the present invention include the following advantages:

[0018] The embodiment of the present invention provides a construction method, device, electronic device and readable medium of a time series database system, wherein the microservice cluster of the time series database system includes multiple Routers, multiple Writers, multiple Queriers, multiple Absorbers and multiple Compactors; the method obtains the value of the performance indicator of the microservice cluster dynamically monitored by the self-monitoring and self-diagnosis module, and increases or decreases the number of Absorbers, Writers and Queriers according to the value of the performance indicator. The method dynamically adjusts the number of Absorber microservice nodes, Writer microservice nodes and Querier microservice nodes through the value of the performance indicator of the microservice cluster dynamically monitored by the self-monitoring and self-diagnosis module, realizes automatic scaling of the number of microservice nodes to adapt to the ever-changing load and traffic, and is imperceptible to users, thereby improving user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a schematic diagram of a network topology diagram of a time series database system provided in an embodiment of the present invention;

[0020] Figure 2 is a schematic diagram of the kernel structure of a time series database system provided in an embodiment of the present invention;

[0021] Figure 3 is a schematic diagram of a Routers processing flow provided in an embodiment of the present invention;

[0022] Figure 4 is a schematic diagram of a Writers processing flow provided in an embodiment of the present invention;

[0023] Figure 5 is a schematic diagram of a Queriers processing flow provided in an embodiment of the present invention;

[0024] Figure 6 is a schematic diagram of an absorbers processing flow provided in an embodiment of the present invention;

[0025] Figure 7 is a flowchart of the steps of a method for constructing a time series database system provided in an embodiment of the present invention;

[0026] Figure 8 is a structural block diagram of a device for constructing a time series database system provided in an embodiment of the present invention;

[0027] Fig. 9 is a block diagram of an electronic device provided in an embodiment of the present invention;

[0028] Fig.10 is a schematic diagram of a computer-readable medium provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0029] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0030] In the process of implementing the present invention, the inventors found that time series databases such as InfluxDB provide open source stand-alone versions and commercial cluster versions. The stand-alone version is inferior to the cluster version in terms of disaster recovery and performance. The cluster version is directly provided by InfluxDB as a hosting service, which is difficult to customize and cannot guarantee data security. However, both the stand-alone version and the cluster version rely on the traditional operation and maintenance model, requiring regular attention to the health status of the database and manual operation and maintenance.

[0031] Based on this, an embodiment of the present invention provides a construction method, device, electronic device and readable medium of a time series database system, wherein the microservice cluster of the time series database system includes multiple Routers, multiple Writers, multiple Queriers, multiple Absorbers and multiple Compactors; the method obtains the value of the performance indicator of the microservice cluster dynamically monitored by the self-monitoring and self-diagnosis module, and increases or decreases the number of Absorbers, Writers and Queriers according to the value of the performance indicator. The method dynamically adjusts the number of Absorber microservice nodes, Writer microservice nodes and Querier microservice nodes through the value of the performance indicator of the microservice cluster dynamically monitored by the self-monitoring and self-diagnosis module, realizes automatic scaling of the number of microservice nodes to adapt to the ever-changing load and traffic, and is imperceptible to users, thereby improving user experience.

[0032] The method for constructing a time series database system provided in this embodiment is used to construct a time series database system with automatically scalable number of microservice nodes.

[0033] The time series database system includes: a microservice cluster, a self-monitoring and self-diagnosis module, an identity management module and a storage base.

[0034] The microservice cluster is the core of the time series database system, including functions such as routing, writing, querying, data encoding and compression; the self-monitoring and self-diagnosis module realizes dynamic monitoring of database performance and occupies resources on demand; the identity management module provides a unified identity authentication function, including external user authentication and communication authentication between the kernel (i.e., the microservice cluster) and the storage base; the storage base includes an object storage service and a relational database cluster. The object storage provides services for storing data written by users, and the relational database cluster stores some metadata generated by the operation of the system, including database status, user data, etc.

[0035] Figure 1 shows a network topology diagram of a time series database system. Figure 2 shows the kernel structure of the time series database system, see Figure 1 and Figure 2 Describe each part in detail.

[0036] 1. Identity management module

[0037] The identity management module is used to perform user identity authentication and communication authentication between the microservice cluster and the storage base.

[0038] The identity management module provides a unified identity authentication function for the time series database system, including external user identity authentication and internal communication authentication between the microservice cluster and the storage base.

[0039] Among them, external identity authentication can be implemented in the form of tokens. For example, when a user accesses a time series database system, the user needs to provide a valid token.

[0040] In addition, in order to increase the usability of the time series database system, the identity management module can be customized. The time series database system can reserve the interface of the identity management module. Users can choose a suitable identity management platform to manage the time series database system, allowing customized configuration of token permissions. If the user has special needs, a customized identity management platform can be developed, and only the reserved identity authentication interface needs to be implemented.

[0041] For example, a user wants to use the OAuth2 identity authentication platform. Since this system provides a set of identity authentication interfaces, as long as the external authentication platform implements this interface, it can switch to the OAuth2 identity authentication platform to manage the system database.

[0042] The gRPC protocol can be used for communication between interfaces to improve communication efficiency.

[0043] The communication authentication between the microservice cluster and the storage base can be pre-configured by the user.

[0044] 2. Storage base

[0045] The storage base can implement object storage and metadata storage.

[0046] For example, the storage base includes an object storage service and a relational database cluster. Object storage can be implemented through the object storage service, and metadata storage can be implemented through the relational database cluster.

[0047] Object storage is the storage of data written by users. In other words, object storage services can be used to store data written by users.

[0048] The relational database cluster stores some metadata generated by system operation, including database status, user data, etc.

[0049] When the time series database system in the prior art, such as InfluxDB, is expanded, it is necessary to migrate Shards, and the load balancing and performance of each node must be considered comprehensively to rebalance the data of each node. This process requires complex algorithms or manual intervention, during which the database's external service capabilities are significantly reduced. The process is time-consuming and error-prone. The time series database system constructed by the construction method of the time series database system provided in this embodiment uses a storage base, which can flexibly expand the storage space, and shields the database kernel from complex technical details, without paying attention to the specific operation implementation methods such as data replication and deletion.

[0050] In addition, in order to increase the usability of the time series database system, the time series database system is compatible with the S3 protocol. According to actual needs, you can choose an object storage service that is compatible with this protocol, such as minio and AWS.

[0051] Among them, the S3 protocol is a protocol for accessing and operating object storage, which defines a set of API operations and corresponding request / response formats.

[0052] For example, a user wants to use the minio storage base and OAuth2 identity authentication platform. Since the database of this system is compatible with the S3 protocol, it is only necessary to configure the monio access token in the configuration file when starting the database, and the database can append the data compressed by the Compactor to the minio storage bucket.

[0053] 3. Self-monitoring and self-diagnosis module

[0054] The self-monitoring and self-diagnosis module is used to dynamically monitor the performance of the microservice cluster.

[0055] The self-monitoring and self-diagnosis modules can dynamically monitor database performance and occupy resources as needed.

[0056] In specific implementation, the self-monitoring and self-diagnosis module can periodically monitor the performance indicators of the microservice cluster. For example, the performance indicators include but are not limited to: the total CPU (Central Processing Unit) and memory usage of each microservice node in the microservice cluster, the number of bytes written and read per second in the data directory, the amount of data received and written to the storage base, and the request response time.

[0057] The time series database system maintains performance indicators internally. The performance indicators are stored in memory and initialized when the time series database system is started. The performance indicators are shown in Table 1 below:

[0058] Table 1

[0059]

[0060]

[0061] 4. Microservice Cluster

[0062] The microservice cluster is the core of the time series database system, which is used to respond to user requests based on the authentication results of the identity management module, write data to the storage base, or read data from the storage base. Therefore, the microservice cluster can realize functions such as routing, writing, querying, data encoding and compression, and provide the entire time series database with read and write capabilities and data conversion and compression capabilities.

[0063] The microservice cluster includes: multiple microservice nodes.

[0064] Among them, the microservice nodes include the routing Router microservice node, the writing Writer microservice node, the query Querier microservice node, the persistence Absorber microservice node, and the compression Compactor microservice node.

[0065] Moreover, multiple Router microservice nodes constitute the routing microservice set Routers, multiple Writer microservice nodes constitute the writing microservice set Writers, multiple Querier microservice nodes constitute the query microservice set Queriers, multiple Absorber microservice nodes constitute the persistence microservice set Absorbers, and multiple Compactor microservice nodes constitute the compression microservice set Compactors.

[0066] 1) Routers

[0067] Routers is a collection of several Router microservice nodes, responsible for routing queries or writing requests.

[0068] Since Routers are composed of several Router microservice nodes, they can implement reverse proxy and load balancing functions. In addition, they are also responsible for authenticating user requests.

[0069] For example, Routers are used to determine the category of a user's request based on the authentication result of the identity management module, where the category is write or query. Requests with the write category are routed to Writers, and requests with the query category are routed to Queriers.

[0070] by Figure 3 Taking the example of Routers, the processing flow of Routers in specific implementation is as follows:

[0071] When a user sends a request to the time series database system, Routers first verify the token information with the identity management module so that the identity management module can verify whether the token is legal. If it is illegal, the authentication result of the identity management module is authentication failure. If it is legal, the authentication result of the identity management module is authentication success. Routers obtain the authentication result of the identity management module. If the authentication result is authentication failure, the request is not processed. If the authentication result is authentication success, it is forwarded to the next microservice node according to the specific type of the request. The process is specifically as follows: determine the category of the user's request, such as determining whether the request is a write request. If it is a write request, determine the category of the request as write, otherwise determine the category of the request as query. Route requests with the category of write to Writers, and route requests with the category of query to Queriers.

[0072] 2) Writers

[0073] Writers is a collection of several Writer microservice nodes, responsible for processing write requests.

[0074] For example, Writers, which handle write requests, form and persist new data files through Absorbers.

[0075] Among them, the persistent data files are compressed by Compactors and placed in the storage base.

[0076] In the specific implementation, the processing flow of Writers can be to determine the format of the written data after receiving a write request. If the format is a preset format, the written data is parsed into internally represented data points, and the Points are delivered to the Absorbers for consumption through the message queue, so that the Absorbers form and persist new data files.

[0077] The default formats are CSV and line protocol.

[0078] The line protocol format is a text protocol used to represent and transmit data in the form of lines. By sending text in the line protocol format, data can be written to the InfluxDB database.

[0079] by Figure 4 For example, the time series database system can accept data in CSV or line protocol format, then Writers parse the correctly formatted written data into internally represented data points, and then send the Points to the message queue for consumption by Absorbers, which respond to this request based on the consumption results. If the format of the written data is incorrect, a 403 response is made directly to this request.

[0080] The possible response contents of the write request are shown in Table 2:

[0081] Table 2

[0082] condition Status Code Response content Data format error 403 Illegal data format Write Success 204 none

[0083] 3) Queriers

[0084] Queriers is a collection of several Querier microservice nodes, responsible for processing query requests and parsing query statements.

[0085] For example, Queriers is used to process requests classified as queries and reads the query data files from the storage base through Absorbers.

[0086] In the specific implementation, after receiving a request of the query type, Queriers parses the query statement into AST (Abstract Syntax Tree). Determine the query scope and the timeline involved according to AST. Request Absorbers to read the queried data file from the storage base according to the query scope and the timeline involved. Feedback the read data file.

[0087] Among them, AST is a database query statement (such as SQL) or expression (such as conditional expression) that needs to be parsed and analyzed before execution. The parser converts the query statement or expression into an abstract syntax tree, so as to perform various optimization operations, such as query rewriting, index selection, association elimination, etc. In addition, AST can also be used to generate execution plans and execute query operations.

[0088] The following example uses the time series database system to support three query syntaxes: InfluxQL, Flux, and SQL. Figure 5When Queriers receive a query request, they first parse the query statement into AST, and determine the data query scope and the timeline involved based on AST. Then, they request Absorbers to retrieve specific data files from the storage base based on the query scope and the timeline involved, and return them to the user.

[0089] 4) Absorbers

[0090] Absorbers is a collection of several Absorber microservice nodes, responsible for interacting with persistent data files. For example, Absorbers consumes Points obtained by Writers to form new data files. Persist new data files. And, according to the query scope and timeline involved obtained by Queriers, read the queried data files from the storage base.

[0091] WAL is a mechanism used in database systems to persist data and ensure the persistence and consistency of transactions. WAL files are special files used to store and manage WAL. In a database, when a transaction is executed, the transaction operations are usually written to the WAL file first, and then applied to the database's data files. This order is called "Write-Ahead".

[0092] In the specific implementation, Figure 6 As shown, Absorbers consumes Points data to form a new WAL (Write-AheadLog) persistence file, and retrieves data files from the storage base according to the query scope and the timeline involved determined by the AST, such as metadata and / or object storage data.

[0093] In addition, Absorbers store persistent data files on the file system of the Absorber microservice node so that Compactors compress the persistent data files and place the compressed data files in the storage base.

[0094] Absorbers store WAL files on the file system of the Absorber microservice node. Subsequently, Compactor compresses several WAL files into persistent files in columnar storage format and places them in the object storage service.

[0095] 5) Compactors

[0096] Compactors is a collection of several Compactor microservice nodes, responsible for compressing persistent files, optimizing storage file structure, and improving query efficiency and storage efficiency. Each time compression is started, files with low compression levels are compressed into files with high compression levels.

[0097] For example, in a time series database system, persistent files are divided into four levels:

[0098] L0: WAL files generated by the Absorber microservice node after consuming Points, stored in the local system.

[0099] L1: Several L0 files can be merged into one L1 file. L1 files are column-based storage files and are stored in the object storage service.

[0100] L2-L3: Similar to L1, it is formed by merging files from the previous level and stored in the object storage service. L3 has the highest compression level and the data is relatively old; L0 has the lowest compression level and the data is relatively new.

[0101] It can be seen that L0-level files are stored in local files in the running environment of each Absorber microservice node and cannot be accessed by other Absorber microservice nodes; L1-L3 are compressed by the Compactor microservice node, stored in the object storage service, and can be accessed by all Absorber microservice nodes. L0-level files do not require consistency, and write performance is prioritized; L1-L3 are stored in the object storage service, so consistency is maintained. When the Querier microservice node requests the Absorber microservice node, the Absorber microservice node retrieves data from the local WAL file and the object storage service. Depending on the requested Absorber microservice node, the retrieved WAL files are also inconsistent. Therefore, when the Querier queries, if the newly written data falls into the WAL file and is not compressed into an L1-level file by the Compactor microservice node, the data queried twice may be inconsistent; but in a short time, a new L1 file will be formed, and the queried data will maintain final consistency.

[0102] The WAL file of each Absorber microservice node takes the form of a rolling log, and the rolling time is configured by the user. This time window is the time for the data to ensure final consistency. That is, the WAL file will be generated in a rolling manner, and the local system will only retain the latest WAL file. The old WAL files will be merged into the L1 level file, which can be accessed by all Absorber microservice nodes.

[0103] The organic combination of WAL and object storage files balances the database's write performance and eventual consistency requirements.

[0104] Based on the above-mentioned time series database system, this implementation provides a method for constructing a time series database system. Through this method, the number of Absorber microservice nodes, Writer microservice nodes and Querier microservice nodes can be dynamically adjusted by the value of the performance indicator of the microservice cluster dynamically monitored by the self-monitoring and self-diagnosis module, so as to realize automatic scaling of the number of microservice nodes to adapt to the ever-changing load and traffic without user perception, thereby improving user experience.

[0105] Reference Figure 7 , shows a flowchart of a method for constructing a time series database system provided in an embodiment of the present invention, which may specifically include the following steps:

[0106] 701, obtaining the value of the performance indicator of the microservice cluster dynamically monitored by the self-monitoring and self-diagnosis module.

[0107] Among them, performance indicators include the total utilization of the central processing unit CPU and memory, the number of bytes written and read per second in the data directory, the amount of data received and written to the storage base, and the request response time.

[0108] 702, automatically scale the number of microservice nodes according to the value of the performance indicator.

[0109] If the total CPU and memory usage values ​​continue to be higher than the preset first usage threshold, or the data directory write and read bytes per second values ​​continue to be higher than the preset first ratio of the disk IO upper limit, or the value of the amount of data received and written to the storage base continues to be higher than the preset third ratio of the network IO upper limit, then increase the number of Absorber microservice nodes. If the total CPU and memory usage values ​​continue to be lower than the preset second usage threshold, or the data directory write and read bytes per second values ​​continue to be lower than the preset second ratio of the disk IO upper limit, or the value of the amount of data received and written to the storage base continues to be lower than the preset fourth ratio of the network IO upper limit, then reduce the number of Absorber microservice nodes.

[0110] If the request response time value continues to exceed the preset time, increase the number of Writer microservice nodes and Querier microservice nodes.

[0111] The duration is a preset value, such as 1 minute.

[0112] That is, in step 702, the number of microservice nodes is automatically scaled up or down according to whether the values ​​of the performance indicators of the microservice cluster dynamically monitored by the self-monitoring and self-diagnosis module obtained in step 701 meet the elastic expansion conditions and the elastic contraction conditions.

[0113] Taking the duration of 1 minute, the first usage rate threshold of 80%, the first ratio of 80%, the third ratio of 80%, the second usage rate threshold of 10%, the second ratio of 10%, the fourth ratio of 10%, and the preset duration of 500 milliseconds as an example, the above elastic expansion and elastic contraction conditions are explained again.

[0114] 1. Elastic expansion conditions

[0115] Automatic elastic expansion is performed when one of the following conditions is met:

[0116] 1) Node total CPU usage and node total memory usage: If the CPU and memory usage of a node is higher than 80% for more than one minute, it means that the node's computing resources may be insufficient, and you need to consider adding corresponding nodes to increase computing power and memory capacity.

[0117] 2) Node data directory write bytes per second and node data directory read bytes per second: If the node's data directory write and read bytes per second are higher than 80% of the disk IO limit for more than one minute, you need to add Absorber microservice nodes to improve disk throughput.

[0118] 3) The amount of data received by the node and the amount of data written to the object storage base: If the amount of data received by the node for write requests, the amount of data sent in response to queries, or the amount of data on the object storage base exceeds 80% of the network IO limit for more than one minute, you need to add Absorber microservice nodes to improve throughput.

[0119] 4) Node write request response time and node query request response time: If the node's write request response time or query request response time exceeds 500 milliseconds for more than one minute, it means that the node's processing capacity may be insufficient, and you need to consider expanding the capacity of the Writer microservice node and the Querier microservice node to improve the response performance.

[0120] 2. Elastic shrinkage conditions

[0121] When one of the following conditions is met, elastic shrinking is automatically performed:

[0122] 1) Node total CPU usage and node total memory usage: If the CPU and memory usage of a node is less than 10% for more than one minute, it means that the node has sufficient computing resources and you can consider reducing the corresponding nodes to save resources.

[0123] 2) Node data directory write bytes per second and node data directory read bytes per second: If the node's data directory write and read bytes per second are lower than 10% of the disk IO limit for more than one minute, you can reduce the number of Absorber microservice nodes.

[0124] 3) The amount of data received by the node and the amount of data written to the object storage base: If the amount of data received by the node for write requests, the amount of data sent in response to queries, or the amount of data on the object storage base is lower than 10% of the network IO limit for more than one minute, the number of Absorber microservice nodes can be reduced.

[0125] In addition, to ensure high availability of the database, the number of reduced Absorber microservice nodes is not less than the preset minimum node number threshold. If the minimum node number threshold is 3, then the number of reduced Absorber microservice nodes is not less than 3. At the same time, it can also ensure that the number of various microservice nodes is at least 3.

[0126] The method for constructing a time series database system provided in this embodiment can establish a self-diagnosis mechanism based on self-monitoring information, intelligently diagnose system conditions, and elastically scale the number of microservice nodes.

[0127] The following takes a microservice cluster including 3 Router microservice nodes, 3 Writer microservice nodes, 3 Querier microservice nodes, 3 Absorber microservice nodes, and 3 Compactor microservice nodes as an example to illustrate the various business processing processes of the time series database system.

[0128] 1. Processing write requests

[0129] The user sends a write request to the time series database system. Routers first verify the token information with the identity management module. At this time, the verification is passed and it is judged as a write request, and forwarded to Writers. After receiving the request, Writers judge it as a line protocol data format, further parse it into internally represented data points, and then send the Points to the message queue for consumption by Absorbers. Absorbers consume Points data to form a new WAL persistence file. At this time, Compactors periodically compress the generated WAL files and merge them into L1 level files.

[0130] 2. Processing query requests

[0131] The user sends an InfluxQL query request to the time series database system. The Routers first verify the token information with the identity management module. At this time, the verification is passed and it is judged as a query request and forwarded to the Queriers. After receiving the request, the Queriers process the query request, parse the InfluxQL query statement into AST, and determine the data query scope and the timeline involved based on AST. Then, by requesting the Absorbers to go to a specific file to retrieve specific data, and return it to the user.

[0132] 3. Self-monitoring and self-diagnosis

[0133] The user sent a large number of write requests, causing the total CPU usage and node memory usage of the Absorber microservice node to exceed 80%. The self-monitoring and self-diagnosis module in the time series database system detected that this situation lasted for more than one minute, and then dynamically pulled up a new Absorber microservice node and added it to the cluster, which accelerated the digestion of write points and relieved the write pressure. At this time, the number of Absorber microservice nodes in the database cluster is 4.

[0134] 4. Customization of peripheral systems

[0135] A user wants to use the minio storage base and OAuth2 identity authentication platform. Since the database of the time series database system is compatible with the S3 protocol, it is only necessary to configure the monio access token in the configuration file when starting the database, and the database can append the data compressed by the Compactor to the minio storage bucket. In addition, the time series database system provides a set of identity authentication interfaces to the outside world. As long as the external authentication platform implements this set of interfaces, it can switch to the OAuth2 identity authentication platform to manage the database of this system.

[0136] The time series database system constructed by the construction method of the time series database system provided by this embodiment is implemented based on the microservice architecture and can realize distributed services. The time series database system splits the database functions into a group of small services. Each service forms an independent module, such as dividing the database into microservice nodes such as Router, Writer, Querier, Absorber, etc., with a single function and customizable configuration, which can be independently deployed and elastically expanded. This split makes the database more modular, and can dynamically add and delete redundant services and node groups according to user needs, and customize database performance and functional requirements. The database based on the microservice architecture greatly improves the maintainability and flexibility of the system.

[0137] In addition, the time series database system constructed by the time series database system construction method provided by this embodiment separates the computing service and the storage service to achieve storage and computing separation. The upper layer of the time series database system is the microservice nodes that provide computing services, and the lower layer data storage base relies on the object storage service provided by cloud storage related companies. Since storage and computing are separated, resources can be flexibly configured and adjusted according to changes in data volume and computing requirements. This can improve resource utilization, avoid resource waste, and reduce cost overhead. In addition, storage and computing separation can achieve data isolation and protection. Data storage is managed independently to facilitate the protection of data confidentiality and integrity.

[0138] The time series database system constructed by the construction method of the time series database system provided by this embodiment can perform self-monitoring and self-diagnosis. Establish various monitoring indicators in advance, and dynamically monitor changes in indicators by integrating information such as CPU, memory, IO, etc., so as to confirm the current database status and flexibly expand various computing services to adapt to the ever-changing load and traffic. Through self-monitoring and self-diagnosis, performance is monitored in real time, and resources are occupied on demand, eliminating the tediousness and lag of manual expansion and contraction of users. Thanks to the superiority of the time series database for time series processing, the time series database system adopts a self-monitoring and self-diagnosis mode, establishes various monitoring indicators in advance, and dynamically monitors changes in indicators, so as to confirm the current database status and flexibly expand various computing services to adapt to the ever-changing load and traffic. Through the self-diagnosis function, the system computing resources can be automatically scaled without user perception, thereby improving the user experience.

[0139] The time series database system constructed by the construction method of the time series database system provided by this embodiment can realize the customization of peripheral services. In terms of unified identity authentication, the system kernel provides an authentication interface, and the external authentication platform is flexibly plugged in; in terms of the storage base, the system kernel is compatible with the S3 protocol, and the base can choose any object storage service compatible with this protocol. For example, if identity management customization is provided, the time series database system relies on the object storage base and the metadata storage service, and provides a comprehensive identity management interface for the time series database system and its corresponding dependencies, using token access and customized token permissions to achieve a secure and scalable identity management method. For another example, if storage base customization is provided, the time series database system is compatible with the storage base of the S3 protocol, and users can flexibly choose the storage base when actually using it.

[0140] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0141] This embodiment provides a method for constructing a time series database system, wherein the microservice cluster of the time series database system includes multiple Routers, multiple Writers, multiple Queriers, multiple Absorbers, and multiple Compactors; the method obtains the value of the performance indicator of the microservice cluster dynamically monitored by the self-monitoring and self-diagnosis module, and increases or decreases the number of Absorbers, Writers, and Queriers according to the value of the performance indicator. This method dynamically adjusts the number of Absorber microservice nodes, Writer microservice nodes, and Querier microservice nodes through the value of the performance indicator of the microservice cluster dynamically monitored by the self-monitoring and self-diagnosis module, and realizes automatic scaling of the number of microservice nodes to adapt to the ever-changing load and traffic, without user perception, thereby improving user experience.

[0142] Reference Figure 8 , shows a structural block diagram of a device for constructing a time series database system provided in an embodiment of the present invention. The time series database system includes: a microservice cluster, a self-monitoring and self-diagnosis module, an identity management module and a storage base.

[0143] The identity management module is used to perform user identity authentication and communication authentication between the microservice cluster and the storage base.

[0144] The microservice cluster is used to respond to user requests based on the authentication results of the identity management module and write data to the storage base or read data from the storage base.

[0145] The self-monitoring and self-diagnosis module is used to dynamically monitor the performance of the microservice cluster.

[0146] A microservice cluster includes: multiple microservice nodes. Among them, the microservice nodes include Router microservice nodes, Writer microservice nodes, Querier microservice nodes, Persistent Absorber microservice nodes, and Compactor microservice nodes. In addition, multiple Router microservice nodes constitute a routing microservice set Routers, multiple Writer microservice nodes constitute a writing microservice set Writers, multiple Querier microservice nodes constitute a query microservice set Queriers, multiple Absorber microservice nodes constitute a persistence microservice set Absorbers, and multiple Compactor microservice nodes constitute a compression microservice set Compactors.

[0147] Routers are used to determine the category of the user's request based on the authentication result of the identity management module, where the category is write or query. Requests with the write category are routed to Writers, and requests with the query category are routed to Queriers.

[0148] Writers are used to process write requests and form and persist new data files through Absorbers. The persisted data files are compressed by Compactors and placed in the storage base.

[0149] Queriers are used to process query requests and read query data files from the storage base through Absorbers.

[0150] The device provided in this embodiment may specifically include the following units:

[0151] The acquisition unit 801 is used to obtain the values ​​of performance indicators of the microservice cluster dynamically monitored by the self-monitoring and self-diagnosis module, wherein the performance indicators include the total utilization rate of the central processing unit CPU and memory, the number of bytes written and read per second in the data directory, the amount of data received and written to the storage base, and the request response time.

[0152] Construction unit 802 is used to increase the number of Absorber microservice nodes when the value of the total CPU and memory usage continues to be higher than the preset first usage threshold, or the value of the number of bytes written and read per second in the data directory continues to be higher than the preset first ratio of the disk IO upper limit, or the value of the amount of data received and written to the storage base continues to be higher than the preset third ratio of the network IO upper limit. If the value of the total CPU and memory usage continues to be lower than the preset second usage threshold, or the value of the number of bytes written and read per second in the data directory continues to be lower than the preset second ratio of the disk IO upper limit, or the value of the amount of data received and written to the storage base continues to be lower than the preset fourth ratio of the network IO upper limit, reduce the number of Absorber microservice nodes. When the value of the request response time continues to exceed the preset time, increase the number of Writer microservice nodes and Querier microservice nodes.

[0153] Among them, the number of reduced Absorber microservice nodes is not less than the preset minimum threshold of the number of nodes.

[0154] After receiving a write request, Writers determines the format of the written data.

[0155] If the format is the preset format, the written data is parsed into internally represented data points, and the Points are delivered to the absorbers for consumption through the message queue, so that the absorbers form and persist new data files.

[0156] The default formats are CSV and line protocol.

[0157] Among them, after receiving a request of query type, Queriers parses the query statement into an abstract syntax tree.

[0158] Determine the query scope and the timeline involved based on the abstract syntax tree.

[0159] Requests Absorbers to read the queried data files from the storage base according to the query scope and the timeline involved.

[0160] Feedback read data file.

[0161] Among them, Absorbers consume the Points obtained by Writers to form new data files, persist the new data files, and read the queried data files from the storage base according to the query scope and timeline involved obtained by Queriers.

[0162] After Absorbers persists the new data file, it also includes:

[0163] Absorbers store persistent data files on the file system of the Absorber microservice node so that Compactors compress the persistent data files and place the compressed data files in the storage base.

[0164] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0165] The device provided in this embodiment dynamically adjusts the number of Absorber microservice nodes, Writer microservice nodes, and Querier microservice nodes according to the value of the performance indicator of the microservice cluster dynamically monitored by the self-monitoring and self-diagnosis module, thereby realizing automatic scaling of the number of microservice nodes to adapt to the ever-changing load and traffic without user perception, thereby improving user experience.

[0166] In addition, an embodiment of the present invention further provides an electronic device, such as Fig. 9 As shown, it includes a processor 901, a communication interface 902, a memory 903 and a communication bus 904, wherein the processor 901, the communication interface 902, and the memory 903 communicate with each other through the communication bus 904.

[0167] Memory 903, used for storing computer programs;

[0168] The processor 901 is used to implement the steps of the method for constructing a time series database system when executing the program stored in the memory 903.

[0169] In the process of implementing the construction method of the time series database system, a time series database system is involved. The time series database system includes: a microservice cluster, a self-monitoring and self-diagnosis module, an identity management module and a storage base.

[0170] The identity management module is used to perform user identity authentication and communication authentication between the microservice cluster and the storage base.

[0171] The microservice cluster is used to respond to user requests based on the authentication results of the identity management module and write data to the storage base or read data from the storage base.

[0172] The self-monitoring and self-diagnosis module is used to dynamically monitor the performance of the microservice cluster.

[0173] A microservice cluster includes: multiple microservice nodes. Among them, the microservice nodes include Router microservice nodes, Writer microservice nodes, Querier microservice nodes, Persistent Absorber microservice nodes, and Compactor microservice nodes. In addition, multiple Router microservice nodes constitute a routing microservice set Routers, multiple Writer microservice nodes constitute a writing microservice set Writers, multiple Querier microservice nodes constitute a query microservice set Queriers, multiple Absorber microservice nodes constitute a persistence microservice set Absorbers, and multiple Compactor microservice nodes constitute a compression microservice set Compactors.

[0174] Routers are used to determine the category of the user's request based on the authentication result of the identity management module, where the category is write or query. Requests with the write category are routed to Writers, and requests with the query category are routed to Queriers.

[0175] Writers are used to process write requests and form and persist new data files through Absorbers. The persisted data files are compressed by Compactors and placed in the storage base.

[0176] Queriers are used to process query requests and read query data files from the storage base through Absorbers.

[0177] Specifically, the method for constructing a time series database system includes:

[0178] Get the values ​​of performance indicators of the microservice cluster dynamically monitored by the self-monitoring and self-diagnosis module. The performance indicators include the total usage of the central processing unit (CPU) and memory, the number of bytes written and read per second in the data directory, the amount of data received and written to the storage base, and the request response time.

[0179] If the total CPU and memory usage values ​​continue to be higher than the preset first usage threshold, or the data directory write and read bytes per second values ​​continue to be higher than the preset first ratio of the disk IO upper limit, or the value of the amount of data received and written to the storage base continues to be higher than the preset third ratio of the network IO upper limit, then increase the number of Absorber microservice nodes. If the total CPU and memory usage values ​​continue to be lower than the preset second usage threshold, or the data directory write and read bytes per second values ​​continue to be lower than the preset second ratio of the disk IO upper limit, or the value of the amount of data received and written to the storage base continues to be lower than the preset fourth ratio of the network IO upper limit, then reduce the number of Absorber microservice nodes.

[0180] If the request response time value continues to exceed the preset time, increase the number of Writer microservice nodes and Querier microservice nodes.

[0181] Optionally, the number of reduced Absorber microservice nodes is not less than a preset minimum threshold of the number of nodes.

[0182] Optionally, upon receiving a request of type write, Writers determine the format of the written data.

[0183] If the format is the preset format, the written data is parsed into internally represented data points, and the Points are delivered to the absorbers for consumption through the message queue, so that the absorbers form and persist new data files.

[0184] Optionally, the preset formats are CSV and line protocol formats.

[0185] Optionally, after receiving a request of query type, Queriers parses the query statement into an abstract syntax tree.

[0186] Determine the query scope and the timeline involved based on the abstract syntax tree.

[0187] Requests Absorbers to read the queried data files from the storage base according to the query scope and the timeline involved.

[0188] Feedback read data file.

[0189] Optionally, Absorbers consume Points obtained by Writers to form new data files, persist new data files, and read the queried data files from the storage base according to the query scope and timeline involved obtained by Queriers.

[0190] Optionally, after the absorbers persist the new data files, they also include:

[0191] Absorbers store persistent data files on the file system of the Absorber microservice node so that Compactors compress the persistent data files and place the compressed data files in the storage base.

[0192] The communication bus mentioned in the above terminal can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0193] The communication interface is used for communication between the above terminal and other devices.

[0194] The memory may include a random access memory (RAM) or a non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0195] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0196] The electronic device provided in this embodiment dynamically adjusts the number of Absorber microservice nodes, Writer microservice nodes, and Querier microservice nodes according to the value of the performance indicator of the microservice cluster dynamically monitored by the self-monitoring and self-diagnosis module, thereby realizing automatic scaling of the number of microservice nodes to adapt to the ever-changing load and traffic without user perception, thereby improving user experience.

[0197] like Fig.10 As shown, in another embodiment provided by the present invention, a computer-readable storage medium 1001 is also provided, in which instructions are stored. When the computer-readable storage medium 1001 is run on a computer, the computer executes the method for constructing a time series database system described in the above embodiment.

[0198] In another embodiment provided by the present invention, a computer program product including instructions is also provided, which, when executed on a computer, enables the computer to execute the method for constructing a time series database system described in the above embodiment.

[0199] Among them, the time series database system includes: microservice cluster, self-monitoring and self-diagnosis module, identity management module and storage base.

[0200] The identity management module is used to perform user identity authentication and communication authentication between the microservice cluster and the storage base.

[0201] The microservice cluster is used to respond to user requests based on the authentication results of the identity management module and write data to the storage base or read data from the storage base.

[0202] The self-monitoring and self-diagnosis module is used to dynamically monitor the performance of the microservice cluster.

[0203] A microservice cluster includes: multiple microservice nodes. Among them, the microservice nodes include Router microservice nodes, Writer microservice nodes, Querier microservice nodes, Persistent Absorber microservice nodes, and Compactor microservice nodes. In addition, multiple Router microservice nodes constitute a routing microservice set Routers, multiple Writer microservice nodes constitute a writing microservice set Writers, multiple Querier microservice nodes constitute a query microservice set Queriers, multiple Absorber microservice nodes constitute a persistence microservice set Absorbers, and multiple Compactor microservice nodes constitute a compression microservice set Compactors.

[0204] Routers are used to determine the category of the user's request based on the authentication result of the identity management module, where the category is write or query. Requests with the write category are routed to Writers, and requests with the query category are routed to Queriers.

[0205] Writers are used to process write requests and form and persist new data files through Absorbers. The persisted data files are compressed by Compactors and placed in the storage base.

[0206] Queriers are used to process query requests and read query data files from the storage base through Absorbers.

[0207] Specifically, the method includes:

[0208] Get the values ​​of performance indicators of the microservice cluster dynamically monitored by the self-monitoring and self-diagnosis module. The performance indicators include the total usage of the central processing unit (CPU) and memory, the number of bytes written and read per second in the data directory, the amount of data received and written to the storage base, and the request response time.

[0209] If the total CPU and memory usage values ​​continue to be higher than the preset first usage threshold, or the data directory write and read bytes per second values ​​continue to be higher than the preset first ratio of the disk IO upper limit, or the value of the amount of data received and written to the storage base continues to be higher than the preset third ratio of the network IO upper limit, then increase the number of Absorber microservice nodes. If the total CPU and memory usage values ​​continue to be lower than the preset second usage threshold, or the data directory write and read bytes per second values ​​continue to be lower than the preset second ratio of the disk IO upper limit, or the value of the amount of data received and written to the storage base continues to be lower than the preset fourth ratio of the network IO upper limit, then reduce the number of Absorber microservice nodes.

[0210] If the request response time value continues to exceed the preset time, increase the number of Writer microservice nodes and Querier microservice nodes.

[0211] Optionally, the number of reduced Absorber microservice nodes is not less than a preset minimum threshold of the number of nodes.

[0212] Optionally, upon receiving a request of type write, Writers determine the format of the written data.

[0213] If the format is the preset format, the written data is parsed into internally represented data points, and the Points are delivered to the absorbers for consumption through the message queue, so that the absorbers form and persist new data files.

[0214] Optionally, the preset formats are CSV and line protocol formats.

[0215] Optionally, after receiving a request of query type, Queriers parses the query statement into an abstract syntax tree.

[0216] Determine the query scope and the timeline involved based on the abstract syntax tree.

[0217] Requests Absorbers to read the queried data files from the storage base according to the query scope and the timeline involved.

[0218] Feedback read data file.

[0219] Optionally, Absorbers consume Points obtained by Writers to form new data files, persist new data files, and read the queried data files from the storage base according to the query scope and timeline involved obtained by Queriers.

[0220] Optionally, after the absorbers persist the new data files, they also include:

[0221] Absorbers store persistent data files on the file system of the Absorber microservice node so that Compactors compress the persistent data files and place the compressed data files in the storage base.

[0222] The computer-readable storage medium provided in this embodiment dynamically adjusts the number of Absorber microservice nodes, Writer microservice nodes, and Querier microservice nodes according to the value of the performance indicator of the microservice cluster dynamically monitored by the self-monitoring and self-diagnosis module, thereby realizing automatic scaling of the number of microservice nodes to adapt to the ever-changing load and traffic without user perception, thereby improving user experience.

[0223] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk Solid State Disk (SSD)), etc.

[0224] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0225] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0226] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. A method for constructing a time series database system, characterized in that: The time series database system includes: a microservice cluster, a self-monitoring and self-diagnosis module, an identity management module and a storage base; The identity management module is used to perform user identity authentication and communication authentication between the microservice cluster and the storage base; The microservice cluster is used to respond to a user's request based on the authentication result of the identity management module and write data to the storage base or read data from the storage base; The self-monitoring and self-diagnosis module is used to dynamically monitor the performance of the microservice cluster; The microservice cluster includes: a plurality of microservice nodes; wherein the microservice nodes include a routing Router microservice node, a writing Writer microservice node, a query Querier microservice node, a persistence Absorber microservice node, and a compression Compactor microservice node; and, a plurality of Router microservice nodes constitute a routing microservice set Routers, a plurality of Writer microservice nodes constitute a writing microservice set Writers, a plurality of Querier microservice nodes constitute a query microservice set Queriers, a plurality of Absorber microservice nodes constitute a persistence microservice set Absorbers, and a plurality of Compactor microservice nodes constitute a compression microservice set Compactors; The Routers are used to determine the category of the user's request based on the authentication result of the identity management module, where the category is writing or querying; routing the request of the category of writing to Writers, and routing the request of the category of query to Queriers; The Writers are used to process write requests and form and persist new data files through the Absorbers; wherein the persisted data files are compressed by the Compactors and placed in the storage base; The Queriers are used to process the request of the query type, and read the queried data file from the storage base through the Absorbers; The method comprises: Obtain the values ​​of the performance indicators of the microservice cluster dynamically monitored by the self-monitoring and self-diagnosis module; wherein the performance indicators include the total utilization rate of the central processing unit CPU and memory, the number of bytes written and read per second in the data directory, the amount of data received and written to the storage base, and the request response time; If the total CPU and memory usage values ​​continue to be higher than the preset first usage threshold, or the values ​​of the number of bytes written and read per second in the data directory continue to be higher than the preset first ratio of the disk IO upper limit, or the value of the amount of data received and written to the storage base continues to be higher than the preset third ratio of the network IO upper limit, then increase the number of Absorber microservice nodes; if the total CPU and memory usage values ​​continue to be lower than the preset second usage threshold, or the values ​​of the number of bytes written and read per second in the data directory continue to be lower than the preset second ratio of the disk IO upper limit, or the value of the amount of data received and written to the storage base continues to be lower than the preset fourth ratio of the network IO upper limit, then reduce the number of Absorber microservice nodes; If the request response time value continues to exceed the preset time, increase the number of Writer microservice nodes and Querier microservice nodes.

2. The method according to claim 1, characterized in that The number of reduced Absorber microservice nodes is not less than the preset minimum threshold of the number of nodes.

3. The method according to claim 1, characterized in that After receiving a write request, the Writers determine the format of the written data; If the format is a preset format, the written data is parsed into internally represented data points Points, and the Points are delivered to the absorbers for consumption through a message queue, so that the absorbers form and persist new data files.

4. The method according to claim 3, characterized in that The preset formats are CSV and line protocol formats.

5. The method according to claim 1, characterized in that After receiving a request of the query type, the Queriers parses the query statement into an abstract syntax tree; Determine the query scope and the timeline involved according to the abstract syntax tree; Requesting Absorbers to read the queried data files from the storage base according to the query scope and the timeline involved; Feedback read data file.

6. The method according to claim 1, characterized in that The Absorbers consume the Points obtained by the Writers to form new data files; persist the new data files; and read the queried data files from the storage base according to the query scope and the timeline involved obtained by the Queriers.

7. The method according to claim 6, characterized in that After the absorbers persist the new data file, the method further includes: The Absorbers store the persistent data files on the file system of the Absorber microservice node, so that the Compactors compress the persistent data files and place the compressed data files in the storage base.

8. A device for constructing a time series database system, characterized in that: The time series database system includes: a microservice cluster, a self-monitoring and self-diagnosis module, an identity management module and a storage base; The identity management module is used to perform user identity authentication and communication authentication between the microservice cluster and the storage base; The microservice cluster is used to respond to a user's request based on the authentication result of the identity management module and write data to the storage base or read data from the storage base; The self-monitoring and self-diagnosis module is used to dynamically monitor the performance of the microservice cluster; The microservice cluster includes: a plurality of microservice nodes; wherein the microservice nodes include a routing Router microservice node, a writing Writer microservice node, a query Querier microservice node, a persistence Absorber microservice node, and a compression Compactor microservice node; and, a plurality of Router microservice nodes constitute a routing microservice set Routers, a plurality of Writer microservice nodes constitute a writing microservice set Writers, a plurality of Querier microservice nodes constitute a query microservice set Queriers, a plurality of Absorber microservice nodes constitute a persistence microservice set Absorbers, and a plurality of Compactor microservice nodes constitute a compression microservice set Compactors; The Routers are used to determine the category of the user's request based on the authentication result of the identity management module, where the category is writing or querying; routing the request of the category of writing to Writers, and routing the request of the category of query to Queriers; The Writers are used to process write requests and form and persist new data files through the Absorbers; wherein the persisted data files are compressed by the Compactors and placed in the storage base; The Queriers are used to process the request of the query type, and read the queried data file from the storage base through the Absorbers; The device comprises: An acquisition unit is used to acquire the values ​​of performance indicators of the microservice cluster dynamically monitored by the self-monitoring and self-diagnosis module; wherein the performance indicators include the total utilization rate of the central processing unit CPU and memory, the number of bytes written and read per second in the data directory, the amount of data received and written to the storage base, and the request response time; A construction unit is used to increase the number of Absorber microservice nodes when the total CPU and memory usage values ​​are continuously higher than a preset first usage threshold, or the number of bytes written and read per second of the data directory is continuously higher than a preset first proportion of the disk IO upper limit, or the amount of data received and written to the storage base is continuously higher than a preset third proportion of the network IO upper limit; reduce the number of Absorber microservice nodes if the total CPU and memory usage values ​​are continuously lower than a preset second usage threshold, or the number of bytes written and read per second of the data directory is continuously lower than a preset second proportion of the disk IO upper limit, or the amount of data received and written to the storage base is continuously lower than a preset fourth proportion of the network IO upper limit; and increase the number of Writer microservice nodes and Querier microservice nodes when the request response time value continuously exceeds a preset time.

9. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; The memory is used to store computer programs; The processor is used to implement the method according to any one of claims 1 to 7 when executing the program stored in the memory.

10. A computer-readable medium having instructions stored thereon, which, when executed by one or more processors, cause the processors to perform the method according to any one of claims 1 to 7.

Images