Time-series database cluster construction method, data storage method, and related apparatuses

Abstract

The application discloses a time sequence database cluster construction method, a data storage method and related devices. The method comprises the following steps: a target node acquires an image of an influx-proxy service and an image of an influxDB service; the target node creates an influx-proxy container on the target node according to the image of the influx-proxy service; and the target node creates an influxDB container on the target node according to the image of the influxDB service through the influx-proxy container. The method is beneficial to realizing lightweight deployment of the influxDB cluster and reducing the difficulty of capacity expansion.

Description

Technical Field

[0001] This application relates to the field of computers, and in particular to a method for constructing a time-series database cluster, a data storage method, and related apparatus. Background Technology

[0002] InfluxDB is an open-source time-series database developed by InfluxData. Written in Go, it focuses on high-performance querying and storage of time-series data. InfluxDB is widely used in scenarios such as storing monitoring data for systems and real-time data in the IoT industry. Influx Proxy is a highly available, consistent hashing-based InfluxDB cluster proxy service that implements a high-availability InfluxDB cluster deployment solution, featuring dynamic scaling, fault recovery, and data synchronization capabilities.

[0003] Currently, there are two methods for building an InfluxDB cluster: one is to build an InfluxDB cluster based on a Kubernetes cluster; the other is to build an InfluxDB cluster using the native Docker Run method. Building an InfluxDB cluster based on a Kubernetes cluster requires installing numerous components and performing complex configurations, and is not lightweight. Building an InfluxDB cluster using the native Docker Run method is not easily scalable and cannot achieve high availability and high reliability across nodes. Summary of the Invention

[0004] This application provides a method for constructing a time-series database cluster, a data storage method, and related apparatus, which facilitates the lightweight deployment of time-series database clusters and reduces the difficulty of expansion.

[0005] In a first aspect, embodiments of this application provide a method for constructing a time-series database cluster, the method comprising:

[0006] The target node obtains the images of the cluster agent service and the time series database service. Based on the image of the cluster agent service, the target node creates a cluster agent container on the target node. The target node then uses the cluster agent container to create a time series database container on the target node based on the image of the time series database service. The time series database container is used to build the time series database cluster.

[0007] The target node can be a node in a server cluster such as a Swarm cluster. In one example, the target node can be the management node in a Swarm cluster. Swarm is a cluster management system, a software used to implement cluster management functions.

[0008] In conjunction with the first aspect, in one possible implementation, the cluster agent container includes a cluster agent starter. The target node uses the cluster agent container to create a time-series database container on the target node based on the image of the time-series database service, including:

[0009] The target node calls the Swarm application programming interface (API) of the target node through the cluster agent startup program to create the target node's time series database service; the target node uses the image of the time series database service to create the time series database container on the target node according to the parameters used to create the time series database container in the time series database service.

[0010] In conjunction with the first aspect, in one possible embodiment, the method of this application further includes:

[0011] The target node starts the program through the cluster agent to check whether the time series database service has been created on the target node; if it is determined that the time series database service has not been created on the target node, the target node creates the time series database service on the target node through the cluster agent container based on the image of the time series database service.

[0012] In conjunction with the first aspect, in one possible implementation, the parameters used to create the time-series database container are passed to Swarm via Swarm's socket channel.

[0013] In conjunction with the first aspect, in one possible implementation, embodiments of this application further include:

[0014] The target node obtains the time-series database service information in the target node through the cluster agent startup program in the cluster agent container. The target node then configures the cluster agent open-source software in the cluster agent container according to the time-series database service information in the target node, and starts the cluster agent open-source software in the cluster agent container to allocate the time-series database service in the target node to a hash ring.

[0015] In conjunction with the first aspect, one possible implementation involves the target node mounting the persistent data directory within the time-series database container to the persistent data directory on the host side using the container's mount parameters. This approach avoids data loss after a container restart.

[0016] In conjunction with the first aspect, in one possible implementation, embodiments of this application further include:

[0017] The target node maps the port of the cluster agent container in Swarm to the host side. By mapping the port of the cluster agent container to the host side, processes both inside and outside the container can access the cluster agent service.

[0018] In conjunction with the first aspect, in one possible implementation, containers in a time-series database cluster are based on overlay network communication.

[0019] Secondly, embodiments of this application provide a target node, including:

[0020] The acquisition unit is used to acquire the images of the cluster agent service and the time-series database service.

[0021] The creation unit is used to create a cluster proxy container on the target node based on the image of the cluster proxy service. The target node creates a time series database container on the target node based on the image of the time series database service through the influx-proxy container. The time series database container is used to build a time series database cluster.

[0022] In conjunction with the second aspect, in one possible implementation, the cluster agent container includes a cluster agent starter. Specifically, in the aspect of creating a time-series database container on the target node based on the image of the time-series database service using the cluster agent container, the creation unit is used for:

[0023] The cluster agent starts the program by calling the Swarm API of the target node to create a time-series database service for the target node; and then uses the image of the time-series database service to create a time-series database container on the target node according to the parameters used to create the time-series database container in the time-series database service.

[0024] In conjunction with the second aspect, in one possible embodiment, the creation unit is further configured to:

[0025] The cluster agent startup program checks whether a time-series database service has been created on the target node. If it is determined that a time-series database service has not been created on the target node, the cluster agent startup program calls the Swarm API of the target node to create the time-series database service on the target node.

[0026] In conjunction with the second aspect, in one possible implementation, the parameters used to create the time-series database container are passed to the target node's Swarm via a Swarm socket channel.

[0027] In conjunction with the second aspect, in one possible implementation,

[0028] The acquisition unit is also used to obtain time-series database service information from the target node through the cluster agent startup program in the cluster agent container.

[0029] The target nodes also include:

[0030] The configuration unit is used to configure the cluster agent open-source software in the cluster agent container of the target node according to the time-series database service information in the target node through the cluster agent startup program, and start the cluster agent open-source software in the cluster agent container to allocate the time-series database service in the target node to a hash ring.

[0031] In conjunction with the second aspect, one possible implementation involves mounting the persistent data directory within the time-series database container to the persistent data directory on the host side using the mount parameters of the time-series database container.

[0032] In conjunction with the second aspect, in one possible implementation, the target node also includes:

[0033] The mapping unit is used to map the ports of the cluster agent containers in Swarm to the host side.

[0034] In conjunction with the second aspect, in one possible implementation, containers in a time-series database cluster are based on overlay network communication.

[0035] Thirdly, embodiments of this application provide a data storage method applied to a time-series database cluster created using the first aspect or any implementation thereof. The time-series database cluster includes at least one cluster proxy container and at least one hash ring, each hash ring including at least one time-series database container. The method of this embodiment includes:

[0036] The target node receives a write data request, which includes the data to be written. The target node contains a cluster proxy container and at least one time-series database container. The target node will send the write data request to at least one time-series database container in a hash ring through at least one cluster proxy container based on a load balancing strategy, so that the time-series database container that receives the write data request will write the data to be written.

[0037] It should be noted that the beneficial effects that the third aspect can achieve can be referred to the beneficial effects in the first aspect, and will not be repeated here.

[0038] Fourthly, embodiments of this application provide a target node, including a processor and a memory. The memory is used to store program code. The processor is used to invoke the program code stored in the memory to execute the method provided in the first aspect, the second aspect, or any possible implementation of the first aspect.

[0039] Fifthly, embodiments of this application provide a server cluster, which includes multiple server nodes. For example, in the first aspect, any possible implementation of the first aspect, or the third aspect, the target node is one of the multiple server nodes, or a virtual machine among the multiple server nodes.

[0040] In a sixth aspect, embodiments of this application provide a computer storage medium including computer instructions that, when executed on an electronic device, cause the electronic device to perform a method as provided in the first aspect, the third aspect, or any possible implementation of the first aspect.

[0041] In a seventh aspect, embodiments of this application provide a computer program product that, when run on a computer, causes the computer to perform a method provided as described in the first aspect, the third aspect, or any possible implementation of the first aspect.

[0042] Understandably, the target node described in the second aspect, the target node described in the fourth aspect, the server cluster described in the fifth aspect, the computer storage medium described in the sixth aspect, and the computer program product described in the seventh aspect are all used to implement any of the methods provided in the first aspect. Therefore, the beneficial effects they can achieve can be referred to in the beneficial effects of the corresponding methods, and will not be repeated here. Attached Figure Description

[0043] Figure 1a This is a schematic diagram of the Swarm cluster architecture;

[0044] Figure 1b This is a schematic diagram of the InfluxDB cluster architecture;

[0045] Figure 2 A flowchart illustrating an InfluxDB cluster construction method provided in this application embodiment;

[0046] Figure 3 A flowchart illustrating another method for building an InfluxDB cluster provided in this application embodiment;

[0047] Figure 4 A schematic diagram of a node provided in an embodiment of this application;

[0048] Figure 5 A schematic diagram of an InfluxDB cluster architecture provided in this application embodiment;

[0049] Figure 6 This is a schematic diagram of the structure of a target node provided in an embodiment of this application;

[0050] Figure 7 This is a schematic diagram of the structure of a server node provided in an embodiment of this application. Detailed Implementation

[0051] The terms "first," "second," "third," and "fourth," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0052] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0053] "Multiple" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0054] First, the relevant terms used in this application will be explained.

[0055] A container, a virtualized operating system, is implemented through namespace isolation in the Linux kernel; essentially, it is a highly isolated process. Linux is an operating system.

[0056] A service image is a static representation of a container, and its relationship with a container is similar to that of a software package and a software process.

[0057] Swarm is a cluster management system, a software that provides cluster management capabilities.

[0058] An influxDB service refers to a group of influxDB containers that provide the same business capabilities, collectively offering services and possessing scalability. The definition of an influxDB service includes the following information: the influxDB service name, the image used to create the influxDB containers, the number of influxDB containers to run, and the network on which the influxDB containers will connect.

[0059] An influx-proxy service refers to a group of influx-proxy containers that provide the same business capabilities, can jointly provide business services, and have scalability. The definition of an influx-proxy service includes the following information: the influx-proxy service name, the image used to create the influx-proxy containers, the number of influx-proxy containers to run, and the network on which the influx-proxy containers should be connected.

[0060] influx-proxy is an open-source software used to provide the ability to cluster InfluxDB.

[0061] Overlay networks are a type of container network provided by Swarm. They are isolated from the host network and enable network communication between containers across nodes.

[0062] The following examples, using InfluxDB in a time-series database, the Influx-proxy service in a cluster proxy service, and the Influx-proxy container in a cluster proxy container, illustrate the solution in this application. Correspondingly, the open-source cluster proxy software can be the Influx-proxy open-source software, and the cluster proxy startup program can be the Influx-proxy startup program.

[0063] The embodiments of this application will now be described with reference to the accompanying drawings.

[0064] See Figure 1a , Figure 1a This is a schematic diagram of a Swarm cluster architecture. Figure 1a As shown, the architecture includes at least one management node and at least one worker node.

[0065] The management node can be a physical machine or a virtual machine; the worker node can be a physical machine or a virtual machine.

[0066] The management node is responsible for managing the entire Swarm cluster, including cluster configuration, service management, container orchestration, and all other cluster-related tasks. These services may include influx-proxy and influxDB. When there are multiple management nodes, one is elected as the master node. The master node is responsible for container orchestration. The gossip network is used to implement the gossip protocol, which is used for Swarm cluster election, such as electing a master node from multiple management nodes.

[0067] Worker nodes receive and execute tasks assigned by management nodes, running on the corresponding services.

[0068] A task is essentially a container and the commands that run on that container. There is a one-to-one correspondence between tasks and containers, and they run on the assigned nodes.

[0069] See Figure 1b , Figure 1b This is a schematic diagram of an InfluxDB cluster architecture. Figure 1b As shown, the architecture includes a client, at least one influx-proxy container, and at least one hash ring. Each hash ring includes at least one influxDB container. The at least one hash ring resides on a different physical machine; when multiple hash rings exist, they serve as backups for each other. That is, the data stored across multiple hash rings is identical. For example, hash ring A includes influxDB1, influxDB2, and influxDB3 containers, and hash ring B also includes influxDB1, influxDB2, and influxDB3 containers. Specifically, the data stored in influxDB1 container in hash ring A is the same as the data stored in influxDB1 container in hash ring B; the data stored in influxDB2 container in hash ring A is the same as the data stored in influxDB2 container in hash ring B; and the data stored in influxDB3 container in hash ring A is the same as the data stored in influxDB3 container in hash ring B. When the data stored in influxDB3 container in hash ring A changes, the data stored in influxDB3 container in hash ring B changes accordingly.

[0070] Clients can query data stored in the InfluxDB container within the hash ring or write data to the InfluxDB container within the hash ring through the Influx-proxy container. Communication between the client and the Influx-proxy container is based on an overlay network. Specifically, the client sends query or write commands to the Influx-proxy container. Upon receiving the command, the Influx-proxy container executes the corresponding operation. Optionally, when multiple Influx-proxy containers exist, Swarm will send query or write commands to them according to a load balancing strategy to ensure that the load on the multiple Influx-proxy containers meets preset conditions. These preset conditions may include the difference in the number of commands to be executed on any pair of Influx-proxy containers being less than a certain threshold.

[0071] When multiple influx-proxy containers exist, their functions serve as backups for each other. In other words, if one influx-proxy container fails, the others can continue to function normally.

[0072] See Figure 2 , Figure 2 This is a flowchart illustrating an InfluxDB cluster construction method provided in an embodiment of this application. Figure 2 As shown, the method includes:

[0073] S201. The target node obtains the images of the influx-proxy service and the influxDB service.

[0074] The influx-proxy service image includes the influx-proxy program and its corresponding startup program. This startup program is used to query the influxDB service information on the target node, configure, and start the influx-proxy open-source software.

[0075] InfluxDB is a time-series database. influx-proxy is used to provide the ability to cluster InfluxDB databases.

[0076] The target node is a node in a Swarm cluster (or server cluster, etc.). In a Swarm cluster, a node can be a physical machine or a virtual machine.

[0077] via Docker networkcreate <network-name>The `-d overlay` command creates an overlay network. Overlay networks are used for communication between nodes in a Swarm cluster and between containers within those nodes.

[0078] S202. The target node loads the images of the influx-proxy service and the influxDB service.

[0079] Specifically, the target node loads the service image using the command `docker load -i <service image path>`.

[0080] It should be understood that before loading the influx-proxy and influxDB service images, the target node needs to complete the runtime environment preparation. Specifically, Docker software needs to be installed on the target node's operating system, and an IP address needs to be configured to ensure proper network connectivity. This operating system is a common Linux system, such as Debian, Gentoo, or Ubuntu.

[0081] Enable Swarm mode using the `docker swarminit` command to make the target node a management node of the Swarm cluster.

[0082] Deploy Docker on the operating systems of the other nodes and configure IP addresses to ensure normal network connectivity. The other nodes join the Swarm cluster as management nodes by running the `docker swarmjoin` command. The target node is the master node.

[0083] S203. The target node creates an influx-proxy container on the target node based on the image of the influx-proxy service.

[0084] Specifically, the target node creates its corresponding influx-proxy service through its API. The parameters in the influx-proxy service define the service name, the image used to create the influx-proxy containers, the number of influx-proxy containers to run, and the network on which the influx-proxy containers should connect. Based on the relevant parameters in the influx-proxy service, the target node determines the image of the influx-proxy service that supports it and creates the influx-proxy containers on the target node according to that image. The target node then establishes a network connection for the influx-proxy containers based on the relevant parameters in the influx-proxy service.

[0085] It should be noted here that for the purpose of illustration, this application only gives the image of the influx-proxy service. Of course, other images may also be included to support the influx-proxy service, which is not limited here.

[0086] In one example, the influx-proxy service can be created by the following command:

[0087] docker service create

[0088] --name InfluxProxyService\

[0089] --hostname={{.Node.ID}}-{{.Task.ID}}\

[0090] --network<overlay network>\

[0091] --mode global\

[0092] Among them, nameInfluxProxyService indicates that the name of the created service is: InfluxProxyService; hostname={{.Node.ID}}-{{.Task.ID}} indicates that the hostname of the operating system of the created service is {{.Node.ID}}-{{.Task.ID}}. In this embodiment, Node.ID is the ID of the target node, and Task.ID is the ID of the task that needs to call this service in the target node; network<overlay network name> indicates connecting the created service to the overlay network; mode global indicates creating a container on all nodes in the Swarm cluster.

[0093] The influx-proxy service includes two key programs. One is the influx-proxy startup program, which has two functions: 1) used to configure and start the influx-proxy open-source software in the same container; 2) create an influxDB service belonging to the current node. The other is the influx-proxy open-source software, which is configured and started by the startup program in the same container and cooperates with the influxDB container to form an influxDB cluster.

[0094] After creating the influx-proxy service on the target node, the target node creates an influx-proxy container on the target node according to the parameters of the influx-proxy service and the image of the influx-proxy service.

[0095] S204. The target node creates an influxDB container on the target node based on the image of the influxDB service using the influx-proxy container.

[0096] In one possible implementation, the influx-proxy container includes the influx-proxy starter. The target node uses the influx-proxy container to create an influxDB container on the target node based on the image of the influxDB service, including:

[0097] The target node uses the influx-proxy startup program to call the target node's Swarm API to create the target node's influxDB service. The target node then creates the influxDB container on the target node based on the parameters used to create the influxDB container in the influxDB service and the image of the influxDB service.

[0098] The influxDB service defines the service name, which image to use to create the influxDB container, the number of influxDB containers to run, and which network the influxDB containers should connect to.

[0099] The target node determines the image of the InfluxDB service that supports the InfluxDB service based on relevant parameters in the InfluxDB service, and creates an InfluxDB container on the target node according to the InfluxDB service image. The target node then performs network connection and port mapping for the InfluxDB container based on relevant parameters in the InfluxDB service.

[0100] In one possible implementation, before the target node creates an influxDB container on the target node based on the image of the influxDB service using the influx-proxy container, the method of this embodiment further includes:

[0101] The target node uses the influx-proxy startup program to query whether the influxDB service has been created on the target node; if it is determined that the influxDB service has not been created on the target node, the target node uses the influx-proxy startup program to call the target node's Swarm API to create the influxDB service on the target node.

[0102] In one possible implementation, the method of this embodiment further includes:

[0103] The target node obtains the influxDB service information in the target node through the influx-proxy starter in the influx-proxy container. The target node then configures the open-source influx-proxy software in the influx-proxy container based on the influxDB service information in the target node, and starts the open-source influx-proxy software in the influx-proxy container to allocate the influxDB container in the target node to different hash rings.

[0104] The InfluxDB service contains a node ID, and InfluxDB services located on the same hash ring have the same node ID.

[0105] Specifically, such as Figure 3 As shown:

[0106] S301, the influx-proxy startup program obtains the ID of the node where the container is located by using the hostname.

[0107] Here, "this container" refers to the container to which the influx-proxy startup program belongs, that is, the influx-proxy container mentioned above. The ID of the node where this container is located is the ID of the target node.

[0108] After the influx-proxy container is created, the influx-proxy startup program in the influx-proxy container on the target node obtains the ID of the node where the influx-proxy container is located by the hostname.

[0109] S302 The influx-proxy startup program obtains the influxDB service information on the target node based on the target node's ID. If there is no influxDB service information, it will create it.

[0110] Specifically, the influx-proxy startup program retrieves the influxDB service information on the target node via a socket channel based on the target node's ID. If no influxDB service information is found, it means that the influxDB service has not been created on the target node. The influx-proxy startup program then transmits the parameters used to create the influxDB service to Swarm.

[0111] S303. The target node's Swarm creates the influxDB service based on the parameters passed to the influx-proxy startup program.

[0112] The parameters used to create the InfluxDB service include the service name, the name of the connected network, the node ID, the mounted directory name, and the name of the InfluxDB service image. It should be understood that other parameters can also be included when creating the InfluxDB service; this is not limited here.

[0113] like Figure 3 As shown, the InfluxDB service can be created using the following command:

[0114] dockerservicecreate\

[0115] --network <network-name>\

[0116] --mount type=bind, source=<Host-side data persistence directory>, destination=<In-container data persistence directory>, bind-propagation=reslave\

[0117] --name influxDB-<Node ID>\

[0118] --constraint node.id=<Node ID>

[0119] <InfluxDB service image name>

[0120] Among them, network <network-name>This indicates the network to which the InfluxDB service connects.

[0121] The `mount type=bind, source=<host-side persistent data directory>, destination=<container-side persistent data directory>, bind-propagation=reslave` option mounts the container's persistent data directory to the host-side persistent data directory, ensuring persistent data is stored on the host and preventing data loss after container restarts. For example, the host-side persistent data directory might be ` / var / run / docker.sock`, and ` / var / run / docker.sock` might be the container's persistent data directory.

[0122] The nameinfluxDB-<nodeID> is used to name the InfluxDB service "influxDB-<nodeID>".

[0123] The `--constraint node.id=<nodeID>` option means that Swarm will only create the InfluxDB container on the node named `<nodeID>`.

[0124] After the InfluxDB service is created on the target node, the target node creates an InfluxDB container based on the parameters of the InfluxDB service and the image of the InfluxDB service.

[0125] It should be understood that after the target node has created the influxDB service, the influx-proxy startup program obtains the influxDB service information on the target node through a socket channel.

[0126] S304, the influx-proxy startup program obtains the influxDB service information of the target node.

[0127] Following the above method, the influx-proxy startup program on each node in the Swarm cluster can obtain its own influxDB service information, which will not be described further here.

[0128] The influxDB service information includes, but is not limited to, the ID of the node that created the influxDB service, the name and identifier of the influxDB service, etc.

[0129] Configure the S305 influx-proxy startup program to run the influx-proxy open-source software and start it.

[0130] Specifically, the influx-proxy startup program configures the influx-proxy open-source software in the influx-proxy container of the target node according to the influxDB service information in the target node, and starts the influx-proxy open-source software in the influx-proxy container to allocate the influxDB service in the target node to a hash ring.

[0131] Using the above method, the influxDB service on all nodes in a Swarm cluster can be assigned to different hash rings.

[0132] The configuration results for the influx-proxy open-source software can be as follows:

[0133]

[0134] The above configuration assigns the InfluxDB service named "influxDB-<node ID>" to the hash ring named "circle-1", and configures the port of the InfluxDB service named "influxDB-<node ID>" to 8080. Following this configuration method, the InfluxDB service on all nodes can be assigned to different hash rings.

[0135] It should be noted that assigning the InfluxDB service to the hash ring in this application essentially means assigning the container built on the InfluxDB service to the hash ring.

[0136] In one example, the InfluxDB container's data is mounted to the host directory via mount parameters. This ensures the persistent data is stored on the host side, preventing data loss after a container restart. The mount parameters include the mount type, source directory, and destination directory. The source directory is the host-side persistent data directory, and the destination directory is the container's persistent data directory.

[0137] Following the above method, you can create, for example Figure 4 The diagram shows two nodes: Node 1 and Node 2. Swarm is deployed on the operating systems (host OS) of both nodes. Two containers are created on each node's Swarm: an influx-proxy container and an influxDB container. The influxDB container on Node 1 is created based on the influxDB service named influxDB-Node 1, and the influxDB container on Node 2 is created based on the influxDB service named influxDB-Node 2. The persistent data directories within the influxDB containers on both nodes are mounted to the persistent data directories of the corresponding hosts on both nodes, thus storing persistent data on the host side and preventing data loss after container restarts. Figure 5 As shown, containers on the two nodes communicate via an overlay network. Clients installed on the host distribute tasks to containers on both nodes through the overlay network. Load balancing is considered during task distribution, and the load balancing strategy is executed by Swarm.

[0138] Optionally, after creating the influx-proxy and influxDB containers on the node, map the ports of the influx-proxy containers to that node or the host machine corresponding to that node, so that processes inside and outside the container can access the influx-proxy service. It should be understood that if the service can be provided solely through the network within the container, then port mapping is unnecessary.

[0139] As can be seen, the solution in this application uses Docker's native Swarm as the basic management infrastructure for the Influx cluster, which enables lightweight deployment of the cluster and reduces deployment complexity. By managing the creation of the InfluxDB container and the configuration of the Influx-proxy open-source software through a single Influx-proxy startup program, scaling up only requires adding nodes to the Influx cluster via Swarm; no other operations are needed to scale up the InfluxDB cluster, making Influx cluster scaling simple.

[0140] This application also provides a data storage method, applied to... Figures 2-3 The illustrated embodiment creates an influxDB cluster. The influxDB cluster includes at least one influx-proxy container and at least one hash ring, each hash ring including at least one influxDB container. Both the influx-proxy container and the influxDB container created in the Swarm of the target node are contained within the influxDB cluster.

[0141] The target node receives a write data request, which includes the data to be written. Based on a load balancing strategy, the target node sends the write data request to at least one influxDB container in a hash ring through at least one influx-proxy container, so that the influxDB container that receives the write data request will write the data to be written.

[0142] Specifically, the write data requests received by the target node's Swarm can originate from clients. Upon receiving a write data request, the target node's Swarm forwards it to the influx-proxy container. The influx-proxy container then forwards the received write data request to an influxDB container in each hash ring. The influxDB container then writes the data to be written from the received write data request to its directory. Ultimately, this process ensures that the data to be written is written to each hash ring.

[0143] Since the data to be written in the write request is written to any one of the influxDB containers in each hash ring, the influx-proxy container needs to perform multiple operations to transfer the write request to the influxDB container. When there are multiple influx-proxy containers, in order to balance the workload of multiple influx-proxy containers, the Swarm of the target node will transfer the write request to the appropriate influx-proxy container based on the load balancing strategy, so as to balance the load on multiple influx-proxy containers, that is, to make the difference in the number of tasks executed by multiple influx-proxy containers less than a preset threshold.

[0144] This application also provides a server cluster, which includes multiple server nodes. Figure 2 and Figure 3 In the illustrated embodiment, the target node is a server node or a virtual machine within a server node.

[0145] A server node is a device with computing and storage capabilities.

[0146] See Figure 6 , Figure 6 This is a schematic diagram of the structure of a target node provided in an embodiment of this application. Figure 6 As shown, the target node 600 includes:

[0147] Acquisition unit 601 is used to acquire the images of the influx-proxy service and the influxDB service;

[0148] Create unit 602 to create an influx-proxy container on the target node based on the image of the influx-proxy service. The target node then uses the influx-proxy container to create an influxDB container based on the image of the influxDB service. The influxDB container is used to create an influxDB cluster.

[0149] In one possible implementation, the influx-proxy container includes an influx-proxy starter. Specifically, in the aspect of creating an influxDB container on the target node based on an image of the influxDB service using the influx-proxy container, creation unit 602 is used for:

[0150] The influx-proxy launcher calls the Swarm API of the target node to create the influxDB service on the target node; based on the parameters used to create the influxDB container in the influxDB service, the influxDB container is created on the target node using the image of the influxDB service.

[0151] In one possible embodiment, the creation unit 602 is further configured to:

[0152] The influx-proxy startup program checks whether the influxDB service has been created on the target node. If it is determined that the influxDB service has not been created on the target node, the influx-proxy startup program calls the Swarm API of the target node to create the influxDB service on the target node.

[0153] In one possible implementation, the parameters used to create the InfluxDB container are passed to the Swarm via a Swarm socket channel.

[0154] In one possible implementation

[0155] The acquisition unit 601 is also used to obtain the influxDB service information in the target node through the influx-proxy startup program in the influx-proxy container.

[0156] Target node 600 also includes:

[0157] Configuration unit 603 is used to configure the influx-proxy open source software in the influxDB container of the target node according to the influxDB service information in the target node through the influx-proxy startup program, and start the influx-proxy open source software in the influx-proxy container to allocate the influxDB service in the target node to the hash ring.

[0158] In one possible implementation, the persistent data directory inside the InfluxDB container is mounted to the persistent data directory on the host side via the mount parameters of the InfluxDB container.

[0159] In one possible implementation, target node 600 also includes:

[0160] Mapping unit 604 is used to map the ports of the influx-proxy container in Swarm to the host side.

[0161] In one possible implementation, containers in an InfluxDB cluster communicate via overlay network.

[0162] It is worth noting that the specific functional implementation of target node 600 can be found in the above description. Figure 2 The description of the InfluxDB cluster construction method shown, such as the acquisition unit 601 used to execute the relevant content of S201, and the creation unit 602, configuration unit 603, and mapping unit 604 used to execute the relevant content of S202-S204, will not be repeated here. Each unit or module in the target node 600 can be individually or entirely merged into one or more other units or modules, or some of the units or modules can be further divided into multiple functionally smaller units or modules. This achieves the same operation without affecting the technical effect of the embodiments of the present invention. The above-mentioned units or modules are based on logical function division. In practical applications, the function of one unit (or module) can also be implemented by multiple units (or modules), or the function of multiple units (or modules) can be implemented by one unit (or module).

[0163] Of course, target node 600 is also used to implement, for example Figure 6 The data storage method shown will not be described further here.

[0164] Based on the description of the above method embodiments and target node embodiments, please refer to... Figure 7 This invention also provides a schematic diagram of the structure of a server node 700. This server node is used to execute the aforementioned time-series database cluster construction method or data storage method. The target node 600 can be the server node 700 or a virtual machine within the server node 700.

[0165] Figure 7 The server node 700 shown (which can specifically be a server device) includes a memory 701, a processor 702, a communication interface 703, and a bus 704. The memory 701, processor 702, and communication interface 703 are interconnected via the bus 704.

[0166] The memory 701 can be ROM, static storage device, dynamic storage device, or RAM.

[0167] The memory 701 can store programs. When the program stored in the memory 701 is executed by the processor 702, the processor 702 and the communication interface 703 are used to execute the various steps of the time-series database cluster construction method or data storage method of the embodiments of this application.

[0168] The processor 702 may be a general-purpose CPU, microprocessor, ASIC, GPU or one or more integrated circuits, used to execute relevant programs to achieve the functions required by the units in the server node 700 of this application embodiment, or to execute the time-series database cluster construction method or data storage method of this application method embodiment.

[0169] The processor 702 can also be an integrated circuit chip with signal processing capabilities. During implementation, each step of the InfluxDB cluster construction method or data storage method of this application can be completed by the integrated logic circuits in the hardware of the processor 702 or by instructions in software form. The aforementioned processor 702 can also be a general-purpose processor, DSP, ASIC, FPGA, or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in the memory 701. The processor 702 reads the information in the memory 701 and, in conjunction with its hardware, performs the functions required by the units included in the target node 600 of this application embodiment, or executes the time-series database cluster construction method or data storage method of this application method embodiment.

[0170] The communication interface 703 uses transceiver devices, such as, but not limited to, transceivers, to enable communication between the server node 700 and other devices or communication networks. For example, data can be acquired through the communication interface 703.

[0171] Bus 704 may include a pathway for transmitting information between various components of server node 700 (e.g., memory 701, processor 702, communication interface 703).

[0172] It should be noted that, although Figure 7 The server node 700 shown only illustrates the memory, processor, and communication interface. However, those skilled in the art should understand that in specific implementations, the server node 700 may also include other devices necessary for normal operation. Furthermore, depending on specific needs, those skilled in the art should understand that the server node 700 may also include hardware devices for implementing other additional functions. Moreover, those skilled in the art should understand that the server node 700 may only include the devices necessary for implementing the embodiments of this application, and may not necessarily include... Figure 7 All the devices shown.

[0173] This application also provides a chip, which includes a processor and a data interface. The processor reads instructions stored in the memory through the data interface to implement the time-series database cluster construction method or data storage method.

[0174] Optionally, as one implementation, the chip may further include a memory storing instructions, and the processor is used to execute the instructions stored in the memory. When the instructions are executed, the processor is used to execute the time-series database cluster construction method or data storage method.

[0175] This application also provides a computer-readable storage medium storing instructions that, when executed on a computer or processor, cause the computer or processor to perform one or more steps of any of the above methods.

[0176] This application also provides a computer program product containing instructions. When the computer program product is run on a computer or processor, it causes the computer or processor to perform one or more steps of any of the methods described above.

[0177] Those skilled in the art will appreciate that the functionality described in conjunction with the various illustrative logic blocks, modules, and algorithmic steps disclosed herein can be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functionality described by the various illustrative logic blocks, modules, and steps can be stored or transmitted as one or more instructions or codes on a computer-readable medium and executed by a hardware-based processing unit. The computer-readable medium may comprise a computer-readable storage medium, which corresponds to a tangible medium, such as a data storage medium, or a communication medium that includes any medium facilitating the transfer of a computer program from one place to another (e.g., based on a communication protocol). In this way, the computer-readable medium may substantially correspond to (1) a non-transitory tangible computer-readable storage medium, or (2) a communication medium, such as a signal or carrier wave. The data storage medium may be any available medium accessible by one or more computers or one or more processors to retrieve instructions, code, and / or data structures for implementing the techniques described in this application. A computer program product may comprise a computer-readable medium.

[0178] By way of example and not limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disc storage devices, magnetic disk storage devices or other magnetic storage devices, flash memory, or any other media that can be used to store desired program code in the form of instructions or data structures and is accessible by a computer. Furthermore, any connection is properly referred to as computer-readable media. For example, if instructions are transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of media. However, it should be understood that the computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other temporary media, but are specifically addressed to non-temporary tangible storage media. As used herein, disks and optical discs include compact optical discs (CDs), laser optical discs, optical discs, digital versatile optical discs (DVDs), and Blu-ray discs, where disks typically reproduce data magnetically, while optical discs reproduce data optically using lasers. The combination of the above items should also be included in the scope of computer-readable media.

[0179] Instructions can be executed by one or more processors, such as digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field-programmable arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Therefore, the term "processor" as used herein can refer to any of the foregoing structures or any other structures suitable for implementing the techniques described herein. Furthermore, in some aspects, the functionality described in the various illustrative logic blocks, modules, and steps described herein can be provided within dedicated hardware and / or software modules configured for encoding and decoding, or incorporated into combined codecs. Moreover, the techniques can be fully implemented within one or more circuit or logic elements.

[0180] The technology of this application can be implemented in a wide variety of devices or apparatuses, including wireless handheld devices, integrated circuits (ICs), or a set of ICs (e.g., chipsets). The various components, modules, or units described in this application are intended to emphasize functional aspects of the apparatus for performing the disclosed technology, but do not necessarily need to be implemented by different hardware units. In fact, as described above, the various units can be combined with suitable software and / or firmware within coded hardware units, or provided via interoperable hardware units (containing one or more processors as described above).

[0181] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the specific descriptions of the corresponding steps in the foregoing method embodiments, and will not be repeated here.

[0182] It should be understood that in the description of this application, unless otherwise stated, " / " indicates that the objects before and after are in an "or" relationship. For example, A / B can represent A or B; where A and B can be singular or plural. Furthermore, in the description of this application, unless otherwise stated, "multiple" refers to two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple. Additionally, to facilitate a clear description of the technical solutions of the embodiments of this application, the terms "first" and "second" are used in the embodiments of this application to distinguish identical or similar items with substantially the same function and effect. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and the terms "first" and "second" are not necessarily different. In this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being better or more advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a specific manner to facilitate understanding.

[0183] In the embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the division of units is merely a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. The coupling, direct coupling, or communication connection shown or discussed between each other may be indirect coupling or communication connection through some interfaces, apparatuses, or units, and may be electrical, mechanical, or other forms.

[0184] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0185] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. This computer program product includes one or more computer instructions. When these computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in or transmitted through a computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available media can be ROM, or random access memory (RAM), or magnetic media, such as floppy disks, hard disks, magnetic tapes, magnetic disks, or optical media, such as digital versatile discs (DVDs), or semiconductor media, such as solid-state disks (SSDs).

[0186] The above description is merely a specific implementation of the embodiments of this application, but the protection scope of the embodiments of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the embodiments of this application should be covered within the protection scope of the embodiments of this application. Therefore, the protection scope of the embodiments of this application should be determined by the protection scope of the claims.

Claims

1. A method for constructing a time-series database cluster, characterized in that, The method includes: The target node obtains the images of the cluster proxy service and the time-series database service; The target node creates a cluster agent container on the target node according to the image of the cluster agent service; The target node creates a time-series database container on the target node based on the image of the time-series database service through the cluster proxy container. The time-series database container is used to build the time-series database cluster. The cluster agent container includes a cluster agent startup program. The target node creates a time-series database container on the target node based on the image of the time-series database service through the cluster agent container, including: The target node creates a time-series database service by calling the application programming interface (API) of the cluster management system of the target node through the cluster agent startup program; The target node creates the time series database container on the target node using the image of the time series database service according to the parameters used to create the time series database container in the time series database service; The cluster proxy service is the influx-proxy service, the time-series database service is the influxDB service, and the target node is the management node in the Swarm cluster that is enabled in Swarm mode by the docker swarm init command. The method further includes: The target node obtains its ID via its hostname through the influx-proxy startup program within the container of the influx-proxy service. The influx-proxy startup program obtains the influxDB service information on the target node through the socket channel based on the ID of the target node. If the influxDB service information is not obtained, the influx-proxy startup program transmits the parameters used to create the influxDB service to the Swarm of the target node. The target node's Swarm creates the influxDB service based on the parameters passed to the influx-proxy startup program; The influx-proxy startup program obtains the influxDB service information of the target node; The influx-proxy startup program configures and starts the influx-proxy open-source software to allocate the influxDB service in the target node to the hash ring.

2. The method according to claim 1, characterized in that, The method further includes: The target node queries whether the time-series database service has been created on the target node through the cluster agent startup program; If it is determined that the time-series database service has not been created on the target node, the target node will call the API of the target node's cluster management system through the cluster agent startup program to create the time-series database service on the target node.

3. The method according to claim 1 or 2, characterized in that, The parameters used to create the time-series database container are passed to the cluster management system through the cluster management system's socket channel.

4. The method according to claim 1 or 2, characterized in that, The target node mounts the persistent data directory within the time-series database container to the persistent data directory on the host side using the mounting parameters of the time-series database container.

5. The method according to claim 1 or 2, characterized in that, The method further includes: The target node maps the port of the cluster agent container in the cluster management system to the host side.

6. The method according to claim 1 or 2, characterized in that, The containers in the time-series database cluster are based on overlay network communication.

7. A data storage method, characterized in that, The method is applied to a time-series database cluster, which is obtained by the method described in any one of claims 1-6. The time-series database cluster includes at least one cluster proxy container and at least one hash ring, each hash ring including at least one time-series database container. The method includes: The target node receives a write data request, the write data request including data to be written; the target node contains one of the cluster proxy containers and at least one of the time-series database containers; The target node sends the write data request to the time-series database container in the at least one hash ring through the at least one cluster proxy container based on the load balancing strategy, so that the time-series database container receiving the write data request writes the data to be written. The time-series database cluster is an InfluxDB cluster, the cluster proxy container is a container of the Influx-proxy service, and the target node is a management node in the Swarm cluster that is enabled in Swarm mode by the docker swarm init command.

8. A server cluster system, characterized in that, The server cluster system includes multiple server nodes, which are used to perform the method as described in any one of claims 1-6 or 7.

Images