Database cluster implementation system and method

Abstract

The present invention belongs to the field of computer technology, and discloses a database cluster implementation system and method. The method includes: the business system sends a data processing instruction to the proxy module; the proxy module splits the data processing instruction, determines the corresponding time series database node, and sends the data processing instruction to the corresponding time series data node; the time series database node performs data processing according to the data processing instruction to obtain the corresponding data result, and caches the data result; the proxy module obtains the data result in the time series data node, and sends the data result to the business system. In the above manner, after receiving the data processing instruction of the business system, the proxy module allocates the corresponding time series database node to it, and the obtained result is fed back to the business system through the proxy module, which has high availability and scalability, and at the same time uses the proxy service of the proxy module to replace the cluster service of the existing time series database, reducing the use cost.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a database cluster implementation system and method. Background Art

[0002] Existing open source time-series databases are expensive to use, while stand-alone time-series databases have bottlenecks in storage and performance, and do not have high availability and scalability when applied to the Internet of Things industry.

[0003] The above contents are only used to assist in understanding the technical solution of the present invention and do not constitute an admission that the above contents are prior art. Summary of the invention

[0004] The main purpose of the present invention is to provide a database cluster implementation system and method, aiming to solve the technical problem that the existing time-series database in the prior art has high cost and low availability and scalability when applied.

[0005] To achieve the above-mentioned purpose, the present invention provides a database cluster implementation system, the database cluster implementation system comprising: a business system, an agent module and a time series database node;

[0006] The business system is used to send data processing instructions to the proxy module;

[0007] The proxy module is used to split the data processing instruction, determine the corresponding time series database node, and send the data processing instruction to the corresponding time series data node;

[0008] The time series database node is used to process data according to the data processing instruction to obtain corresponding data results, and cache the data results;

[0009] The proxy module is also used to obtain the data results in the time series data node and send the data results to the business system.

[0010] Optionally, the proxy module includes: an instruction splitting module and an instruction allocating module;

[0011] The instruction splitting module is used to split the data processing instruction to obtain the target tag key and target tag value corresponding to the data processing instruction;

[0012] The instruction distribution module is used to perform hash value calculation based on the target tag key and the target tag value, determine the corresponding time series database node according to the obtained hash value, and send the data processing instruction to the corresponding time series database node.

[0013] Optionally, the database cluster implementation system further includes: a node registration module;

[0014] The node registration module is used to send a status query request to the initial time series database node, and based on the corresponding response data fed back by the initial time series database node, register the time series database node through the response data.

[0015] Optionally, the node registration module includes: a request sending module, a node verification module and an information registration module;

[0016] The request sending module is used to send a status query request to the initial time series database node, so that the initial time series database node feeds back corresponding response data respectively;

[0017] The node verification module is used to determine whether the response data respectively fed back by the initial time series database nodes meet the response data of the preset judgment conditions, obtain the nodes corresponding to the response data fed back by the initial time series database nodes meeting the preset judgment conditions, and use the nodes corresponding to the response data meeting the preset judgment conditions as the time series database nodes;

[0018] The information registration module is used to obtain the target information of the time series database node and complete the registration of the time series database node based on the target information. The target information includes the address information and hash slot range corresponding to the time series database nodes.

[0019] Optionally, the proxy module further includes: a balancing server;

[0020] The balancing server is used to obtain the number of data processing instructions corresponding to each proxy module, and allocate the corresponding proxy module to the data processing instruction according to the number of data processing instructions.

[0021] Furthermore, to achieve the above-mentioned purpose, the present invention also provides a database cluster implementation method, which is applied to the database cluster implementation system as described above, and the database cluster implementation system includes: a business system, an agent module and a time series database node, and the database cluster implementation method includes:

[0022] The business system sends a data processing instruction to the agent module;

[0023] The proxy module splits the data processing instruction, determines the corresponding time series database node, and sends the data processing instruction to the corresponding time series data node;

[0024] The time series database node performs data processing according to the data processing instruction to obtain corresponding data results, and caches the data results;

[0025] The proxy module obtains the data results in the time series data node and sends the data results to the business system.

[0026] Optionally, the proxy module includes: an instruction splitting module and an instruction allocating module;

[0027] The proxy module splits the data processing instruction, determines the corresponding time series database node, and sends the data processing instruction to the corresponding time series data node, including:

[0028] The instruction splitting module splits the data processing instruction to obtain a target tag key and a target tag value corresponding to the data processing instruction;

[0029] The instruction distribution module performs hash value calculation based on the target tag key and the target tag value, determines the corresponding time series database node according to the obtained hash value, and sends the data processing instruction to the corresponding time series database node.

[0030] Optionally, the database cluster implementation system further includes: a node registration module;

[0031] The proxy module splits the data processing instruction, determines the corresponding time series database node, and sends the data processing instruction to the corresponding time series data node, and also includes:

[0032] The node registration module sends a status query request to the initial time series database node, and based on the corresponding response data fed back by the initial time series database node, registers the time series database node through the response data.

[0033] Optionally, the node registration module includes: a request sending module, a node verification module and an information registration module;

[0034] The node registration module sends a status query request to the initial time series database node, and registers the time series database node based on the corresponding response data fed back by the initial time series database node, including:

[0035] The request sending module sends a status query request to the initial time series database node, so that the initial time series database node feeds back corresponding response data respectively;

[0036] The node verification module determines whether the response data corresponding to the initial time series database nodes respectively feedback meets the response data of the preset judgment condition, obtains the node corresponding to the response data of the response data feedback by the initial time series database node that meets the preset judgment condition, and uses the node corresponding to the response data that meets the preset judgment condition as the time series database node;

[0037] The information registration module obtains target information of the time series database node, and completes registration of the time series database node based on the target information, wherein the target information includes address information and hash slot ranges corresponding to the time series database nodes.

[0038] Optionally, the proxy module further includes: a balancing server;

[0039] Before sending the data processing instruction to the corresponding time series data node, the method further includes:

[0040] The balancing server obtains the number of data processing instructions corresponding to each proxy module, and allocates the corresponding proxy module to the data processing instruction according to the number of data processing instructions.

[0041] In the present invention, the business system sends data processing instructions to the proxy module; the proxy module splits the data processing instructions, determines the corresponding time series database nodes, and sends the data processing instructions to the corresponding time series data nodes; the time series database nodes perform data processing according to the data processing instructions to obtain corresponding data results, and cache the data results; the proxy module obtains the data results in the time series data nodes, and sends the data results to the business system. Through the above method, after receiving the data processing instructions of the business system, the proxy module allocates the corresponding time series database node to it, and the results fed back by the time series database node are sorted by the proxy module and sent to the business system, which has high availability and scalability. At the same time, the proxy service of the proxy module is used to replace the cluster service of the existing time series database. The time series database nodes under the proxy module can be infinitely expanded, which solves the bottleneck problem during data storage and reduces the cost of use. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 It is a structural block diagram of the first embodiment of the database cluster implementation system of the present invention;

[0043] Figure 2 It is an overall framework diagram of an embodiment of a database cluster implementation system of the present invention;

[0044] Figure 3 It is a structural block diagram of the second embodiment of the database cluster implementation system of the present invention;

[0045] Figure 4A virtual ring of hash values ​​for a database cluster implementation system according to an embodiment of the present invention;

[0046] Figure 5 This is a flow chart of the first embodiment of the method for implementing a database cluster according to the present invention;

[0047] Figure 6 This is a flow chart of the second embodiment of the method for implementing a database cluster according to the present invention.

[0048] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0049] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0050] Reference Figure 1 , Figure 1 This is a structural block diagram of the first embodiment of the database cluster implementation system of the present invention.

[0051] In this embodiment, the database cluster implementation system includes: a business system 10, an agent module 20 and a time series database node 30; the business system 10: is used to send data processing instructions to the agent module 20; the agent module 20: is used to split the data processing instructions, determine the corresponding time series database node 30, and send the data processing instructions to the corresponding time series data node 30; the time series database node 30: is used to perform data processing according to the data processing instructions to obtain corresponding data results, and cache the data results; the agent module 20: is also used to obtain the data results in the time series data node 30, and send the data results to the business system 10.

[0052] It should be noted that the business system 10 refers to an external system with data query or data writing needs. The business system 10 does not establish a connection with the time series database node 30, namely the InfluxDB database node, but establishes a connection with the proxy module 20, namely the proxy service, in the database cluster implementation system. When the business system 10 has a request to call for query data and data writing, it will issue a write data request or a query data request to the proxy module 20. The write data request or the query data request is a data processing instruction.

[0053] It should be noted that the proxy module 20 refers to the cluster proxy service of the time series database, which supports cluster deployment, that is, the InfluxDB database cluster proxy service. The proxy module 20 is responsible for integrating the data of multiple time series database nodes 30 and providing the business system 10 with a query and write interface for the time series database.

[0054] It can be understood that after the proxy module 20 obtains the data processing instructions issued by the business system 10, the data processing instructions are split, and the target tag value TagValue and the target tag key TagKey obtained after the splitting can be used to determine on which time series database node 30, i.e., the InfluxDB database node, the data for the operation to be performed is distributed. After the time series database node 30 is determined, the data processing instructions are sent to the corresponding time series database node 30 for corresponding operations.

[0055] In a specific implementation, in order to enable all time series database nodes 30 existing in the proxy module 20 to be valid, the database cluster implementation system further includes: a node registration module; the proxy module 20 splits the data processing instructions, determines the corresponding time series database nodes, and before sending the data processing instructions to the corresponding time series data nodes, it also includes: the node registration module sends a status query request to the initial time series database node, and based on the corresponding response data fed back by the initial time series database node, registers the time series database node 30 through the response data.

[0056] It should be noted that the node registration module refers to a mechanism for registering the time series database node 30. The node registration module is used in conjunction with the open source distributed application coordination service ZooKeeper and is responsible for maintaining the status and basic information of the time series database node 30.

[0057] It can be understood that the status query request refers to the verification request sent by the node registration module to the initial time series database node through the frequency preset by the administrator. After the node registration module sends the status query request, the initial time series database node will respectively feedback the corresponding response data, and complete the registration of the time series database nodes in the initial time series database node that meet the preset judgment conditions according to the response data.

[0058] In a specific implementation, in order to be able to register a time series database node 30 that meets the preset judgment conditions, further, the node registration module includes: a request sending module, a node verification module and an information registration module; the node registration module sends a status query request to the initial time series database node, and registers the time series database node based on the corresponding response data fed back by the initial time series database node, including: the request sending module sends a status query request to the initial time series database node, so that the initial time series database node feeds back the corresponding response data respectively; the node verification module determines whether the corresponding response data fed back by the initial time series database node meets the preset judgment conditions, obtains the node corresponding to the response data fed back by the initial time series database node that meets the preset judgment conditions, and uses the node corresponding to the response data that meets the preset judgment conditions as the time series database node; the information registration module obtains the target information of the time series database node, and completes the registration of the time series database node based on the target information, and the target information includes the address information and hash slot range corresponding to the time series database nodes respectively.

[0059] It should be noted that the request sending module refers to a module in the node registration module that is used to send status query requests to the initial time series database nodes at a frequency preset by the administrator, and receive corresponding response data fed back by each initial time series database node.

[0060] It is understandable that the node verification module refers to a module in the node registration module for verifying and judging the corresponding response data fed back by each initial time series database node. The preset judgment condition refers to a pre-set condition for judging whether a time series database node can be registered.

[0061] It should be understood that the information registration module refers to a node registration module used to register time series database nodes 30 that meet preset judgment conditions.

[0062] It should be understood that the request sending module sends a status query request to the initial time series database node at a preset frequency, so that each initial time series database node feeds back corresponding response data, and judges the response data according to the preset judgment conditions to obtain the node corresponding to the response data fed back in the initial time series database node that meets the preset judgment conditions. This node is a normal node, and the hash slot range of the normal time series database node is calculated by the consistent hashing algorithm, and the target information of the node is registered with the ZooKeeper cluster in the node registration module. The target information includes the basic information and core information of the time series database node, and the core information includes the address information and hash slot range corresponding to the time series database nodes. The target information of the node corresponding to the response data fed back in the initial time series database node that does not meet the preset judgment conditions is deleted from the ZooKeeper cluster.

[0063] In a specific implementation, in order to achieve load balancing, the proxy module 20 further includes: a balancing server; before sending the data processing instruction to the corresponding time-series data node 30, it also includes: the balancing server obtains the number of data processing instructions corresponding to each proxy module, and allocates the corresponding proxy module to the data processing instruction according to the number of data processing instructions.

[0064] It should be noted that the balancing server refers to a high-performance HTTP and reverse proxy web server Nginx, and the balancing server allocates corresponding proxy modules 20 to the data processing instructions issued by the business system 10, thereby achieving load balancing.

[0065] It is understandable that if Figure 2 The figure shows the overall block diagram of the database cluster implementation system, where the proxy server is the proxy service system, i.e., the proxy module; the shard node is the registered time-series database node; the alive check service is the self-check service, i.e., the node registration module; when the business system 10 sends a data processing request, the balancing server allocates the corresponding proxy module 20 to it in the proxy module cluster, i.e., the proxy service cluster; the proxy module 20 allocates the corresponding time-series database node 30 to the data processing instruction; before allocating the time-series database node, the node registration module will register the time-series database node 30 that meets the preset judgment conditions.

[0066] It should be noted that the time series database node 30 refers to a node composed of a stand-alone version of the InfluxDB database, which is responsible for storing business data and providing a data operation interface for the proxy module. Different time series database nodes 30 are responsible for storing different data, and the stored data is determined and allocated by the proxy module 20.

[0067] It can be understood that when the data processing instruction is a request to write data, a write operation is performed on each time series database node 30 according to the correspondence between the time series database node and the list of data to be written determined in the proxy module 20, and the time series database node 30 caches the written data.

[0068] In a specific implementation, when the data processing instruction is a data query request, a query request is initiated to the corresponding time series database node 30 according to the time series database list obtained in the proxy module 20, and the time series database node 30 caches the query results.

[0069] It should be noted that the proxy module 20 obtains and aggregates the data results in each time series database node 30 and sends the data results to the business system 10 .

[0070] It can be understood that when the time series database node 30 performs a write operation, after all write operations of the time series database node 30 are completed, the proxy module 20 summarizes the cached results of each time series database node 30 and sends the cached results to the business system. When the time series database node 30 performs a query operation, after all query requests of the time series database node 30 are completed, the proxy module 20 summarizes the query results. If there are paging parameters in the query parameters, the query result list is sorted by time, and then the actual queried data is obtained according to the paging parameters, and the actual results are sent to the business system 10.

[0071] In this embodiment, the business system sends data processing instructions to the proxy module; the proxy module splits the data processing instructions, determines the corresponding time series database nodes, and sends the data processing instructions to the corresponding time series data nodes; the time series database nodes process data according to the data processing instructions to obtain corresponding data results, and cache the data results; the proxy module obtains the data results in the time series data nodes, and sends the data results to the business system. Through the above method, after receiving the data processing instructions from the business system, the proxy module allocates the corresponding time series database node to it, and the results fed back by the time series database node are sorted by the proxy module and sent to the business system, which has high availability and scalability. At the same time, the proxy service of the proxy module is used to replace the cluster service of the existing time series database. The time series database nodes under the proxy module can be infinitely expanded, which solves the bottleneck problem during data storage and reduces the cost of use.

[0072] Reference Figure 3 , Figure 3 This is a structural block diagram of the first embodiment of the database cluster implementation system of the present invention. Based on the above first embodiment, a second embodiment of the database cluster implementation system of the present invention is proposed.

[0073] In this embodiment, the proxy module 20 includes: an instruction splitting module 201 and an instruction allocating module 202 .

[0074] It should be noted that the instruction splitting module 201 refers to a module in the proxy module 20 for splitting data processing instructions.

[0075] It is understandable that when the business system calls the query and write interface, two parameters will be passed in the request header, namely the name of the target tag key TagKey and the target tag value TagValue participating in this query. There can be multiple target tag values. The instruction splitting module 201 splits the data processing instructions to obtain the target tag value and the target tag key.

[0076] It should be noted that the instruction distribution module 202 refers to a module in the proxy module for determining the corresponding time series database node 30 .

[0077] It can be understood that the instruction distribution module 202 uses consistent hashing to organize the entire hash value space into a virtual ring. After each time-series database node calculates the hash value through IP+server name, the hash value corresponding to the node will be distributed in the ring.

[0078] In a specific implementation, after the instruction distribution module 202 obtains the target tag value and the target tag key, it calculates the hash value of the target tag key TagKey + the target tag value TagValue, searches for the corresponding time series database node 30 in the virtual ring according to the obtained hash value, and sends the data processing instruction to the corresponding time series database node 30. Figure 4 The virtual ring organized by the hash value space is shown. After obtaining the hash value of the target tag key TagKey + the target tag value TagValue, the first time series database node 30 is searched in the clockwise direction with the obtained hash value as seven points. The node is the node where the target tag value sits ( Figure 4 Object B in the example is distributed on Node B. Object refers to the target tag value, and Node refers to the time series database node).

[0079] In this embodiment, the instruction splitting module splits the data processing instruction to obtain the target tag key and target tag value corresponding to the data processing instruction; the instruction allocation module calculates the hash value based on the target tag key and the target tag value, determines the corresponding time series database node according to the obtained hash value, and sends the data processing instruction to the corresponding time series database node. The corresponding time series database node is determined by the consistent hashing algorithm, which ensures that the data can be evenly distributed to multiple nodes, and the corresponding time series database node can be accurately determined.

[0080] Reference Figure 5 , Figure 5 This is a flow chart of the first embodiment of the database cluster implementation method of the present invention. The database cluster implementation method is applied to the database cluster implementation system as described above. The database cluster implementation system includes: a business system, an agent module and a time series database node. The database cluster implementation method includes:

[0081] Step S10: The business system sends a data processing instruction to the proxy module.

[0082] It should be noted that the executor of this embodiment is a database cluster implementation system. After the business system issues a data processing instruction, the proxy module splits and processes the data processing instruction, finds the corresponding time series database node for the data processing instruction to operate and cache it, and the proxy module aggregates the data obtained in the time series database node and uniformly feeds back the aggregated data results to the business system.

[0083] It can be understood that the business system refers to an external system with data query or data writing needs. The business system does not establish a connection with the time series database node, i.e., the InfluxDB database node, but establishes a connection with the proxy module, i.e., the proxy service, in the database cluster implementation system. When the business system has a request to query data and write data, it will issue a write data request or a query data request to the proxy module. The write data request or the query data request is a data processing instruction.

[0084] Step S20: The proxy module splits the data processing instruction, determines the corresponding time series database node, and sends the data processing instruction to the corresponding time series database node.

[0085] It should be noted that the proxy module refers to the cluster proxy service of the time series database, which supports cluster deployment, that is, the InfluxDB database cluster proxy service. The proxy module is responsible for integrating the data of multiple time series database nodes and providing the business system with a query and write interface for the time series database.

[0086] It can be understood that after the proxy module obtains the data processing instructions issued by the business system, the data processing instructions are split, and the target tag value TagValue and the target tag key TagKey obtained after the split can be used to determine on which time series database node, i.e., the InfluxDB database node, the data to be operated is distributed. After the time series database node is determined, the data processing instructions are sent to the corresponding time series database node for corresponding operations.

[0087] In a specific implementation, in order to enable all time series database nodes existing in the proxy module to be valid, the database cluster implementation system further includes: a node registration module; the proxy module splits the data processing instructions, determines the corresponding time series database nodes, and before sending the data processing instructions to the corresponding time series data nodes, it also includes: the node registration module sends a status query request to the initial time series database node, and based on the corresponding response data fed back by the initial time series database node, registers the time series database node through the response data.

[0088] It should be noted that the node registration module refers to a mechanism for registering time-series database nodes. The node registration module is used in conjunction with the open source distributed application coordination service ZooKeeper and is responsible for maintaining the status and basic information of time-series database nodes.

[0089] It can be understood that the status query request refers to the verification request sent by the node registration module to the initial time series database node through the frequency preset by the administrator. After the node registration module sends the status query request, the initial time series database node will respectively feedback the corresponding response data, and complete the registration of the time series database nodes in the initial time series database node that meet the preset judgment conditions according to the response data.

[0090] In a specific implementation, in order to be able to register time series database nodes that meet preset judgment conditions, further, the node registration module includes: a request sending module, a node verification module and an information registration module; the node registration module sends a status query request to the initial time series database node, and registers the time series database node based on the corresponding response data fed back by the initial time series database node, including: the request sending module sends a status query request to the initial time series database node, so that the initial time series database node feeds back corresponding response data respectively; the node verification module determines whether the corresponding response data fed back by the initial time series database node meets the preset judgment condition, obtains the node corresponding to the response data fed back by the initial time series database node that meets the preset judgment condition, and uses the node corresponding to the response data that meets the preset judgment condition as the time series database node; the information registration module obtains the target information of the time series database node, and completes the registration of the time series database node based on the target information, and the target information includes the address information and hash slot range corresponding to the time series database nodes respectively.

[0091] It should be noted that the request sending module refers to a module in the node registration module that is used to send status query requests to the initial time series database nodes at a frequency preset by the administrator, and receive corresponding response data fed back by each initial time series database node.

[0092] It is understandable that the node verification module refers to a module in the node registration module for verifying and judging the corresponding response data fed back by each initial time series database node. The preset judgment condition refers to a pre-set condition for judging whether a time series database node can be registered.

[0093] It should be understood that the information registration module refers to a node registration module used to register time series database nodes that meet preset judgment conditions.

[0094] It should be understood that the request sending module sends a status query request to the initial time series database node at a preset frequency, so that each initial time series database node feeds back corresponding response data, and judges the response data according to the preset judgment conditions to obtain the node corresponding to the response data fed back in the initial time series database node that meets the preset judgment conditions. This node is a normal node, and the hash slot range of the normal time series database node is calculated by the consistent hashing algorithm, and the target information of the node is registered with the ZooKeeper cluster in the node registration module. The target information includes the basic information and core information of the time series database node, and the core information includes the address information and hash slot range corresponding to the time series database nodes. The target information of the node corresponding to the response data fed back in the initial time series database node that does not meet the preset judgment conditions is deleted from the ZooKeeper cluster.

[0095] In a specific implementation, in order to achieve load balancing, the proxy module further includes: a balancing server; before sending the data processing instruction to the corresponding time-series data node, it also includes: the balancing server obtains the number of data processing instructions corresponding to each proxy module, and allocates the corresponding proxy module to the data processing instruction according to the number of data processing instructions.

[0096] It should be noted that the balancing server refers to a high-performance HTTP and reverse proxy web server Nginx, and the balancing server allocates corresponding proxy modules to the data processing instructions issued by the business system, thereby achieving load balancing.

[0097] Step S30: The time series database node performs data processing according to the data processing instruction to obtain corresponding data results, and caches the data results.

[0098] It should be noted that the time series database node refers to a node composed of a stand-alone version of the InfluxDB database, which is responsible for storing business data and providing a data operation interface for the proxy module. Different time series database nodes are responsible for storing different data, and the stored data is determined and allocated by the proxy module.

[0099] It can be understood that when the data processing instruction is a request to write data, a write operation is performed on each time series database node according to the correspondence between the time series database node and the list of data to be written determined in the proxy module, and the time series database node caches the written data.

[0100] In a specific implementation, when the data processing instruction is a data query request, a query request is initiated to the corresponding time series database node according to the time series database list obtained in the proxy module, and the time series database node caches the query results.

[0101] Step S40: the proxy module obtains the data results in the time series data node and sends the data results to the business system.

[0102] It should be noted that the proxy module obtains and aggregates the data results in each time series database node and sends the data results to the business system.

[0103] It can be understood that when a time series database node performs a write operation, after all write operations of the time series database node are completed, the proxy module summarizes the cached results of each time series database node and sends the cached results to the business system. When a time series database node performs a query operation, after all query requests of the time series database node are completed, the proxy module summarizes the query results. If there are paging parameters in the query parameters, the query result list is sorted by time, and then the actual queried data is obtained according to the paging parameters, and the actual results are sent to the business system.

[0104] In this embodiment, the business system sends data processing instructions to the proxy module; the proxy module splits the data processing instructions, determines the corresponding time series database nodes, and sends the data processing instructions to the corresponding time series data nodes; the time series database nodes process data according to the data processing instructions to obtain corresponding data results, and cache the data results; the proxy module obtains the data results in the time series data nodes, and sends the data results to the business system. Through the above method, after receiving the data processing instructions from the business system, the proxy module allocates the corresponding time series database node to it, and the results fed back by the time series database node are sorted by the proxy module and sent to the business system, which has high availability and scalability. At the same time, the proxy service of the proxy module is used to replace the cluster service of the existing time series database. The time series database nodes under the proxy module can be infinitely expanded, which solves the bottleneck problem during data storage and reduces the cost of use.

[0105] Figure 6 This is a flow chart of a second embodiment of a method for implementing a database cluster according to the present invention. Based on the above first embodiment, a second embodiment of a method for implementing a database cluster according to the present invention is proposed.

[0106] In this embodiment, the proxy module includes: an instruction splitting module and an instruction distribution module, and the step S20 includes:

[0107] Step S201: The instruction splitting module splits the data processing instruction to obtain a target tag key and a target tag value corresponding to the data processing instruction.

[0108] It should be noted that the instruction splitting module refers to a module in the proxy module for splitting data processing instructions.

[0109] It is understandable that when the business system calls the query and write interface, it will pass two parameters in the request header, namely the name of the target tag key TagKey and the target tag value TagValue participating in this query. There can be multiple target tag values. The instruction splitting module splits the data processing instructions to obtain the target tag value and target tag key.

[0110] Step S202: The instruction distribution module performs hash value calculation based on the target tag key and the target tag value, determines the corresponding time series database node according to the obtained hash value, and sends the data processing instruction to the corresponding time series database node.

[0111] It should be noted that the instruction distribution module refers to a module in the proxy module that is used to determine the corresponding time series database node.

[0112] It can be understood that consistent hashing is used in the instruction allocation module to organize the entire hash value space into a virtual ring. After each time-series database node calculates the hash value through IP+server name, the hash value corresponding to the node will be distributed in the ring.

[0113] In the specific implementation, after the instruction allocation module obtains the target tag value and target tag key, it calculates the hash value of the target tag key TagKey + target tag value TagValue, searches for the corresponding time series database node in the virtual ring according to the obtained hash value, and sends the data processing instruction to the corresponding time series database node.

[0114] In this embodiment, the instruction splitting module splits the data processing instruction to obtain the target tag key and target tag value corresponding to the data processing instruction; the instruction allocation module calculates the hash value based on the target tag key and the target tag value, determines the corresponding time series database node according to the obtained hash value, and sends the data processing instruction to the corresponding time series database node. The corresponding time series database node is determined by the consistent hashing algorithm, which ensures that the data can be evenly distributed to multiple nodes, and the corresponding time series database node can be accurately determined.

[0115] In addition, it should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0116] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0117] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as a read-only memory (ROM) / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0118] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A database cluster implementation system, characterized in that: The database cluster implementation system includes: a business system, an agent module and a time series database node. The time series database node refers to a node composed of a stand-alone version of the InfluxDB database, which is responsible for storing business data and providing a data operation interface to the agent module. The stored business data is determined and allocated by the agent module. The business system refers to an external system with data query or data writing requirements. The business system does not establish a connection with the time series database node, but establishes a connection with the agent module. The business system is used to send data processing instructions to the proxy module; The proxy module is used to split the data processing instruction, determine the corresponding time series database node, and send the data processing instruction to the corresponding time series data node; The time series database node is used to process data according to the data processing instruction to obtain corresponding data results, and cache the data results; The proxy module is also used to obtain the data results in the time series data node and send the data results to the business system; Wherein, the proxy module includes: an instruction splitting module and an instruction allocating module; The instruction splitting module is used to split the data processing instruction to obtain the target tag key and target tag value corresponding to the data processing instruction; The instruction distribution module is used to perform hash value calculation based on the target tag key and the target tag value, determine the corresponding time series database node according to the obtained hash value, and send the data processing instruction to the corresponding time series database node; Wherein, the proxy module further includes: a balancing server; The balancing server is used to obtain the number of data processing instructions corresponding to each proxy module, and allocate the corresponding proxy module to the data processing instruction according to the number of data processing instructions; Wherein, the database cluster implementation system further includes: a node registration module; The node registration module is used to send a status query request to the initial time series database node, and based on the corresponding response data fed back by the initial time series database node, register the time series database node through the response data. The node registration module refers to a mechanism for registering time series database nodes. The node registration module is used in combination with the open source distributed application coordination service ZooKeeper, and is responsible for maintaining the status and basic information of the time series database nodes. Wherein, the node registration module includes: a request sending module, a node verification module and an information registration module; The request sending module is used to send a status query request to the initial time series database node, so that the initial time series database node feeds back corresponding response data respectively; The node verification module is used to determine whether the response data respectively fed back by the initial time series database nodes meet the response data of the preset judgment conditions, obtain the nodes corresponding to the response data fed back by the initial time series database nodes meeting the preset judgment conditions, and use the nodes corresponding to the response data meeting the preset judgment conditions as the time series database nodes; The information registration module is used to obtain the target information of the time series database node and complete the registration of the time series database node based on the target information. The target information includes the address information and hash slot range corresponding to the time series database nodes.

2. A database cluster implementation method, characterized in that: Applied to the database cluster implementation system as claimed in claim 1, the database cluster implementation system includes: a business system, an agent module and a time series database node, the time series database node refers to a node composed of a stand-alone version of InfluxDB database, responsible for storing business data, providing a data operation interface to the agent module, the stored business data is determined and allocated by the agent module, the business system refers to an external system with data query or data writing requirements, the business system does not establish a connection with the time series database node, the business system establishes a connection with the agent module, and the database cluster implementation method includes: The business system sends a data processing instruction to the agent module; The proxy module splits the data processing instruction, determines the corresponding time series database node, and sends the data processing instruction to the corresponding time series data node; The time series database node performs data processing according to the data processing instruction to obtain corresponding data results, and caches the data results; The proxy module obtains the data results in the time series data node and sends the data results to the business system; Wherein, the proxy module includes: an instruction splitting module and an instruction allocating module; The proxy module splits the data processing instruction, determines the corresponding time series database node, and sends the data processing instruction to the corresponding time series data node, including: The instruction splitting module splits the data processing instruction to obtain a target tag key and a target tag value corresponding to the data processing instruction; The instruction distribution module performs hash value calculation based on the target tag key and the target tag value, determines the corresponding time series database node according to the obtained hash value, and sends the data processing instruction to the corresponding time series database node; Wherein, the proxy module further includes: a balancing server; Before sending the data processing instruction to the corresponding time series data node, the method further includes: The balancing server obtains the number of data processing instructions corresponding to each proxy module, and allocates the corresponding proxy module to the data processing instruction according to the number of data processing instructions.

3. The database cluster implementation method according to claim 2, characterized in that: The database cluster implementation system also includes: a node registration module; The proxy module splits the data processing instruction, determines the corresponding time series database node, and sends the data processing instruction to the corresponding time series data node, and also includes: The node registration module sends a status query request to the initial time series database node, and based on the corresponding response data fed back by the initial time series database node, registers the time series database node through the response data.

4. The database cluster implementation method according to claim 3, characterized in that: The node registration module includes: a request sending module, a node verification module and an information registration module; The node registration module sends a status query request to the initial time series database node, and registers the time series database node based on the corresponding response data fed back by the initial time series database node, including: The request sending module sends a status query request to the initial time series database node, so that the initial time series database node feeds back corresponding response data respectively; The node verification module determines whether the response data corresponding to the initial time series database nodes respectively feedback meets the response data of the preset judgment condition, obtains the node corresponding to the response data of the response data feedback by the initial time series database node that meets the preset judgment condition, and uses the node corresponding to the response data that meets the preset judgment condition as the time series database node; The information registration module obtains target information of the time series database node, and completes registration of the time series database node based on the target information, wherein the target information includes address information and hash slot ranges corresponding to the time series database nodes.

Images