Multi-data center distributed database cluster node distribution method and related equipment

Abstract

The invention discloses a distributed database cluster node distribution method for multiple data centers, and aims to solve the problems that in the prior art, when a fixed node configuration mode of a GaussDB cluster is adopted to configure multiple data center nodes, flexibility is lacked, and efficient operation and maintenance and disaster recovery functions of a database are limited. The method comprises the following steps: determining an expected number of database transactions in the multi-data center; based on the expected number of database transactions and the available processing capability of a single node of the distributed database cluster, determining a node demand number of the distributed database cluster; and distributing the distributed database cluster nodes corresponding to the node demand quantity to each data center according to the access request quantity proportion and the node demand quantity among the data centers in the multiple data centers. The invention further discloses a distributed database cluster node distribution device for the multiple data centers, electronic equipment, a computer readable storage medium and a computer program product.

Description

Technical Field

[0001] The present application relates to the field of database management technology, and in particular to a distributed database cluster node allocation method for multiple data centers and related equipment. Background Art

[0002] In high-frequency transaction scenarios in the insurance industry, there is a close dependency between data centers and databases. Data centers provide physical infrastructure and resource support for database operations, while databases can provide data storage and management services for business needs by making rational use of data center resources. This interdependent relationship ensures efficient management and stable operation of enterprise data. At the same time, the stability and availability of databases are important foundations for ensuring business continuity.

[0003] Currently, insurance companies face significant technical challenges in meeting the multi-active mode requirements for data disaster recovery, especially in scenarios using distributed database clusters, such as GaussDB clusters. Existing GaussDB clusters usually only support a few fixed node configuration modes. For example, GaussDB clusters are usually configured as a fixed three-node mode, which assigns a master node and two slave nodes to the GaussDB cluster. This fixed node configuration method lacks flexibility and cannot meet the dynamic deployment requirements of insurance companies in a multi-data center environment. For example, the number of nodes in a GaussDB cluster is usually fixed, and it is impossible to flexibly and dynamically configure it according to business growth and changes in the access pressure of the data center. Especially in the scenario of multiple data centers, it limits the efficient operation and maintenance and disaster recovery functions of the database.

[0004] Based on this, how to flexibly configure the number of distributed database cluster nodes to meet the dynamic deployment requirements in a multi-data center environment and achieve efficient operation and maintenance and disaster recovery functions of the database is a technical problem that technical personnel in this field urgently need to solve. Summary of the invention

[0005] An embodiment of the present application provides a distributed database cluster node allocation method for multiple data centers, which is used to solve the problem of lack of flexibility in configuring multiple data center nodes using the fixed node configuration mode of the GaussDB cluster in the prior art, thereby limiting the efficient operation and maintenance and disaster recovery functions of the database.

[0006] Embodiments of the present application also provide a distributed database cluster node allocation device for multiple data centers, an electronic device, a computer-readable storage medium, and a computer program product.

[0007] The present application embodiment adopts the following technical solutions: In a first aspect, the present application provides a distributed database cluster node allocation method for multiple data centers, comprising: Determine the expected volume of database transactions across multiple data centers; Determine the required number of nodes of the distributed database cluster based on the expected number of database transactions and the available processing capacity of a single node of the distributed database cluster; According to the ratio of the number of access requests between the data centers in the multi-data center and the number of node requirements, the distributed database cluster nodes corresponding to the number of node requirements are allocated to each data center according to the ratio of the number of access requests. Optionally, determine the expected number of database transactions in multiple data centers, including: Estimate the maximum and peak database processing demands of multiple data centers; Determine the expected number of database transactions in the multiple data centers based on the maximum database processing demand, the peak database processing demand, and the average number of database transactions corresponding to the database processing demand.

[0008] Optionally, based on the number of database transactions and the available processing capacity of a single node of the distributed database cluster, determine the required number of nodes of the distributed database cluster, including: Calculate the total demand for computing resources in multiple data centers based on the expected number of database transactions; Determine the number of nodes required for the distributed database cluster based on the total demand for computing resources and the available processing power of a single node.

[0009] Optionally, the method further includes: Obtain hardware configuration information of distributed database cluster nodes allocated to each data center; Determine the average database transaction execution time of the distributed database cluster nodes allocated to each data center according to the hardware configuration information; According to the average database transaction execution time of the distributed database cluster nodes, the access traffic forwarding coefficient of the distributed database cluster nodes is calculated; According to the access traffic forwarding coefficient, the access request corresponding to the access traffic forwarding coefficient is forwarded to the distributed database cluster node.

[0010] Optionally, the method further includes: Monitor key performance indicators of distributed database cluster nodes; Analyze whether the distributed database cluster nodes have failed based on key performance indicators; When a distributed database cluster node fails, access requests to be allocated to the distributed database cluster node are allocated to other distributed database cluster nodes, or the distributed database cluster node is rebuilt.

[0011] Optionally, the distributed database cluster includes a master node and a slave node, and the distributed database cluster nodes allocated to the data center include the master node and the slave node, and the method further includes: Forward access requests for write data to be allocated to a data center in multiple data centers to the master node of the data center; Forward access requests for read data to be allocated to a data center in multiple data centers to slave nodes in the data center.

[0012] In a second aspect, the present application provides a distributed database cluster node allocation device for multiple data centers, including a transaction information determination module, a node quantity determination module, and a node quantity allocation module, wherein: A transaction information determination module, for determining an expected number of database transactions in multiple data centers; A node quantity determination module, used to determine the required number of nodes of the distributed database cluster based on the expected number of database transactions and the available processing capacity of a single node of the distributed database cluster; The node quantity allocation module is used to allocate the distributed database cluster nodes corresponding to the node demand quantity to each data center according to the access request quantity ratio and the node demand quantity between the multiple data centers.

[0013] Optionally, the transaction information determination module is used to: Estimate the maximum and peak database processing demands of multiple data centers; Determine the expected number of database transactions in the multiple data centers based on the maximum database processing demand, the peak database processing demand, and the average number of database transactions corresponding to the database processing demand.

[0014] Optionally, a node quantity determination module is used to: Calculate the total demand for computing resources in multiple data centers based on the expected number of database transactions; Determine the number of nodes required for the distributed database cluster based on the total demand for computing resources and the available processing power of a single node.

[0015] Optionally, the device is also used for: Obtain hardware configuration information of distributed database cluster nodes allocated to each data center; Determine the average database transaction execution time of the distributed database cluster nodes allocated to each data center according to the hardware configuration information; According to the average database transaction execution time of the distributed database cluster nodes, the access traffic forwarding coefficient of the distributed database cluster nodes is calculated; According to the access traffic forwarding coefficient, the access request corresponding to the access traffic forwarding coefficient is forwarded to the distributed database cluster node.

[0016] Optionally, the device is also used for: Monitor key performance indicators of distributed database cluster nodes; Analyze whether the distributed database cluster nodes have failed based on key performance indicators; When a distributed database cluster node fails, access requests to be allocated to the distributed database cluster node are allocated to other distributed database cluster nodes, or the distributed database cluster node is rebuilt.

[0017] Optionally, the distributed database cluster includes a master node and a slave node, and the distributed database cluster nodes allocated to the data center include the master node and the slave node, and the device is further used to: Forward access requests for write data to be allocated to a data center in multiple data centers to the master node of the data center; Forward access requests for read data to be allocated to a data center in multiple data centers to slave nodes in the data center.

[0018] In a third aspect, the present application provides an electronic device comprising: a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein when the computer program is executed by the processor, the steps of the distributed database cluster node allocation method for multiple data centers are implemented as described above.

[0019] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the distributed database cluster node allocation method for multiple data centers as described above are implemented.

[0020] In a fifth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the distributed database cluster node allocation method for multiple data centers as described above.

[0021] At least one of the above technical solutions adopted in the embodiments of the present application can achieve the following beneficial effects: By adopting the method provided in the embodiment of the present application, the number of nodes required for the distributed database cluster can be determined based on the number of database transactions and the available processing capacity of a single node of the distributed database cluster; then, according to the ratio of the number of access requests between each data center in the multi-data center and the number of nodes required, the distributed database cluster nodes corresponding to the number of nodes required are allocated to each data center according to the ratio of the number of access requests. In this way, the number of nodes required for the distributed database cluster can be dynamically determined according to the number of database transactions in the multi-data center, and the number of distributed database cluster nodes in each data center can be dynamically adjusted according to the ratio of the number of access requests between each data center in the multi-data center, so that the distributed database cluster can adapt to the dynamic deployment requirements in the multi-data center environment, and is no longer limited to a fixed distributed database cluster node configuration mode, thereby improving the efficient operation and maintenance and disaster recovery capabilities of the database. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings: Figure 1a A schematic diagram of an implementation flow of a distributed database cluster node allocation method for multiple data centers provided in an embodiment of the present application; Figure 1b A schematic diagram of an implementation flow of a method for determining an expected number of database transactions in multiple data centers provided in an embodiment of the present application; Figure 1c A schematic diagram of an implementation flow of a method for determining an expected number of database transactions in multiple data centers provided in an embodiment of the present application; Figure 1d A schematic diagram of an implementation flow of a method for determining the number of access requests that each distributed database cluster node can process provided in an embodiment of the present application; Figure 2 A specific structural diagram of a distributed database cluster node allocation device for multiple data centers is provided for an embodiment of the present application; Figure 3 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0024] The technical solutions provided by various embodiments of the present application are described in detail below in conjunction with the accompanying drawings.

[0025] Example 1 In order to solve the problem that the fixed node configuration mode of the GaussDB cluster is used in the prior art to configure multi-data center nodes, which lacks flexibility and limits the efficient operation and maintenance and disaster recovery functions of the database, an embodiment of the present application provides a distributed database cluster node allocation method for multiple data centers.

[0026] The execution subject of the method may be various types of computing devices, or an application or application (Application, APP) installed on the computing device. The computing device may be, for example, a user terminal such as a mobile phone, a tablet computer, a smart wearable device, or a server.

[0027] For ease of description, the present application embodiment takes the execution subject of the method as an example of a server to introduce the method. Those skilled in the art can understand that the present application embodiment takes the server as an example to introduce the method, which is only an exemplary description and does not limit the scope of protection of the claims corresponding to the present solution.

[0028] Specifically, the implementation process of the method provided in the embodiment of the present application is as follows: Figure 1a As shown, the following steps are included: Step 11, determine the expected number of database transactions in multiple data centers.

[0029] Among them, multi-data center refers to multiple independent data centers distributed in different geographical locations, which are used to improve the reliability, availability and access speed of data storage. These data centers can be interconnected through the network to form a distributed data storage or processing architecture.

[0030] A database transaction is a sequence of database operations that access and possibly manipulate various data items. A database transaction consists of all database operations performed between the start and end of a transaction.

[0031] The expected number refers to the number of database transactions that are expected to be generated in multiple data centers within a certain period of time or under specific conditions.

[0032] In an embodiment of the present application, it is possible to predict how many transactions may be generated in a multi-data center environment by analyzing historical data, request patterns, or business needs of the multi-data centers.

[0033] In an optional embodiment, if Figure 1b As shown, the expected number of database transactions in multiple data centers may be determined according to the following steps 110 to 112: Step 110, estimating the maximum database processing demand and peak database processing demand of multiple data centers.

[0034] Among them, the peak database processing demand refers to the number of database transactions that the multi-data center system actually needs to process during the peak period of business traffic.

[0035] In an embodiment of the present application, when determining the peak database processing demand, the number of database transactions that need to be processed by the multi-data center system per unit time can be determined based on the total number of database transactions that need to be processed within a preset time period and the working hours; then, the peak database processing demand is determined based on information such as the number of database transactions that need to be processed by the multi-data center system per unit time and the peak period of business traffic.

[0036] For example, taking a day as an example, assuming that the number of database transactions that need to be processed by the multi-data center system per month is X, and the working hours are 22 days, then the number of database transactions that need to be processed within a unit of time (1 day) is x / 22. Secondly, assuming that most database transactions (for example, 80% of database transactions) are completed within 4 hours (that is, the peak period of business traffic is 4 hours), then the peak database processing demand can be determined as: x / 22*80% / 4.

[0037] The maximum database processing demand refers to the maximum number of database transactions that the multi-data center system needs to process per unit time.

[0038] In the embodiment of the present application, the maximum database processing demand can be determined based on the peak database processing demand. For example, based on actual work experience, it can be considered that the maximum database processing demand is equal to 2 times the peak database processing demand. Using the above example, assuming that the peak database processing demand is x / 22*80% / 4, the maximum database processing demand can be determined to be: x / 22*80% / 4*2.

[0039] Step 112, determining the expected number of database transactions in the multiple data centers according to the maximum database processing demand, the peak database processing demand, and the average number of database transactions corresponding to the database processing demand.

[0040] In an embodiment of the present application, the number of database transactions that need to be processed per minute by multiple data centers can be determined based on the maximum database processing demand and the peak database processing demand; then, the expected number of database transactions in the multiple data centers can be determined based on the number of database transactions that need to be processed per minute by the multiple data centers and the average number of database transactions corresponding to the database processing demand.

[0041] For example, continuing with the above example, assuming that the maximum database processing demand is x / 22*80% / 4*2, and the average number of database transactions corresponding to the database processing demand is y, we can first determine that the number of database transactions that need to be processed per minute in multiple data centers is x / 22*80% / 4*2 / 60, and then determine the expected number of database transactions in the multiple data centers as x / 22*80% / 4*2 / 60*y based on the number of database transactions that need to be processed per minute in multiple data centers and the average number of database transactions corresponding to the database processing demand.

[0042] In an optional implementation, considering that the expected number of database transactions will directly affect the subsequent node allocation, and the node allocation result will affect the stability and efficiency of the multi-data center during the transaction peak period, therefore, in order to ensure the stability and efficiency of the multi-data center during the transaction peak period, in the embodiment of the present application, it is necessary to ensure the accuracy of the expected number of database transactions. Based on this, in an optional implementation, such as Figure 1c As shown, the expected number of database transactions in multiple data centers may also be determined according to the following steps 114 to 116: Step 114, obtaining information such as the historical average database transaction volume in the multiple data centers, the peak volatility in the multiple data centers, the time weighting coefficient, the load feedback coefficient of the distributed database cluster nodes, etc.

[0043] The peak volatility in the multi-data center is used to characterize the relative volatility of the database processing demand during the peak period and the database processing demand during the non-peak period (in this application, the database processing demand during the non-peak period defaults to the historical average database transaction volume in the multi-data center). The peak volatility can be determined based on the ratio of the difference between the database processing demand during the peak period and the historical average database transaction volume in the multi-data center to the historical average database transaction volume in the multi-data center.

[0044] The time weighting coefficient can be set according to the fluctuation characteristics of database transactions in different time periods. For example, the time weighting coefficient during peak hours can be set to a higher value, while the time weighting coefficient during non-peak hours can be set to a lower value to improve the accuracy of time period prediction.

[0045] The load feedback coefficient of the distributed database cluster node is used to reflect the impact of the load status of the distributed database cluster node on the database transaction processing capability.

[0046] Optionally, in the embodiment of the present application, the load feedback coefficient of the distributed database cluster node can be determined according to multiple real-time performance indicators of the distributed database cluster node, such as CPU utilization, memory utilization, I / O load and network bandwidth utilization, in the following manner:

[0047] in, Indicates the load feedback coefficient of the distributed database cluster node; Indicates the weight of CPU utilization in the load feedback coefficient of the distributed database cluster node; Indicates the actual CPU utilization at time t; Indicates the maximum CPU utilization; Indicates the weight of memory usage in the load feedback coefficient of the distributed database cluster node; Indicates the actual memory usage; Indicates the maximum memory usage; Represents the weight of the I / O load in the load feedback coefficient of the distributed database cluster node; Indicates the actual I / O load; Indicates the maximum I / O load; Represents the weight of network bandwidth utilization in the load feedback coefficient of distributed database cluster nodes; Indicates the actual network bandwidth utilization; Indicates the maximum network bandwidth utilization.

[0048] Step 116, based on the acquired information, the expected number of database transactions in the multi-data center is determined by the multi-factor dynamic transaction prediction model in the following manner:

[0049] in, represents the expected number of database transactions in multiple data centers in period t; Represents the historical average database transaction volume in multiple data centers; is the peak volatility, which is used to characterize the relative fluctuation value of the database processing demand during the peak period and the database processing demand during the non-peak period; Represents the time weighting coefficient. Represents the preset adjustment factor used to control the distributed database cluster node load feedback in Weights in prediction models; Indicates the load feedback coefficient of the distributed database cluster node. It should be noted that the so-called multi-factor dynamic transaction prediction model is a transaction prediction formula that is dynamically and real-time adjusted by introducing multiple factors that affect the database transaction volume, such as the historical average transaction rate, peak volatility, time weighting coefficient, node load feedback, etc. (that is, the above calculation formula).

[0050] By adopting the method provided in the embodiment of the present application, when determining the expected number of database transactions in multiple data centers in accordance with steps 114 to 116, the multi-factor dynamic transaction prediction model can not only more accurately predict the expected number of database transactions in multiple data centers, but also ensure the efficient and stable operation of multiple data centers during transaction peak periods.

[0051] Step 12: Determine the required number of nodes of the distributed database cluster based on the expected number of database transactions and the available processing capacity of a single node of the distributed database cluster.

[0052] The available processing capacity of a single node refers to the actual computing capacity of the distributed database cluster node that can process database requests within a specific period of time. It is usually a dynamic value, which is affected by the node's hardware resources, current load, network conditions, and other resource consumption factors.

[0053] In an embodiment of the present application, based on a numerical table of computing power of the CPU of the distributed database cluster node, the number of transactions that can be processed by the CPU of the distributed database cluster node per minute can be tested and counted, thereby determining the available processing power of a single node of the distributed database cluster.

[0054] The number of nodes required for a distributed database cluster refers to the number of nodes required in a distributed database cluster to meet the expected performance, capacity, and high availability requirements of multiple data centers.

[0055] In an optional implementation, the required number of nodes of the distributed database cluster may be determined according to the following steps 120 to 122: Step 120 , calculating the total demand for computing resources required by multiple data centers based on the expected number of database transactions.

[0056] In the embodiment of the present application, the total demand for computing resources required by multiple data centers can be calculated based on the expected number of database transactions and the performance redundancy of a preset distributed database cluster.

[0057] For example, in order to ensure that the CPUs of the nodes of the distributed database cluster can fully utilize their capabilities and provide reliable guarantees, in an embodiment of the present application, it is assumed that the performance redundancy of the preset distributed database cluster is 30%, and the expected number of database transactions is U. Therefore, the total demand for computing resources required by multiple data centers can be calculated as: U*(1+30%).

[0058] Step 122: Determine the required number of nodes of the distributed database cluster according to the total required amount of computing resources and the available processing capacity of a single node.

[0059] In an embodiment of the present application, the total number of CPUs required for the nodes of the distributed database cluster can be first determined based on the ratio between the total demand for computing resources and the available processing power of a single node; then, the required number of nodes of the distributed database cluster can be determined based on the total number of CPUs required for the nodes of the distributed database cluster.

[0060] For example, using the above example, assuming that the total demand for computing resources is U*(1+30%) and the available processing capacity of a single node is z, the total number of CPUs required for the nodes of the distributed database cluster can be determined as: U*(1+30%) / z. Finally, the number of nodes required for the distributed database cluster is determined based on the total number of CPUs required for the nodes.

[0061] In an embodiment of the present application, when determining the number of nodes required for a distributed database cluster based on the total number of CPUs required for the nodes, the number of available CPU cores of a single node can be first determined based on the available processing power of a single node; and then the number of nodes required for the distributed database cluster can be determined based on the total number of CPUs required for the nodes and the number of available CPU cores of a single node.

[0062] Step 13, according to the ratio of the number of access requests and the number of node requirements among the multiple data centers, the distributed database cluster nodes corresponding to the number of node requirements are allocated to each data center according to the ratio of the number of access requests. For example, suppose there are three data centers A, B, and C, and the entire distributed database cluster requires 100 nodes to meet the total processing requirements of data centers A, B, and C. The access request ratio of each data center is as follows: Data Center A: 50% of all access requests; Data Center B: 30% of total access requests; Data Center C: 20% of total access requests; Based on this, it can be determined that the number of nodes required for data center A is: 100*50%=50; the number of nodes required for data center B is: 100*30%=30; the number of nodes required for data center C is: 100*20%=20.

[0063] By allocating nodes based on the ratio of access requests, the number of nodes in each data center is proportional to the number of access requests it carries, thereby achieving a reasonable distribution of resources, improving the overall efficiency of the system, reducing performance bottlenecks of high-load nodes, and improving response speed and service stability.

[0064] In an optional embodiment, if Figure 1d As shown, in order to reasonably utilize the resources of the distributed database cluster nodes, the number of access requests that each distributed database cluster node can process may be further determined according to the following steps 140 to 143: Step 140: Obtain hardware configuration information of the distributed database cluster nodes allocated to each data center.

[0065] Among them, the hardware configuration information includes the type of hardware device chip of the distributed database cluster node, such as IBM P780, IBM E880, Intel, Kirin chip, etc.

[0066] Step 141 : determining the average database transaction execution time of the distributed database cluster nodes allocated to each data center according to the hardware configuration information.

[0067] In the embodiment of the present application, considering that the differences in the hardware device chips of the distributed database cluster nodes will lead to differences in their transaction processing speeds, first, standard database benchmark tools (such as TPC-C, Sysbench) can be used to test the performance of each type of chip in processing database transactions, that is, the number of transactions per second (TPS) of each chip when executing database transactions.

[0068] Secondly, after obtaining the transactions per second (TPS) of each chip when executing database transactions, the unit transaction execution time of each chip can be calculated by taking the inverse method.

[0069] Finally, the average database transaction execution time of the distributed database cluster nodes can be calculated according to the chip type and the unit transaction execution time of the chip.

[0070] For example, assuming that a data center contains N nodes with IBM E880 chips and M nodes with Intel chips, the average database transaction execution time of the distributed database cluster nodes can be calculated as follows:

[0071] in, Represents the average database transaction execution time of distributed database cluster nodes; Indicates the unit transaction execution time of IBM E880 type chip; Indicates the unit transaction execution time of Intel chips.

[0072] Step 142: Calculate the access traffic forwarding coefficient of the distributed database cluster node according to the average database transaction execution time of the distributed database cluster node.

[0073] (1) Determine the hardware conversion factor of the distributed database cluster node according to the hardware configuration information of the distributed database cluster node; (2) Calculate the access traffic forwarding coefficient of the distributed database cluster node based on the hardware conversion coefficient of the distributed database cluster node and the average database transaction execution time of the distributed database cluster node.

[0074] In an optional implementation, the ratio between the hardware conversion coefficient of the distributed database cluster node and the average database transaction execution time of the distributed database cluster node may be used as the access traffic forwarding coefficient of the distributed database cluster node.

[0075] Step 143: forwarding the access request corresponding to the access traffic forwarding coefficient to the distributed database cluster node according to the access traffic forwarding coefficient.

[0076] In an optional implementation, in order to ensure the processing success rate of access requests from multiple data centers, in an embodiment of the present application, key performance indicators of distributed database cluster nodes can also be monitored; whether the distributed database cluster nodes fail is analyzed based on the key performance indicators; when a distributed database cluster node fails, the access requests to be allocated to the distributed database cluster node are allocated to other distributed database cluster nodes, or the distributed database cluster node is rebuilt.

[0077] For example, assume that there are data center A and data center B, and data center A includes 1 distributed database cluster master node and 2 distributed database cluster slave nodes, and data center B includes 2 distributed database cluster slave nodes. If the distributed database cluster master node in data center A fails, the access request to be allocated to the distributed database cluster master node in data center A can be allocated to other distributed database cluster nodes in data center A, such as the distributed database cluster slave node of data center A.

[0078] Alternatively, if a distributed database cluster slave node in data center A fails, the distributed database cluster slave node can be rebuilt. During the reconstruction of the distributed database cluster slave node, the load balancing will not forward access requests to the distributed database cluster slave node until the distributed database cluster slave node resumes service, and then forward access requests to the slave node again.

[0079] Alternatively, if both the distributed database cluster master node and the distributed database cluster slave node in data center A fail, a distributed database cluster slave node can be selected in data center B and upgraded to the distributed database cluster master node to continue providing services. Until the distributed database cluster master node in data center A resumes service, the node that was just upgraded to the distributed database cluster master node in data center B can be downgraded to a distributed database cluster slave node, while the distributed database cluster master node in data center A continues to provide services.

[0080] In an optional implementation, if the distributed database cluster includes a master node and a slave node, and the distributed database cluster nodes allocated to the data center include the master node and the slave node, then in order to ensure data consistency, the method further includes: The access request for write data to be allocated to a data center in multiple data centers is forwarded to the master node of the data center; the access request for read data to be allocated to a data center in multiple data centers is forwarded to the slave node of the data center.

[0081] In an optional implementation, in order to intuitively display the operation status of the distributed database cluster nodes, the current number of connections, data synchronization delays, database wait events, database TPS, database used capacity and other key operation and maintenance information, and to be able to handle abnormalities in the distributed database cluster nodes in a timely manner. In an embodiment of the present application, the distributed database cluster nodes can also be monitored by docking with Prometheus, and exporters can be installed on each node of the distributed database cluster to collect information, and then the monitoring information can be obtained through Prometheus and displayed.

[0082] Optionally, the monitored distributed database cluster node information may be analyzed in combination with preset indicators, rules, thresholds and other information, and faulty distributed database cluster nodes may be processed in a timely manner.

[0083] Optionally, if the master node in the distributed database cluster changes from read-write to read-only or cannot be connected, or the response rate and tps of all access requests of the node do not fluctuate within the normal range and are lower than 70% of the average value, it means that the master node can no longer provide services normally, and the master node is switched.

[0084] Optionally, if a slave node in a distributed database cluster node has the following status, rebuild the distributed database cluster slave node: 1. The node status is down; 2. The node is disconnected and unresponsive; 3. The node RTO is too long, and the synchronization status with the master node of the distributed database cluster is abnormal; 4. The synchronization relationship with the master node of the distributed database cluster is disconnected; 5. The response rate and tps of all access requests of the node do not fluctuate within the normal range and are lower than 70% of the average value.

[0085] By adopting the method provided in the embodiment of the present application, the number of nodes required for the distributed database cluster can be determined based on the number of database transactions and the available processing capacity of a single node of the distributed database cluster; then, according to the ratio of the number of access requests between each data center in the multi-data center and the number of nodes required, the distributed database cluster nodes corresponding to the number of nodes required are allocated to each data center according to the ratio of the number of access requests. In this way, the number of nodes required for the distributed database cluster can be dynamically determined according to the number of database transactions in the multi-data center, and the number of distributed database cluster nodes in each data center can be dynamically adjusted according to the ratio of the number of access requests between each data center in the multi-data center, so that the distributed database cluster can adapt to the dynamic deployment requirements in the multi-data center environment, and is no longer limited to a fixed distributed database cluster node configuration mode, thereby improving the efficient operation and maintenance and disaster recovery capabilities of the database.

[0086] Example 2 In order to solve the problem in the prior art that the target image file composed of the part of the video image that the user wants to share does not exist on the network, thus resulting in the inability to share the target image file, the embodiment of the present application provides a distributed database cluster node allocation device 20 in multiple data centers, and the specific structural diagram of the device is shown in FIG. Figure 2 As shown, it includes a transaction information determination module 21, a node quantity determination module 22 and a node quantity allocation module 23. The functions of each module are as follows: A transaction information determination module 21, used to determine the expected number of database transactions in multiple data centers; A node quantity determination module 22, configured to determine the required number of nodes of the distributed database cluster based on the expected number of database transactions and the available processing capacity of a single node of the distributed database cluster; The node quantity allocation module 23 is used to allocate the distributed database cluster nodes corresponding to the node demand quantity to each data center according to the access request quantity ratio and the node demand quantity among the multiple data centers.

[0087] Optionally, the transaction information determination module 21 is used to: Estimate the maximum and peak database processing demands of multiple data centers; Determine the expected number of database transactions in the multiple data centers based on the maximum database processing demand, the peak database processing demand, and the average number of database transactions corresponding to the database processing demand.

[0088] Optionally, the node quantity determination module 22 is used to: Calculate the total demand for computing resources in multiple data centers based on the expected number of database transactions; Determine the number of nodes required for the distributed database cluster based on the total demand for computing resources and the available processing power of a single node.

[0089] Optionally, the device is also used for: Obtain hardware configuration information of distributed database cluster nodes allocated to each data center; Determine the average database transaction execution time of the distributed database cluster nodes allocated to each data center according to the hardware configuration information; According to the average database transaction execution time of the distributed database cluster nodes, the access traffic forwarding coefficient of the distributed database cluster nodes is calculated; According to the access traffic forwarding coefficient, the access request corresponding to the access traffic forwarding coefficient is forwarded to the distributed database cluster node.

[0090] Optionally, the device is also used for: Monitor key performance indicators of distributed database cluster nodes; Analyze whether the distributed database cluster nodes have failed based on key performance indicators; When a distributed database cluster node fails, access requests to be allocated to the distributed database cluster node are allocated to other distributed database cluster nodes, or the distributed database cluster node is rebuilt.

[0091] Optionally, the distributed database cluster includes a master node and a slave node, and the distributed database cluster nodes allocated to the data center include the master node and the slave node, and the device is further used to: Forward access requests for write data to be allocated to a data center in multiple data centers to the master node of the data center; Forward access requests for read data to be allocated to a data center in multiple data centers to slave nodes in the data center.

[0092] By using the device provided in the embodiment of the present application, the number of nodes required for the distributed database cluster can be determined based on the number of database transactions and the available processing capacity of a single node of the distributed database cluster; then, according to the ratio of the number of access requests between each data center in the multi-data center and the number of nodes required, the distributed database cluster nodes corresponding to the number of nodes required are allocated to each data center according to the ratio of the number of access requests. In this way, the number of nodes required for the distributed database cluster can be dynamically determined according to the number of database transactions in the multi-data center, and the number of distributed database cluster nodes in each data center can be dynamically adjusted according to the ratio of the number of access requests between each data center in the multi-data center, so that the distributed database cluster can adapt to the dynamic deployment requirements in the multi-data center environment, and is no longer limited to a fixed distributed database cluster node configuration mode, thereby improving the efficient operation and maintenance and disaster recovery capabilities of the database.

[0093] Example 3 Figure 3 A hardware structure diagram of an electronic device for implementing various embodiments of the present application, the electronic device may include a processor 301 and a memory 302 storing computer program instructions. Specifically, the processor 301 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0094] The memory 302 may include a large capacity memory for data or instructions. By way of example and not limitation, the memory 302 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. Where appropriate, the memory 302 may include a removable or non-removable (or fixed) medium. Where appropriate, the memory 302 may be inside or outside the electronic device. In a particular embodiment, the memory 302 may be a non-volatile solid-state memory.

[0095] In one embodiment, the memory 302 may be a read-only memory (ROM). In one embodiment, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically rewritable ROM (EAROM), or a flash memory, or a combination of two or more of these.

[0096] The processor 301 implements any one of the time series data compression methods or time series data decompression methods in the above embodiments by reading and executing computer program instructions stored in the memory 302 .

[0097] In one example, the electronic device may further include a communication interface 303 and a bus 310. Figure 3 As shown, the processor 301, the memory 302, and the communication interface 303 are connected via a bus 310 and communicate with each other.

[0098] The communication interface 303 is mainly used to implement communication between various modules, devices, units and / or equipment in the embodiments of the present application.

[0099] The bus 310 includes hardware, software or both, coupling the components of the electronic device to each other. For example, but not limitation, the bus may include an accelerated graphics port (AGP) or other graphics bus, an enhanced industrial standard architecture (EISA) bus, a front-side bus (FSB), a hypertransport (HT) interconnect, an industrial standard architecture (ISA) bus, an infinite bandwidth interconnect, a low pin count (LPC) bus, a memory bus, a microchannel architecture (MCA) bus, a peripheral component interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a serial advanced technology attachment (SATA) bus, a video electronics standard association local (VLB) bus or other suitable bus or a combination of two or more of these. Where appropriate, the bus 310 may include one or more buses. Although the present application embodiment describes and shows a specific bus, the present application considers any suitable bus or interconnect.

[0100] In addition, in combination with the time series data compression method or the time series data decompression method in the above embodiments, the embodiments of the present application may provide a computer-readable storage medium for implementation. The computer-readable storage medium stores computer program instructions; when the computer program instructions are executed by the processor, any time series data compression method or time series data decompression method in the above embodiments is implemented.

[0101] It should be clear that the present application is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present application.

[0102] The above is only a specific implementation example of the present application. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the above-described system, module and unit can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. It should be understood that the protection scope of the present application is not limited to this. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application.

Claims

1. A distributed database cluster node allocation method for multiple data centers, characterized in that: include: Determine the expected volume of database transactions across multiple data centers; Determining a required number of nodes of the distributed database cluster based on the expected number of database transactions and the available processing capacity of a single node of the distributed database cluster; According to the ratio of the number of access requests between the data centers in the multiple data centers and the number of node requirements, the distributed database cluster nodes corresponding to the number of node requirements are allocated to the data centers according to the ratio of the number of access requests.

2. The method according to claim 1, characterized in that The determining an expected number of database transactions in multiple data centers includes: estimating the maximum database processing demand and peak database processing demand of the multiple data centers; An expected number of database transactions in the multiple data centers is determined based on the maximum database processing demand, the peak database processing demand, and an average number of database transactions corresponding to the database processing demand.

3. The method according to claim 1, characterized in that The determining the required number of nodes of the distributed database cluster based on the number of database transactions and the available processing capacity of a single node of the distributed database cluster includes: Calculating a total demand for computing resources required by the multiple data centers based on the expected number of database transactions; The required number of nodes of the distributed database cluster is determined according to the total required amount of the computing resources and the available processing capacity of the single node.

4. The method according to claim 1, characterized in that The method further comprises: Obtaining hardware configuration information of the distributed database cluster nodes allocated to each data center; Determine the average database transaction execution time of the distributed database cluster nodes allocated to each data center according to the hardware configuration information; Calculating the access traffic forwarding coefficient of the distributed database cluster node according to the average database transaction execution time of the distributed database cluster node; According to the access traffic forwarding coefficient, the access request corresponding to the access traffic forwarding coefficient is forwarded to the distributed database cluster node.

5. The method according to claim 1, characterized in that The method further comprises: Monitoring key performance indicators of the distributed database cluster nodes; Analyze whether a failure occurs in the distributed database cluster node according to the key performance indicators; When the distributed database cluster node fails, the access request to be allocated to the distributed database cluster node is allocated to other distributed database cluster nodes, or the distributed database cluster node is rebuilt.

6. The method according to claim 1, characterized in that The distributed database cluster includes a master node and a slave node, and the distributed database cluster nodes allocated to the data center include the master node and the slave node, and the method further includes: Forwarding access requests for write data to be allocated to the data center in the multiple data centers to the master node of the data center; Forwarding access requests for read data to be allocated to the data center in the multiple data centers to slave nodes of the data center.

7. A distributed database cluster node allocation device for multiple data centers, characterized in that: It includes a transaction information determination module, a node quantity determination module and a node quantity allocation module, wherein: A transaction information determination module, for determining an expected number of database transactions in multiple data centers; A node quantity determination module, configured to determine the required number of nodes of the distributed database cluster based on the expected number of database transactions and the available processing capacity of a single node of the distributed database cluster; The node quantity allocation module is used to allocate the distributed database cluster nodes corresponding to the node demand quantity to each data center according to the access request quantity ratio between each data center in the multiple data centers and the node demand quantity.

8. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed by the processor, the steps of the distributed database cluster node allocation method for multiple data centers as described in any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the distributed database cluster node allocation method for multiple data centers as described in any one of claims 1 to 6 are implemented.

10. A computer program product, characterized in that It comprises a computer program, which, when executed by a processor, implements the distributed database cluster node allocation method for multiple data centers as described in any one of claims 1 to 6.

Images