A master-slave machine operation control method, device, equipment and readable storage medium

Abstract

The application relates to the technical field of distributed cluster technology, and particularly discloses a master-slave machine operation control method, device and equipment and a readable storage medium, wherein the business processing capacity of each node is tested when a distributed cluster is created, a management area is divided according to the test result, at least one master machine and one slave machine are included in one management area, and the initial business processing capacity test results of the nodes are similar; the business processing capacity of each node is monitored during the operation of the distributed cluster, real-time business processing capacity test results of the nodes are obtained, a target management area is determined, and a master-slave machine switching operation is performed on the target management area, so that the master machine of each management area is not a performance-lagging master machine in the management area, a distributed cluster master-slave machine distribution scheme based on business performance is realized, the business performance of the nodes in the distributed cluster can be fully utilized, and the overall business performance of the distributed cluster is improved.

Description

Technical Field

[0001] This invention relates to the field of distributed cluster technology, and in particular to a master-slave operation control method, apparatus, device, and readable storage medium. Background Technology

[0002] A distributed cluster is a system composed of multiple computer nodes that communicate and collaborate with each other over a network to complete a task or provide a service. Distributed clusters have advantages such as high availability, high performance, and scalability, and are therefore widely used in large-scale computing, storage, and processing scenarios.

[0003] To manage a distributed cluster effectively, one or more nodes are elected as masters and the others as slaves during cluster setup. Typically, the node with the smallest network address is chosen as the master by default, or the masters are elected through a vote among the nodes. If a master fails during cluster operation, a new master is elected, and the services from the old master are switched to the new master.

[0004] During the operation of a distributed cluster, the data processing capabilities and storage capacity of each node device will continuously change as business operations progress. If the roles of master and slave devices are not adjusted according to performance, high-performance devices will not be able to fully utilize their performance advantages, resulting in a waste of cluster resources.

[0005] Providing a distributed cluster master-slave implementation scheme based on business performance to fully leverage the performance advantages of distributed clusters is a technical problem that needs to be solved by those skilled in the art. Summary of the Invention

[0006] The purpose of this invention is to provide a master-slave operation control method, apparatus, device, and readable storage medium for implementing a distributed cluster master-slave allocation scheme based on business performance, so as to give full play to the performance advantages of each node in the distributed cluster and improve the overall business performance of the distributed cluster.

[0007] To solve the above-mentioned technical problems, the present invention provides a master-slave machine operation control method, comprising:

[0008] When creating a distributed cluster, the business processing capability of each node is tested according to the business type that each node of the distributed cluster needs to execute, and the initial business processing capability test results of each node are obtained.

[0009] Based on the initial service processing capability test results, multiple hosts are selected from each of the nodes and the management areas of each host are divided, so that each management area includes at least one host and one slave, and the initial service processing capability test results of each node in the same management area are similar.

[0010] During the operation of the distributed cluster, the business processing capabilities of each node are monitored to obtain the real-time business processing capability test results of each node;

[0011] The target management area to be subjected to master-slave switchover operation is determined based on the real-time business processing capability test results.

[0012] A master-slave switch operation is performed on the target management area to ensure that the host in each management area is not a lagging host in the management area.

[0013] In some implementations, the step of selecting multiple hosts from each node and dividing the management areas of each host based on the initial service processing capability test results, so that each management area includes at least one host and one slave, and the initial service processing capability test results of each node within the same management area are similar, includes:

[0014] Randomly select multiple nodes from each of the nodes as initial cluster centroid nodes;

[0015] The initial business processing capability test results are used to assign values, and the clustering radius between the remaining nodes and the initial cluster centroid node is calculated so that the remaining nodes are assigned to the initial management area where the initial cluster centroid node is located with the closest radius distance.

[0016] The process involves iteratively selecting a new centroid node for each of the management regions, using the initial service processing capability test result as the assignment value, calculating the cluster radius between the remaining nodes and the centroid node, and assigning the remaining nodes to the management regions where the centroid node is located with the closest radius, until each management region includes at least one host and one slave, and the initial service processing capability test results of the nodes in the same management region are similar.

[0017] In some implementations, determining the target management area to perform the master-slave switchover operation based on the real-time service processing capability test results includes:

[0018] If the real-time service processing capability test result of the host in the management area is lower than that of the slave in the management area (a first number threshold), and there is a high-performance slave whose real-time service processing capability test result is within the first number threshold and whose remaining storage space is greater than the second threshold, and there is an auxiliary slave that reaches the third number threshold, then the management area is determined to be the target management area.

[0019] In some implementations, performing a master-slave switchover operation on the target management area includes:

[0020] The host service data of the host is transferred to the auxiliary slave through the fiber optic switch, and the increment of the host service data processed by the auxiliary slave is recorded;

[0021] After transferring all the host service data of the host to the auxiliary slave, the management permissions and monitoring permissions of the host are transferred to the new host in the management area;

[0022] The new host sends control commands to each of the slave devices, and determines that the permission switch is complete after receiving response commands from all the slave devices in the management area;

[0023] After confirming the completion of the permission switch, the new host receives the host service data fed back by each of the auxiliary slaves and the incremental processing of the host service data by each of the auxiliary slaves, so as to continue processing the host service data from the latest execution time point of the host service data.

[0024] In some implementations, the distributed cluster is a distributed storage cluster;

[0025] The step of testing the business processing capabilities of each node according to the required business types of each node in the distributed cluster, and obtaining the initial business processing capability test results of each node, includes:

[0026] After each node is powered on, the business data of the fourth quantity threshold is read and written to each node.

[0027] The initial business processing capability test result of the node is determined by at least one of the following: throughput, average number of operations per second, and latency time when the node processes business data.

[0028] The monitoring of the service processing capabilities of each node, and the obtaining of real-time service processing capability test results for each node, includes:

[0029] By monitoring at least one of the throughput, average number of operations per second, and latency of the nodes when processing actual business data, the test results of the real-time business processing capabilities of each node are obtained.

[0030] In some implementations, this also includes:

[0031] For the distributed cluster, business data with priority from high to low and / or business data with large to small data volume are allocated sequentially according to the real-time business processing capability test results of the management area from high to low.

[0032] If all business data of the distributed cluster has been allocated to the management area, then the cluster-level allocation optimization of the current batch of business data ends.

[0033] If there is still remaining business data after all the management areas have been allocated, the remaining business data will continue to be allocated in descending order of the real-time business processing capability test results of the management areas, with priority from high to low and / or data volume from large to small, until all the business data of the distributed cluster is allocated to the management areas.

[0034] For the management area, data is sequentially assigned to the nodes with the highest to lowest real-time business processing capability test results for processing and disk writing, based on the priority of business data from high to low and / or the amount of business data from large to small. The processed business data is then backed up by selecting the nodes in order of the real-time business processing capability test results from high to low and the remaining storage space from large to small.

[0035] In some implementations, the nodes are interconnected via gigabit switches to transmit control data;

[0036] Each of the nodes is interconnected via a fiber optic switch to transmit service data.

[0037] In some implementations, the step of selecting multiple hosts from each node and dividing the management areas of each host based on the initial service processing capability test results, so that each management area includes at least one host and one slave, and the initial service processing capability test results of each node within the same management area are similar, includes:

[0038] Randomly select multiple nodes from each of the nodes as initial cluster centroid nodes;

[0039] The initial business processing capability test results are used to assign values, and the clustering radius between the remaining nodes and the initial cluster centroid node is calculated so that the remaining nodes are assigned to the initial management area where the initial cluster centroid node is located with the closest radius distance.

[0040] The process involves iteratively selecting a new cluster centroid node within each of the management regions, using the initial service processing capability test result as the assignment value, calculating the cluster radius between the remaining nodes and the cluster centroid node, and assigning the remaining nodes to the management regions where the cluster centroid node is located with the closest radius distance, until each management region includes at least one host and one slave, and the initial service processing capability test results of the nodes within the same management region are similar;

[0041] The step of determining the target management area to perform the master-slave switchover operation based on the real-time service processing capability test results includes:

[0042] If the real-time service processing capability test result of the host in the management area is lower than that of the slave in the management area (a first number threshold), and there is a high-performance slave among the slaves whose real-time service processing capability test result is within the first number threshold and whose remaining storage space is greater than the second threshold, and there is an auxiliary slave among the slaves that reaches the third number threshold, then the management area is determined to be the target management area.

[0043] The step of performing a master-slave switchover operation on the target management area to ensure that the host in each management area is not a lagging host in the management area includes:

[0044] The host service data of the host is transferred to the auxiliary slave through the fiber optic switch, and the increment of the host service data processed by the auxiliary slave is recorded;

[0045] After transferring all the host service data of the host to the auxiliary slave, the management permissions and monitoring permissions of the host are transferred to the new host in the management area;

[0046] The new host sends control commands to each of the slave devices, and determines that the permission switch is complete after receiving response commands from all the slave devices in the management area;

[0047] After confirming the completion of the permission switch, the new host receives the host service data fed back by each of the auxiliary slaves and the incremental processing of the host service data by each of the auxiliary slaves, so as to continue processing the host service data from the latest execution time point of the host service data.

[0048] To address the aforementioned technical problems, the present invention also provides a master-slave operation control device, comprising:

[0049] The testing unit is used to test the business processing capabilities of each node when creating a distributed cluster, based on the business types that each node of the distributed cluster needs to execute, and to obtain the initial business processing capability test results of each node.

[0050] The partitioning unit is used to select multiple hosts in each of the nodes and partition the management areas of each host according to the initial service processing capability test results, so that each management area includes at least one host and one slave, and the initial service processing capability test results of each node in the same management area are similar.

[0051] The monitoring unit is used to monitor the service processing capabilities of each node during the operation of the distributed cluster and obtain the real-time service processing capability test results of each node.

[0052] The determining unit is used to determine the target management area to be subjected to master-slave switching operation based on the real-time service processing capability test results.

[0053] The control unit is used to perform master-slave switching operations on the target management area to ensure that the host in each management area is not a lagging host in the management area.

[0054] To address the aforementioned technical problems, the present invention also provides a master-slave operation control device, comprising:

[0055] Memory, used to store computer programs;

[0056] A processor for executing the computer program, which, when executed by the processor, implements the steps of the master-slave operation control method as described in any of the above.

[0057] To address the aforementioned technical problems, the present invention also provides a readable storage medium storing a computer program thereon, wherein the computer program, when executed by a processor, implements the steps of the master-slave operation control method as described in any of the above claims.

[0058] The master-slave operation control method provided by this invention tests the business processing capabilities of each node when creating a distributed cluster, and divides management areas according to the test results, ensuring that each management area includes at least one master and one slave, and that the initial business processing capability test results of each node are similar; during the operation of the distributed cluster, the business processing capabilities of each node are monitored to obtain the real-time business processing capability test results of each node, thereby determining the target management area and performing a master-slave switch operation on the target management area, so that the host in each management area is not the host with lagging performance in that management area. This realizes a distributed cluster master-slave allocation scheme based on business performance, which can give full play to the business performance of the nodes in the distributed cluster and improve the overall business performance of the distributed cluster.

[0059] The master-slave operation control method provided by this invention uses a clustering algorithm to divide management areas, ensuring that nodes with similar initial business processing capability test results are assigned to the same management area, and then selects the one with the strongest initial business capability as the master, laying a good foundation for optimizing the master-slave operation performance of the distributed cluster.

[0060] The master-slave operation control method provided by the present invention selects a higher-performance auxiliary slave to temporarily process the master's business data when the master's performance in the management area is lagging behind, and then switches the master's management and monitoring permissions to the high-performance slave in the management area, so as to maintain the master in the management area as a high-performance master and achieve zero-latency master-slave switching.

[0061] The present invention also provides a master-slave operation control device, equipment, and readable storage medium, which have the above-mentioned features and will not be repeated here. Attached Figure Description

[0062] To more clearly illustrate the technical solutions of the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0063] Figure 1 An architecture diagram of a distributed cluster provided in an embodiment of the present invention;

[0064] Figure 2 A flowchart of a master-slave machine operation control method provided in an embodiment of the present invention;

[0065] Figure 3 This is a schematic diagram illustrating a method for dividing management regions based on a clustering algorithm, as provided in an embodiment of the present invention.

[0066] Figure 4 This is a schematic diagram of the structure of a master-slave operation control device provided in an embodiment of the present invention;

[0067] Figure 5 This is a schematic diagram of a master-slave operation control device provided in an embodiment of the present invention. Detailed Implementation

[0068] The core of this invention is to provide a master-slave operation control method, apparatus, device, and readable storage medium for implementing a distributed cluster master-slave allocation scheme based on business performance, so as to give full play to the performance advantages of each node in the distributed cluster and improve the overall business performance of the distributed cluster.

[0069] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0070] The first embodiment of the present invention will be described below.

[0071] Figure 1 This is an architecture diagram of a distributed cluster provided in an embodiment of the present invention.

[0072] To facilitate understanding, the system architecture to which this invention applies will first be introduced. The specific implementation methods provided in this invention can be applied to any type of distributed cluster, such as distributed storage clusters, distributed computing clusters, distributed training clusters, etc. A distributed cluster is a system composed of multiple computer nodes (hereinafter referred to as nodes). During operation, the nodes communicate and cooperate with each other through a network to jointly complete a task or provide a service.

[0073] like Figure 1 As shown in the embodiment of the present invention, in the distributed cluster, each node can be interconnected via a network formed by switches. When the number of network ports on a switch is insufficient, multiple switches can be cascaded to achieve interconnection between the nodes. A distributed cluster may include dozens or even hundreds of nodes, each of which can act as a master or slave.

[0074] To facilitate the decoupling of control functions and business functions in a distributed cluster, the distributed cluster provided in this embodiment of the invention can employ separate switches for transmitting control data and for transmitting business data. For example... Figure 1 As shown, the switch used for transmitting control data is a control link switching unit and is not used for transmitting business data. Therefore, it does not need to have a high transmission strategy. It can be composed of multiple S5700S-52P-LI-AC 48-port gigabit switches 102. This switch has 48 gigabit network ports, which can meet the data interaction requirements of multiple storage devices. It can be expanded to a maximum of 48 devices. At the same time, the power consumption of the device is less than 50W during normal operation, which meets the green and energy-saving requirements for building a cluster.

[0075] The switch used for transmitting service data is used to transfer service data between nodes. If there are certain requirements for the transmission rate, multiple BR-310-B-0008 24-port fiber optic switches can be used. This switch has 24 fiber optic interfaces, which can support high-speed data transmission of SAN storage devices through external FC cards. The transmission rate can reach 8Gb / s, and multiple fiber optic switches can be cascaded to form a data transmission interaction unit.

[0076] In the distributed cluster provided in this embodiment of the invention, each node is divided into different management areas, and a master and slave machine are determined in each management area to manage the master and slave operation of each management area. Compared with the traditional scheme of selecting one or more hosts to manage the entire distributed cluster, this approach is more adaptable to the management scenarios of large-scale distributed clusters and fully utilizes node performance.

[0077] Based on the above architecture, the master-slave operation control method provided by the embodiments of the present invention will be described below with reference to the accompanying drawings.

[0078] The second embodiment of the present invention will be described below.

[0079] Figure 2 A flowchart of a master-slave machine operation control method provided in an embodiment of the present invention.

[0080] like Figure 2 As shown, the master-slave operation control method provided in this embodiment of the invention includes:

[0081] S201: When creating a distributed cluster, the business processing capability of each node is tested according to the business type that each node of the distributed cluster needs to execute, and the initial business processing capability test results of each node are obtained.

[0082] S202: Based on the initial service processing capability test results, select multiple hosts in each node and divide the management areas of each host so that each management area includes at least one host and one slave, and the initial service processing capability test results of each node in the same management area are similar.

[0083] S203: During the operation of the distributed cluster, monitor the business processing capabilities of each node and obtain the real-time business processing capability test results of each node.

[0084] S204: Determine the target management area to be switched from master to slave based on the real-time business processing capability test results.

[0085] S205: Perform a master-slave switch operation on the target management area to ensure that the host in each management area is a host whose performance is lagging behind the host in the non-managed area.

[0086] In specific implementations, the master-slave operation control method provided in this embodiment of the invention can be executed on a host computer outside the distributed cluster. In other implementations, the master-slave operation control method provided in this embodiment of the invention can be deployed on all nodes of the distributed cluster, and any node can perform business processing capability monitoring tasks and cooperate to complete the master-slave switchover operation after becoming the master.

[0087] In the master-slave operation control method provided in this embodiment of the invention, each node can be interconnected via a gigabit switch to transmit control data; in addition, each node can be interconnected via a fiber optic switch to transmit service data. For specific implementation details, please refer to the description of Embodiment 1 of this invention.

[0088] For S201 and S202, since the distributed cluster includes a large number of nodes, randomly assigning hosts and slaves may lead to poor business processing, or many nodes may apply to become hosts, resulting in communication abnormalities and system crashes. Therefore, in the master-slave operation control method provided in this embodiment of the invention, after initiating the cluster establishment command, the host computer immediately evaluates the business processing capabilities of all nodes in the distributed cluster, and divides the management area and selects the host based on the evaluation results.

[0089] It should be noted that testing the business processing capabilities of each node in the distributed cluster, based on the types of business operations required by each node, focuses on testing the common business operations that the distributed cluster needs to perform. For example, if the distributed cluster is a distributed storage cluster, only the read and write capabilities of each node need to be tested. If the distributed cluster is a distributed computing cluster, then testing the data computation capabilities of each node may also be included. By having the nodes perform data preprocessing when they boot up, and collecting information such as the time and accuracy of all nodes completing business data, the initial business processing capability test results of each node are assigned values.

[0090] In the master-slave machine operation control method provided in this embodiment of the invention, if the distributed cluster is a distributed storage cluster, S201: according to the service type to be executed by each node of the distributed cluster, perform service processing capability testing on each node to obtain the initial service processing capability test results of each node, which may include:

[0091] After each node is powered on, read and write operations on the business data of each node are performed according to the fourth quantity threshold (e.g., 64GB).

[0092] The initial business processing capability test result of a node is determined by at least one of the following: throughput, average number of operations per second, and latency time when the node processes business data.

[0093] For example, each node can perform data preprocessing on a certain amount of business data (e.g., 64GB of business data) upon device startup. The time required for each device to complete processing this set of data, and the accuracy of the data processing, can be statistically analyzed to obtain a precise score. For a distributed storage cluster, the scoring method can be set such that higher throughput, more average operations per second, and lower latency result in a higher score. This score characterizes the initial business processing capability test results of each node.

[0094] For S202, management regions are divided and hosts are selected for each region based on the initial business processing capacity test results. To reduce the latency of master-slave switching during distributed cluster operation, in this embodiment of the invention, nodes with similar initial business processing capacity test results are assigned to the same management region. Simultaneously, a minimum number of nodes is determined for each management region (e.g., at least 20 nodes per management region) to ensure normal data processing capacity. The number of nodes in each management region can be increased or decreased.

[0095] During partitioning, all nodes in the distributed cluster can first be sorted according to the initial business processing capability test results, and then divided into multiple management regions based on the minimum number of nodes in each management region. The node with the best initial business processing capability test result in each management region is designated as the master. After the master is determined, the master sends a delayed system startup signal to other nodes in the management region. This ensures that other nodes will not start up and compete for the master after the master starts. The startup signal is transmitted through the control link interaction unit. The delayed signal sent in each management region carries a check bit, which can only be correctly recognized by the slave devices in this management region. This is because during the signal transmission through the switch, delayed signals may be sent to other management regions. This check bit recognition method ensures that the master-slave selection process between management regions will not interfere with each other.

[0096] After creating the distributed cluster, business data is allocated to each management area based on the business processing status of the distributed cluster. The sum or average of the initial business processing capability test results (scores) of all nodes in the management area is used as the overall initial business processing capability test result (score) of the management area. Important data services with large data volumes, such as financial data and medical data, can be allocated to management areas with strong initial business processing capability test results (high scores). Secondary data such as databases and log records are allocated to management areas with weak initial business processing capability test results (low scores) to ensure that each management area of ​​the distributed cluster can achieve maximum performance. Within the management area, business data can also be allocated according to the initial business processing capability test results (scores) of each node. Based on this, the master-slave machine operation control method provided in this embodiment of the invention may further include:

[0097] For distributed clusters, business data with priority from high to low and / or business data with large to small data volume are allocated in descending order of the real-time business processing capability test results of the management area.

[0098] If all business data of the distributed cluster has been allocated to the management area, then the cluster-level allocation optimization of the current batch of business data ends.

[0099] If there is still remaining business data after all management areas have been allocated, the remaining business data will continue to be allocated in descending order of the real-time business processing capability test results of the management areas, with priority from high to low and / or data volume from large to small, until all business data of the distributed cluster is allocated to the management areas.

[0100] For the management area, data is allocated to nodes with the highest to lowest real-time business processing capability test results for processing and disk writing, based on the priority of business data from high to low and / or the amount of business data from large to small. Nodes are then selected to back up the processed business data based on the real-time business processing capability test results from high to low and the remaining storage space from large to small.

[0101] The real-time business processing capability test results can be the initial business processing capability test results measured when creating the distributed cluster and / or the real-time business processing capability test results during the operation of the distributed cluster. This embodiment of the invention provides a two-layer optimization algorithm for business data allocation. Each time business data is allocated, it is first divided into multiple parts (Business 1, Business 2, etc.). The business data is then sequentially allocated to management areas with scores from high to low according to its importance and / or volume. If all business data can be allocated after one round of allocation, the cluster-level allocation optimization ends. If there is still unallocated business data, it is then reallocated to management areas with scores from high to low.

[0102] Building upon the first-level cluster-level allocation optimization, the second-level optimization is performed within each management region. Based on the deviation of the real-time business processing capability test results (assigned scores) of each node within the management region, nodes with higher scores are prioritized for processing and persisting business data to disk. The processed business data is then transmitted via fiber optic switches to nodes with lower scores but larger remaining storage space for data backup. Once data processing is complete, the next round of business processing can begin. All processed data is backed up in the background, ensuring optimal data processing speed within each management region.

[0103] For S203, during the operation of the distributed cluster, each node is processing business data, so the business processing capability of each node can be directly monitored. For example, for a distributed storage cluster, the read and write performance of each node is monitored. For a distributed computing cluster, the computing performance of each node is tested. Therefore, in the master-slave operation control method provided in this embodiment of the invention, if the distributed cluster is a distributed storage cluster, monitoring the business processing capability of each node in S203 to obtain the real-time business processing capability test results of each node may include: monitoring at least one of the following when the node processes actual business data: throughput, average operations per second, and latency, to obtain the real-time business processing capability test results of each node.

[0104] For S204 and S205, the real-time service processing capability test results of each node are used to determine whether a master-slave switch is needed within the management area, so as to ensure that a high-performance node is always the master in each management area.

[0105] If the real-time service processing capability test result of a host in the management area is in the lower-preset percentage of the management area, then the host is considered to be lagging behind. In this case, a slave host with a higher real-time service processing capability test result in the management area should be selected as the new host. At this time, the management area can be considered as the target management area, and a master-slave switch operation needs to be performed.

[0106] When performing a master-slave switchover operation on a target management area, the services running on the primary host can be paused first, and the management and monitoring permissions of the primary host can be transferred to the new host. However, this will cause the primary host's services to be suspended. To achieve a zero-latency master-slave switchover, the business data related to the control and management permissions of the primary host can be copied to the new host before performing the switchover of control and management permissions.

[0107] The master-slave operation control method provided in this invention performs business processing capability tests on each node when creating a distributed cluster, divides management areas based on the test results, and ensures that each management area includes at least one master and one slave with similar initial business processing capability test results for each node. During the operation of the distributed cluster, the business processing capability of each node is monitored to obtain the real-time business processing capability test results of each node, thereby determining the target management area and performing a master-slave switch operation on the target management area. This ensures that the host in each management area is not a lagging host in that management area, realizing a distributed cluster master-slave allocation scheme based on business performance. This scheme can fully utilize the business performance of nodes in the distributed cluster to improve the overall business performance of the distributed cluster.

[0108] The following describes Embodiment 3 of the present invention.

[0109] Figure 3 This is a schematic diagram illustrating a method for dividing management areas based on a clustering algorithm, as provided in an embodiment of the present invention.

[0110] Based on the above embodiments, this invention provides another scheme for dividing management areas. In the master-slave operation control method provided in this invention, S202: Based on the initial service processing capability test results, multiple masters are selected from each node and management areas are divided for each master, so that each management area includes at least one master and one slave, and the initial service processing capability test results of each node within the same management area are similar, which may include:

[0111] Randomly select multiple nodes from each node as the initial cluster centroid nodes;

[0112] The initial business processing capacity test results are used to assign values, and the clustering radius between the remaining nodes and the initial cluster centroid node is calculated so that the remaining nodes are assigned to the initial management area where the initial cluster centroid node with the closest radius distance is located.

[0113] The process iteratively selects a new centroid node for each cluster within each management region. Using the initial business processing capability test result as the assignment, it calculates the cluster radius between the remaining nodes and the centroid node, and assigns the remaining nodes to the management regions where the centroid node with the closest radius is located, until each management region includes at least one host and one slave, and the initial business processing capability test results of the nodes within the same management region are similar.

[0114] In practice, after initiating the cluster establishment command, the host computer immediately evaluates the business processing capabilities of all nodes in the distributed cluster and assigns values ​​to the initial business processing capability test results of each node. For example... Figure 3 As shown, the assigned scores of each node are plotted on the coordinate axis, and cluster analysis is performed on the assigned scores. The host computer executes the clustering number based on the data processing volume. The number of clusters is related to the data volume and the total number of nodes. As described in Embodiment 2 of this invention, the minimum number of nodes in each management area (e.g., at least 20 nodes are allocated to each management area) is used. Therefore, the clustering number k = total number of nodes / 20, which ensures that there are 20 nodes in each cluster after clustering to ensure normal data processing capacity. As the data volume increases, the number of nodes in each cluster can fluctuate around 20.

[0115] According to the K-means clustering algorithm, k devices are first randomly selected as initial centroid nodes. Then, the cluster radius between the assigned score of each node and the assigned score of the initial centroid node is calculated. The closer the radius is to the initial centroid node, the more likely the device belongs to the assigned value interval of that initial centroid node. Assuming four assigned value intervals (assignment interval 1, assignment interval 2, assignment interval 3, and assignment interval 4), the calculation is performed sequentially to obtain four initial centroid nodes and their corresponding nodes. The centroid nodes are then recalculated within each assigned value interval to obtain new centroid nodes. The cluster radius is then calculated and the nodes are partitioned again. This process is repeated until the assigned scores of all nodes in each assigned value interval are equivalent, at which point the clustering is complete.

[0116] This method of dividing management areas and determining host 301 based on clustering algorithms can ensure that the business processing capacity of nodes in a single management area is balanced. The node at the centroid is selected as the host 301 of the management area, and the other devices in the management area are slaves 302.

[0117] The master-slave operation control method provided in this embodiment of the invention, based on the above embodiments, uses a clustering algorithm to divide the management area, ensuring that nodes with similar initial business processing capability test results are assigned to the same management area, and then selects the one with the strongest initial business capability as the master, laying a good foundation for optimizing the master-slave operation performance of the distributed cluster.

[0118] The following describes Embodiment 4 of the present invention.

[0119] Based on the above embodiments, the present invention further describes the master-slave switching scheme.

[0120] To achieve zero-latency master-slave switching, in the master-slave operation control method provided in this embodiment of the invention, S204: determining the target management area to be used for master-slave switching operation based on the real-time service processing capability test results may include:

[0121] If the real-time service processing capability test result of the host in the management area is lower than that of the slave in the management area, and there is a high-performance slave whose real-time service processing capability test result is within the first threshold and whose remaining storage space is greater than the second threshold, and there is an auxiliary slave that reaches the third threshold, then the management area is determined as the target management area.

[0122] In practice, the first threshold can be 30%, meaning that when the real-time business processing capability test result of the host is lower than the top 30% in the management area, the host is considered to be lagging behind in performance.

[0123] Directly switching between master and slave servers would cause the master server's services to suspend, posing security risks to the managed area and resulting in a poor user experience. Therefore, it is necessary to determine whether the managed area has redundant slave servers to share the master server's service data during the switchover. To avoid impacting the original services of the slave servers, multiple slave servers should be selected, and their remaining storage space and computing performance should be sufficient to share the master server's service data. The third quantity threshold can be three.

[0124] In addition, the selected new host should have better performance than the old host and have enough space to store the old host's business data. The new host should meet the following requirements: the real-time business processing capability test result is within the first threshold and the remaining storage space is greater than the second threshold. For example, the slave host should be in the top 30% of the management area for real-time business processing capability test results and have sufficient remaining space.

[0125] Only after meeting the above conditions is the management area considered a target management area with master-slave switching capabilities. If the management area only has a lagging master performance but no auxiliary slave to assist the master in completing a zero-latency switch, the management area can be considered not to meet the master-slave switching conditions, and the master-slave switch will be performed only after a suitable auxiliary slave becomes available.

[0126] Based on this, S205: Performing a master-slave switch operation on the target management area may include:

[0127] The host's service data is transferred to the auxiliary slave via a fiber optic switch, and the incremental processing of host service data by the auxiliary slave is recorded.

[0128] After transferring all host business data from the host to the auxiliary slave, the host's management and monitoring permissions are transferred to the new host in the management area.

[0129] The new host sends control commands to each slave device, and confirms the completion of the permission switch after receiving response commands from all slave devices in the management area;

[0130] After confirming the completion of the permission switch, the new host receives the host business data fed back by each auxiliary slave and the incremental processing of host business data by each auxiliary slave, and continues to process host business data based on the latest execution time of the host business data.

[0131] In practical implementation, a fiber optic switch can be used to quickly transmit host service data to auxiliary slave devices. The auxiliary slave devices then begin executing host services and record incremental content generated after receiving host service data. After confirming the transfer of all host service data to the auxiliary slave devices, the task of transferring the host's management and monitoring permissions to the new host in the management area is executed. Specifically, this may further include copying the host's control data to the new host before switching management and monitoring permissions. Then, as described in Embodiment 2 of this invention, the new host sends control commands carrying a specific identifier of its management area to each slave device to inform them of the host switch. The permission switch is considered complete only after receiving response commands from all slave devices. If no response commands are received from all slave devices within a timeout period, the master-slave switch is considered a failure. After confirming the completion of the permission switch, the new host retrieves host service data and incremental data generated during the master-slave switch from each auxiliary slave device, continuing to process host data from the latest execution time of the host service data. As a result, zero-latency and seamless master-slave switching is achieved for the entire management area and for users of this part of the business data.

[0132] The master-slave operation control method provided in this embodiment of the invention selects a higher-performance auxiliary slave to temporarily process the master's business data when the master's performance in the management area is lagging behind, and then switches the master's management and monitoring permissions to the high-performance slave in the management area, so as to maintain the master in the management area as a high-performance master and achieve zero-latency master-slave switching.

[0133] The fifth embodiment of the present invention will be described below.

[0134] Based on the above embodiments, this invention provides another master-slave machine operation control method.

[0135] For a detailed description of the implementation of S201, please refer to the above embodiment.

[0136] In this embodiment of the invention, S202: Based on the initial service processing capability test results, multiple hosts are selected from each node and the management areas of each host are divided, so that each management area includes at least one host and one slave, and the initial service processing capability test results of each node in the same management area are similar, which may include:

[0137] Randomly select multiple nodes from each node as the initial cluster centroid nodes;

[0138] The initial business processing capacity test results are used to assign values, and the clustering radius between the remaining nodes and the initial cluster centroid node is calculated so that the remaining nodes are assigned to the initial management area where the initial cluster centroid node with the closest radius distance is located.

[0139] The process iteratively selects a new centroid node for each cluster within each management region. Using the initial business processing capability test result as the assignment, it calculates the cluster radius between the remaining nodes and the centroid node, and assigns the remaining nodes to the management regions where the centroid node with the closest radius is located, until each management region includes at least one host and one slave, and the initial business processing capability test results of the nodes within the same management region are similar.

[0140] For a detailed description of the implementation of S203, please refer to the above embodiment.

[0141] S204: Based on the real-time service processing capability test results, determine the target management area to perform the master-slave switchover operation, which may include:

[0142] If the real-time service processing capability test result of the host in the management area is lower than that of the slave in the management area, and there is a high-performance slave whose real-time service processing capability test result is within the first threshold and whose remaining storage space is greater than the second threshold, and there is an auxiliary slave that reaches the third threshold, then the management area is determined as the target management area.

[0143] S205: Perform a master-slave switchover operation on the target management area to ensure that the hosts in each management area are not lagging behind in performance in the non-managed area. This may include:

[0144] The host's service data is transferred to the auxiliary slave via a fiber optic switch, and the incremental processing of host service data by the auxiliary slave is recorded.

[0145] After transferring all host business data from the host to the auxiliary slave, the host's management and monitoring permissions are transferred to the new host in the management area.

[0146] The new host sends control commands to each slave device, and confirms the completion of the permission switch after receiving response commands from all slave devices in the management area;

[0147] After confirming the completion of the permission switch, the new host receives the host business data fed back by each auxiliary slave and the incremental processing of host business data by each auxiliary slave, and continues to process host business data based on the latest execution time of the host business data.

[0148] The master-slave operation control method provided by this invention tests the business processing capabilities of each node when creating a distributed cluster. Based on the test results, management regions are divided, ensuring that each management region includes at least one master and one slave, and that the initial business processing capability test results of each node are similar. During the operation of the distributed cluster, the business processing capabilities of each node are monitored to obtain the real-time business processing capability test results. This allows for the determination of target management regions, and master-slave switching operations are performed on these regions. This ensures that the host in each management region is not the lagging host within that region. This achieves a distributed cluster master-slave allocation scheme based on business performance, which can fully utilize the business performance of nodes in the distributed cluster to improve the overall business performance of the distributed cluster. Furthermore, clustering algorithms are used to divide management regions, ensuring that nodes with similar initial business processing capability test results are assigned to the same management region. Then, the node with the strongest initial business capability is selected as the master, laying a solid foundation for optimizing the master-slave operation performance of the distributed cluster. Furthermore, when the host performance in the management area lags behind, a higher-performance auxiliary slave is selected to temporarily handle the host's business data. Then, the host's management and monitoring permissions are switched to the high-performance slave in the management area, thus maintaining the host in the management area as a high-performance host and achieving zero-latency master-slave switching.

[0149] The above details various embodiments of the master-slave operation control method. Based on this, the present invention also discloses a master-slave operation control device, equipment, and readable storage medium corresponding to the above method.

[0150] The following describes Embodiment Six of the present invention.

[0151] Figure 4 This is a schematic diagram of a master-slave operation control device provided in an embodiment of the present invention.

[0152] like Figure 4 As shown, the master-slave operation control device provided in this embodiment of the invention includes:

[0153] Test unit 401 is used to test the business processing capabilities of each node when creating a distributed cluster, based on the business types that each node of the distributed cluster needs to execute, and to obtain the initial business processing capability test results of each node.

[0154] The partitioning unit 402 is used to select multiple hosts in each node and partition the management area of ​​each host according to the initial service processing capability test results, so that each management area includes at least one host and one slave, and the initial service processing capability test results of each node in the same management area are similar.

[0155] Monitoring unit 403 is used to monitor the business processing capabilities of each node during the operation of the distributed cluster and obtain the real-time business processing capability test results of each node.

[0156] Determining unit 404 is used to determine the target management area to be subjected to master-slave switchover operation based on the real-time business processing capability test results.

[0157] Control unit 405 is used to perform master-slave switching operations on the target management area to ensure that the host in each management area is a host whose performance is lagging behind the host in the non-managed area.

[0158] In some implementations, the partitioning unit 402 selects multiple hosts from each node and partitions the management areas of each host based on the initial service processing capability test results, so that each management area includes at least one host and one slave, and the initial service processing capability test results of the nodes within the same management area are similar, including:

[0159] Randomly select multiple nodes from each node as the initial cluster centroid nodes;

[0160] The initial business processing capacity test results are used to assign values, and the clustering radius between the remaining nodes and the initial cluster centroid node is calculated so that the remaining nodes are assigned to the initial management area where the initial cluster centroid node with the closest radius distance is located.

[0161] The process iteratively selects a new centroid node for each cluster within each management region. Using the initial business processing capability test result as the assignment, it calculates the cluster radius between the remaining nodes and the centroid node, and assigns the remaining nodes to the management regions where the centroid node with the closest radius is located, until each management region includes at least one host and one slave, and the initial business processing capability test results of the nodes within the same management region are similar.

[0162] In some implementations, the determination unit 404 determines the target management area to be used for master-slave switchover operations based on the results of real-time service processing capability tests, including:

[0163] If the real-time service processing capability test result of the host in the management area is lower than that of the slave in the management area, and there is a high-performance slave whose real-time service processing capability test result is within the first threshold and whose remaining storage space is greater than the second threshold, and there is an auxiliary slave that reaches the third threshold, then the management area is determined as the target management area.

[0164] In some implementations, the control unit 405 performs a master-slave switchover operation on the target management area, including:

[0165] The host's service data is transferred to the auxiliary slave via a fiber optic switch, and the incremental processing of host service data by the auxiliary slave is recorded.

[0166] After transferring all host business data from the host to the auxiliary slave, the host's management and monitoring permissions are transferred to the new host in the management area.

[0167] The new host sends control commands to each slave device, and confirms the completion of the permission switch after receiving response commands from all slave devices in the management area;

[0168] After confirming the completion of the permission switch, the new host receives the host business data fed back by each auxiliary slave and the incremental processing of host business data by each auxiliary slave, and continues to process host business data based on the latest execution time of the host business data.

[0169] In some implementations, the distributed cluster is a distributed storage cluster;

[0170] Test unit 401 performs business processing capability tests on each node according to the business types that each node in the distributed cluster needs to execute, and obtains the initial business processing capability test results of each node, including:

[0171] After each node is powered on, the business data of the fourth quantity threshold is read and written to each node.

[0172] The initial business processing capability test result of a node is determined by at least one of the following: throughput, average number of operations per second, and latency when the node processes business data.

[0173] Monitoring unit 403 monitors the service processing capabilities of each node and obtains the real-time service processing capability test results of each node, including:

[0174] The test results of each node's real-time business processing capability are obtained by monitoring at least one of the following parameters when the node processes actual business data: throughput, average number of operations per second, and latency.

[0175] In some implementations, the master-slave operation control device provided in the embodiments of the present invention may further include:

[0176] The first allocation unit is used to allocate business data from the distributed cluster in descending order of the real-time business processing capability test results of the management area, and / or in descending order of data volume. If all business data of the distributed cluster has been allocated to the management area, the cluster-level allocation optimization of the current batch of business data ends. If there is still remaining business data after all management areas have been allocated, the remaining business data in descending order of the real-time business processing capability test results of the management area, and / or in descending order of data volume, continues to be allocated until all business data of the distributed cluster has been allocated to the management area.

[0177] The second allocation unit is used to allocate the management area to nodes with high to low real-time business processing capability test results for data processing and disk writing, according to the priority of business data from high to low and / or the amount of business data from large to small. It also selects nodes to back up the processed business data according to the real-time business processing capability test results from high to low and the remaining storage space from large to small.

[0178] In some implementations, the nodes are interconnected via gigabit switches to transmit control data;

[0179] Each node is interconnected via a fiber optic switch to transmit service data.

[0180] In some implementations, the partitioning unit 402 selects multiple hosts from each node and partitions the management areas of each host based on the initial service processing capability test results, so that each management area includes at least one host and one slave, and the initial service processing capability test results of the nodes within the same management area are similar, including:

[0181] Randomly select multiple nodes from each node as the initial cluster centroid nodes;

[0182] The initial business processing capacity test results are used to assign values, and the clustering radius between the remaining nodes and the initial cluster centroid node is calculated so that the remaining nodes are assigned to the initial management area where the initial cluster centroid node with the closest radius distance is located.

[0183] The process iterates through each management region to reselect the centroid node for clustering. Using the initial business processing capability test result as the assignment, the cluster radius between the remaining nodes and the centroid node is calculated. The remaining nodes are then assigned to the management regions where the centroid nodes are closest in radius, until each management region includes at least one host and one slave, and the initial business processing capability test results of the nodes in the same management region are similar.

[0184] Based on the real-time service processing capability test results, the determination unit 404 determines the target management area to be subjected to master-slave switchover operation, including:

[0185] If the real-time service processing capability test result of the host in the management area is lower than that of the slave in the management area, and there is a high-performance slave among the slaves whose real-time service processing capability test result is within the first threshold and whose remaining storage space is greater than the second threshold, and there is an auxiliary slave among the slaves that reaches the third threshold, then the management area is determined as the target management area.

[0186] Control unit 405 performs master-slave switchover operations on the target management area to ensure that the host in each management area is a lagging host in the non-managed area, including:

[0187] The host's service data is transferred to the auxiliary slave via a fiber optic switch, and the incremental processing of host service data by the auxiliary slave is recorded.

[0188] After transferring all host business data from the host to the auxiliary slave, the host's management and monitoring permissions are transferred to the new host in the management area.

[0189] The new host sends control commands to each slave device, and confirms the completion of the permission switch after receiving response commands from all slave devices in the management area;

[0190] After confirming the completion of the permission switch, the new host receives the host business data fed back by each auxiliary slave and the incremental processing of host business data by each auxiliary slave, and continues to process host business data based on the latest execution time of the host business data.

[0191] Since the embodiments of the apparatus and the embodiments of the method correspond to each other, please refer to the description of the embodiments of the method for the embodiments of the apparatus, which will not be repeated here.

[0192] The following describes Embodiment Seven of the present invention.

[0193] Figure 5 This is a schematic diagram of a master-slave operation control device provided in an embodiment of the present invention.

[0194] like Figure 5 As shown, the master-slave operation control device provided in this embodiment of the invention includes:

[0195] Memory 510 is used to store computer program 511;

[0196] The processor 520 is configured to execute a computer program 511, which, when executed by the processor 520, implements the steps of the master-slave operation control method as described in any of the above embodiments.

[0197] The processor 520 may include one or more processing cores, such as a 3-core processor or an 8-core processor. The processor 520 may be implemented using at least one hardware form selected from Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), and Programmable Logic Array (PLA). The processor 520 may also include a main processor and a coprocessor. The main processor, also known as a Central Processing Unit (CPU), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, the processor 520 may integrate a Graphics Processing Unit (GPU), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, the processor 520 may also include an Artificial Intelligence (AI) processor, which handles computational operations related to machine learning.

[0198] The memory 510 may include one or more readable storage media, which may be non-transitory. The memory 510 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In this embodiment, the memory 510 is used to store at least the following computer program 511, wherein, after being loaded and executed by the processor 520, the computer program 511 is able to implement the relevant steps in the master-slave operation control method disclosed in any of the foregoing embodiments. In addition, the resources stored in the memory 510 may also include an operating system 512 and data 513, and the storage method may be temporary storage or permanent storage. The operating system 512 may be Windows. The data 513 may include, but is not limited to, the data involved in the above methods.

[0199] In some embodiments, the master-slave operation control device may further include a display screen 530, a power supply 540, a communication interface 550, an input / output interface 560, a sensor 570, and a communication bus 580.

[0200] Those skilled in the art will understand that Figure 5 The structure shown does not constitute a limitation on the operation control equipment of the master and slave machines, and may include more or fewer components than shown.

[0201] The master-slave operation control device provided in this embodiment of the invention includes a memory and a processor. When the processor executes the program stored in the memory, it can implement the master-slave operation control method as described above, with the same effect.

[0202] The following describes Embodiment 8 of the present invention.

[0203] It should be noted that the device and equipment embodiments described above are merely illustrative. For example, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or modules may be electrical, mechanical, or other forms. Modules described as separate components may or may not be physically separate, and components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0204] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0205] If the integrated module is implemented as a software functional module and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and executes all or part of the steps of the methods described in the various embodiments of the present invention.

[0206] Therefore, embodiments of the present invention also provide a readable storage medium storing a computer program, which, when executed by a processor, implements steps such as the master-slave operation control method.

[0207] The readable storage medium may include: USB flash drive, portable hard drive, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk, and other media that can store program code.

[0208] The computer program contained in the readable storage medium provided in this embodiment can implement the steps of the master-slave operation control method described above when executed by a processor, with the same effect as above.

[0209] The foregoing provides a detailed description of a master-slave machine operation control method, apparatus, device, and readable storage medium provided by the present invention. The various embodiments in the specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus, device, and readable storage medium disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple, and relevant parts can be referred to the method section. It should be noted that those skilled in the art can make various improvements and modifications to the present invention without departing from its principles, and these improvements and modifications also fall within the protection scope of the claims of the present invention.

[0210] It should also be noted that, in this specification, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

Claims

1. A master-slave machine operation control method, characterized in that, include: When creating a distributed cluster, the business processing capability of each node is tested according to the business type that each node of the distributed cluster needs to execute, and the initial business processing capability test results of each node are obtained. Based on the initial service processing capability test results, multiple hosts are selected from each of the nodes and the management areas of each host are divided, so that each management area includes at least one host and one slave, and the initial service processing capability test results of each node in the same management area are similar. During the operation of the distributed cluster, the business processing capabilities of each node are monitored to obtain the real-time business processing capability test results of each node; The target management area to be subjected to master-slave switchover operation is determined based on the real-time business processing capability test results. A master-slave switch operation is performed on the target management area to ensure that the host in each management area is not a lagging host in the management area; The step of determining the target management area to perform the master-slave switchover operation based on the real-time service processing capability test results includes: If the real-time service processing capability test result of the host in the management area is lower than that of the slave in the management area (a first number threshold), and there is a high-performance slave among the slaves whose real-time service processing capability test result is within the first number threshold and whose remaining storage space is greater than the second threshold, and there is an auxiliary slave among the slaves that reaches the third number threshold, then the management area is determined to be the target management area. The master-slave switch operation performed on the target management area includes: The host service data of the host is transferred to the auxiliary slave through the fiber optic switch, and the increment of the host service data processed by the auxiliary slave is recorded; After transferring all the host service data of the host to the auxiliary slave, the management permissions and monitoring permissions of the host are transferred to the new host in the management area; The new host sends control commands to each of the slave devices, and determines that the permission switch is complete after receiving response commands from all the slave devices in the management area; After confirming the completion of the permission switch, the new host receives the host service data fed back by each of the auxiliary slaves and the incremental processing of the host service data by each of the auxiliary slaves, so as to continue processing the host service data from the latest execution time point of the host service data.

2. The master-slave machine operation control method according to claim 1, characterized in that, The step of selecting multiple hosts from each node and dividing the management areas of each host based on the initial service processing capability test results, so that each management area includes at least one host and one slave, and the initial service processing capability test results of each node within the same management area are similar, includes: Randomly select multiple nodes from each of the nodes as initial cluster centroid nodes; The initial business processing capability test results are used to assign values, and the clustering radius between the remaining nodes and the initial cluster centroid node is calculated so that the remaining nodes are assigned to the initial management area where the initial cluster centroid node is located with the closest radius distance. The process involves iteratively selecting a new centroid node for each of the management regions, using the initial service processing capability test result as the assignment value, calculating the cluster radius between the remaining nodes and the centroid node, and assigning the remaining nodes to the management regions where the centroid node is located with the closest radius, until each management region includes at least one host and one slave, and the initial service processing capability test results of the nodes in the same management region are similar.

3. The master-slave machine operation control method according to claim 1, characterized in that, The distributed cluster is a distributed storage cluster; The step of testing the business processing capabilities of each node according to the required business types of each node in the distributed cluster, and obtaining the initial business processing capability test results of each node, includes: After each node is powered on, the business data of the fourth quantity threshold is read and written to each node. The initial business processing capability test result of the node is determined by at least one of the following: throughput, average number of operations per second, and latency time when the node processes business data. The monitoring of the service processing capabilities of each node, and the obtaining of real-time service processing capability test results for each node, includes: By monitoring at least one of the throughput, average number of operations per second, and latency when the node processes actual business data, the test results of the real-time business processing capability of each node are obtained.

4. The master-slave machine operation control method according to claim 1, characterized in that, Also includes: For the distributed cluster, business data with priority from high to low and / or business data with large to small data volume are allocated sequentially according to the real-time business processing capability test results of the management area from high to low. If all business data of the distributed cluster has been allocated to the management area, then the cluster-level allocation optimization of the current batch of business data ends. If there is still remaining business data after all the management areas have been allocated, the remaining business data will continue to be allocated in descending order of the real-time business processing capability test results of the management areas, with priority from high to low and / or data volume from large to small, until all the business data of the distributed cluster is allocated to the management areas. For the management area, data is sequentially assigned to the nodes with the highest to lowest real-time business processing capability test results for processing and disk writing, based on the priority of business data from high to low and / or the amount of business data from large to small. The processed business data is then backed up by selecting the nodes in order of the real-time business processing capability test results from high to low and the remaining storage space from large to small.

5. The master-slave machine operation control method according to claim 1, characterized in that, Each of the nodes is interconnected via a gigabit switch to transmit control data; Each of the nodes is interconnected via a fiber optic switch to transmit service data.

6. The master-slave machine operation control method according to claim 1, characterized in that, The step of selecting multiple hosts from each node and dividing the management areas of each host based on the initial service processing capability test results, so that each management area includes at least one host and one slave, and the initial service processing capability test results of each node within the same management area are similar, includes: Randomly select multiple nodes from each of the nodes as initial cluster centroid nodes; The initial business processing capability test results are used to assign values, and the clustering radius between the remaining nodes and the initial cluster centroid node is calculated so that the remaining nodes are assigned to the initial management area where the initial cluster centroid node is located with the closest radius distance. The process involves iteratively selecting a new cluster centroid node within each of the management regions, using the initial service processing capability test result as the assignment value, calculating the cluster radius between the remaining nodes and the cluster centroid node, and assigning the remaining nodes to the management regions where the cluster centroid node is located with the closest radius distance, until each management region includes at least one host and one slave, and the initial service processing capability test results of the nodes within the same management region are similar; The step of determining the target management area to perform the master-slave switchover operation based on the real-time service processing capability test results includes: If the real-time service processing capability test result of the host in the management area is lower than that of the slave in the management area (a first number threshold), and there is a high-performance slave among the slaves whose real-time service processing capability test result is within the first number threshold and whose remaining storage space is greater than the second threshold, and there is an auxiliary slave among the slaves that reaches the third number threshold, then the management area is determined to be the target management area. The step of performing a master-slave switchover operation on the target management area to ensure that the host in each management area is not a lagging host in the management area includes: The host service data of the host is transferred to the auxiliary slave through the fiber optic switch, and the increment of the host service data processed by the auxiliary slave is recorded; After transferring all the host service data of the host to the auxiliary slave, the management permissions and monitoring permissions of the host are transferred to the new host in the management area; The new host sends control commands to each of the slave devices, and determines that the permission switch is complete after receiving response commands from all the slave devices in the management area; After confirming the completion of the permission switch, the new host receives the host service data fed back by each of the auxiliary slaves and the incremental processing of the host service data by each of the auxiliary slaves, so as to continue processing the host service data from the latest execution time point of the host service data.

7. A master-slave machine operation control device, characterized in that, include: The testing unit is used to test the business processing capabilities of each node when creating a distributed cluster, based on the business types that each node of the distributed cluster needs to execute, and to obtain the initial business processing capability test results of each node. The partitioning unit is used to select multiple hosts in each of the nodes and partition the management areas of each host according to the initial service processing capability test results, so that each management area includes at least one host and one slave, and the initial service processing capability test results of each node in the same management area are similar. The monitoring unit is used to monitor the service processing capabilities of each node during the operation of the distributed cluster and obtain the real-time service processing capability test results of each node. The determining unit is used to determine the target management area to be subjected to master-slave switching operation based on the real-time service processing capability test results. The control unit is used to perform a master-slave switch operation on the target management area to ensure that the host in each management area is not a lagging host in the management area; The step of determining the target management area to perform the master-slave switchover operation based on the real-time service processing capability test results includes: If the real-time service processing capability test result of the host in the management area is lower than that of the slave in the management area (a first number threshold), and there is a high-performance slave among the slaves whose real-time service processing capability test result is within the first number threshold and whose remaining storage space is greater than the second threshold, and there is an auxiliary slave among the slaves that reaches the third number threshold, then the management area is determined to be the target management area. The master-slave switch operation performed on the target management area includes: The host service data of the host is transferred to the auxiliary slave through the fiber optic switch, and the increment of the host service data processed by the auxiliary slave is recorded; After transferring all the host service data of the host to the auxiliary slave, the management permissions and monitoring permissions of the host are transferred to the new host in the management area; The new host sends control commands to each of the slave devices, and determines that the permission switch is complete after receiving response commands from all the slave devices in the management area; After confirming the completion of the permission switch, the new host receives the host service data fed back by each of the auxiliary slaves and the incremental processing of the host service data by each of the auxiliary slaves, so as to continue processing the host service data from the latest execution time point of the host service data.

8. A master-slave operation control device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program, which, when executed by the processor, implements the steps of the master-slave operation control method as described in any one of claims 1 to 6.

9. A readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the master-slave operation control method as described in any one of claims 1 to 6.

Images